Vipul Gupta (@vipul_1011) 's Twitter Profile
Vipul Gupta

@vipul_1011

Research Scientist Intern FAIR @Meta. PhD Candidate @Penn_State. BTech @IITDelhi. Past: @JohnsHopkins, @UBC. Interested in responsible AI. I don’t hallucinate

ID: 1292002106468048896

linkhttps://vipulgupta1011.github.io/ calendar_today08-08-2020 07:38:28

623 Tweet

1,1K Followers

714 Following

Vipul Gupta (@vipul_1011) 's Twitter Profile Photo
Vipul Gupta
@vipul_1011

3 months ago

🚨There is serious lack of robustness with MMLU! In our new work we find that “Changing Answer Order Can Decrease MMLU Accuracy” and the accuracy of top models can drop by 10-20%📉 This means leaderboards might not be as reliable as we thought! 📄arxiv.org/abs/2406.19470 (1/N)

🚨There is serious lack of robustness with MMLU! In our new work we find that “Changing Answer Order Can Decrease MMLU Accuracy” and the accuracy of top models can drop by 10-20%📉 This means leaderboards might not be as reliable as we thought! 📄arxiv.org/abs/2406.19470 (1/N)