If you are at WEAI, I will be presenting preliminary results from the Many-economists project (152 authors!) at 8:15-10am tomorrow in 702 Clearwater. #WEAI2024

I have repeated this test with both GPT-4o and Claude 3.5 Sonnet and neither improves on this performance. Claude is worse than GPT-4 at tracking boardstate.

what is the best resource for someone who knows R's data.table very well to learn how to use pydatatable? i dislike pandas so deeply

At their core, LLMs like ChatGPT are big predictive models built on tons of data. LLMs are unintuitive, but we know a lot about predictive models! What can failures in traditional stats tell us about *when* and *why* LLMs fail, even as scales increase? open.substack.com/pub/nickchk/p/…

GPT-o1 has some improved performance on this task but still in the end fails it. Details: bsky.app/profile/nickch…