Super excited to visit the Stanford NLP group and talk about the science of LM evaluation: How can we make inferences about LMs' latent capabilities, based on observable behaviors? Full abstract here: nlp.stanford.edu/seminar/detail… The talk is open to the public! Register below 👇🌟

💡New work! Do LLMs learn foundational concepts required to build world models? We address this question with 🌐🐨EWoK (Elements of World Knowledge)🐨🌐, a flexible cognition-inspired framework to test knowledge across physical and social domains ewok-core.github.io 🧵👇

👇To appear at COLM 2024! 🦙 Do you want to evaluate LMs' cognitive abilities? Then you should care about task demands! These often-overlooked factors threaten the validity of our evals/benchmarks, and asymmetrically affect smaller LMs. Camera-ready: bit.ly/3VMdl2u