Finally got to read Chantal ‘s very cool work on syntactic templates in AI generated text. Sharing some findings; The pattern DT NN IN NN doesn’t exist at all in the human written text from the data I am using while it’s extremely common in LLM generations #NLProc

Knowledge updates for code LLMs: Code LLMs generate calls to libraries like numpy and pandas. What if these libraries change? Can we update LLMs with modified function definitions? Test this with our new benchmark, CodeUpdateArena. Findings: updating LLMs’ API knowledge is hard!

🎉Our cognitive reappraisal paper has been accepted by Conference on Language Modeling! We introduce the RESORT framework to instill cognitive abilities into LLMs for emotional support. Evaluation by psychology experts shows that even LLMs at the 7B scale can perform well. arxiv.org/pdf/2404.01288

Excited to share that our work has been accepted to Conference on Language Modeling ! 🎉 thanks to all reviewers! We show how LLMs can be leveraged to understand nuances in human judgment via natural language explanations! arxiv.org/pdf/2305.14770

📢Anyone who talked to me in the past year heard my rant of *LLM memorization is beyond form* & output overlap! ©️Reproducing similar series of events, or character traits also has copyright issues. 👩‍⚖️In new work we look at non-literal copying in LLMs! arxiv.org/abs/2407.07087

Very much enjoyed the talks at NSF's New Horizons workshop! My commentary here youtube.com/watch?v=0Yrith…, where I give some thoughts on how the present LLM era is continuous with 2008-2016 "Obama era psycholinguistics" and express optimism about LLMs telling us about human lang.

(1/12) Can different LLMs give you unique and novel ideas? Very likely NO! 🤖 "𝗦𝗵𝗮𝗿𝗲𝗱 𝗜𝗺𝗮𝗴𝗶𝗻𝗮𝘁𝗶𝗼𝗻: 𝗟𝗟𝗠𝘀 𝗛𝗮𝗹𝗹𝘂𝗰𝗶𝗻𝗮𝘁𝗲 𝗔𝗹𝗶𝗸𝗲" reveals: LLMs often 𝗮𝗴𝗿𝗲𝗲 on purely imaginary and hallucinated contents! Explore 🧵or full paper:

🚨 We all complain a lot about reviewers/ACs/SACs in the ML/NLP community.   But why not look at the data to see what’s going on? I found some crazy statistics about who is doing/not doing this service in the *CL community. 😱 orionweller.github.io/blog/2024/revi… 🧵

📣 New paper! 📣 Hierarchical Indexing for Retrieval-Augmented Opinion Summarization (to appear in TACL) Discrete latent variable models are scalable and attributable, but LLMs are wayyy more fluent/coherent. Can we get the best of both worlds? (yes) 👀 arxiv.org/abs/2403.00435

🤔 Want to know if your LLMs are factual? You need LLM fact-checkers. ​ 📣 Announcing the LLM-AggreFact leaderboard to rank LLM fact-checkers. ​ 📣 Want the best model? Check out BespokeLabsAI’s’ Bespoke-Minicheck-7B model, which is the current SOTA fact-checker and is cheap and

Will sadly miss #ACL2024 but Jan Trienes Sebastian Joseph Lily Chen will be in Bangkok for: 🚀 InfoLossQA, a new way to recover information loss in simplification infolossqa.ikim.nrw 🚀 FactPICO, a factuality benchmark for summarizing clinical trials arxiv.org/abs/2402.11456

Excited for the 2nd edition of Natural Language Reasoning and Structured Explanations #ACL2024NLP this Thursday August 15! Invited talks by Sherry Tongshuang Wu Karthik Narasimhan Thomas Icard Xiang Lorraine Li Plus 6 oral talks and 39 total accepted papers! See schedule below and link in🧵

Excited to be in Thailand ☀️😎 🌺🥥 to attend #ACL2024! If you run into my PhD students from Georgia Tech NLP group - Yao Dou Yao Dou, Tarek Naous Tarek Naous, and Jonathan Zheng Jonathan Zheng, please say hi! 👋 Please also say hi to my other collaborators!

Great keynote by the wonderful Barbara Plank! Loved the point about variation. Her talk was full of great insights, and the slides were so beautifully supporting the flow. Will come back to watch this one online soon! ACL 2024 kudos to PCs for the great plenary choice!

🎉SAC Awards: 2) Discursive Socratic Questioning: Evaluating the Faithfulness of Language Models’ Understanding of Discourse Relations by Miao et al. 3) Steering Llama 2 via Contrastive Activation Addition by Rimsky et al. #NLProc #ACL2024NLP

Our critical survey on Self-Correction of LLMs has been accepted to TACL! We analyze when LLMs can/cannot refine their own responses. When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs arxiv.org/abs/2406.01297 [updated!]

🧐🔡🤖 Can LMs/NNs inform CogSci? This question has been (re)visited by many people across decades. Najoung Kim 🫠 and I contribute to this debate by using NN-based LMs to generate novel experimental hypotheses which can then be tested with humans!

☀️🐂 -> 🍃 🏙️ Some news! I’ve moved to Chicago to start as a [Research [Assistant Professor]] (independent postdoc w/ PI status) at TTIC next week! But I’m feeling very bittersweet, because I spent a wonderful 12 months at UT Austin postdoc-ing with Kyle Mahowald

LLMs have been proposed for annotation tasks. But, LLMs are biased and make errors. Can we draw * valid * conclusions from LLM annotations? arxiv.org/abs/2408.15204

🚨🚨NAACL is holding an ALL-MEMBER VOTE whether to change its name from "North American Chapter" to "Nations of the Americas Chapter" of the ACL; acronym stays the same: NAACL. If you're a current NAACL member, look for an email from [email protected]. PLEASE VOTE!!!