LoRA Learns Less and Forgets Less Dan Biderman, Jacob Portes, Jose Javier Gonzalez Ortiz et al.. Action editor: Zhe Gan. openreview.net/forum?id=aloEr… #learns #regularization #rank

📢We're organizing an AI bagel gathering this Thursday morning at Madison Sq. Park in NYC. Bagels sponsored by Lux Capital DM me or Grace Isford if interested. (and yes, we're calling it bagels & lux 🥯+🤖=❤️)

In May, LLM PEFT expert Daniel Han carefully read our paper and posted his thoughts here. We took his suggestions to heart, and Daniel now appears in the Acknowledgements section of our published paper.

A new meta-analysis suggesting that on average, people don't seem to enjoy thinking. Around the world, exerting mental effort correlates strongly with negative affect. psycnet.apa.org/doiLanding?doi…

Excited to share that I will be joining UCSD CSE as an assistant professor in January 2026! I'll be recruiting PhD students from the 2024 application pool - if you're interested in anything ML Sys/efficiency/etc please reach out & put my name on your application! Until then

Some personal news: I'm thrilled to have joined @Databricks Databricks Mosaic Research as a Research Scientist last month, before I start as MIT faculty in July 2025! Expect increased investment into the open-source DSPy community, new research, & strong emphasis on production concerns 🧵.

Awesome work presenting a new paradigm for parameter-efficient finetuning in LLMs

I promise I've done other things since 2018!!! That said, neat looking paper, and cool to see my most recent work with Dan Biderman (first author) connected to my most notorious work. Just wish Databricks Slack wasn't so active every time "lottery ticket" is in a paper title.

I will keep submitting my best work to TMLR. Extremely high signal-to-noise ratio, fast turnaround, with meaningful technical feedback.

TMLR is also in need of experienced and engaged reviewers to keep the turnaround fast and SNR high! If you're interested in dedicating some time, and have publications to demonstrate experience, please reach out to me! Cog/neuro areas and explainability/interpretability are 1/2

Creative work from a strong team!

If each step of an ai agent is 95% accurate. None of the 30 step work flows will work. Going from 95-> 99.9 is a similar last mile problem as with self driving cars. Easy to hack up a prototype. Hard to make it work reliably at scale.

I discovered many important papers through Patrick Mineault. Kudos to my collaborator Matt Whiteway who’s overseeing a lot of development now. Try us out: github.com/danbider/light…

I'm delighted to announce the lab's first paper is finally out! Huge congrats to Eric and the team. rdcu.be/dRT7z

The Mamba in the Llama: Distilling and Accelerating Hybrid Models abs: arxiv.org/abs/2408.15237 code: github.com/jxiw/MambaInLl… "We demonstrate that it is feasible to distill large Transformers into linear RNNs by reusing the linear projection weights from attention layers with

Delighted to share the open access article Roy Weintraub and I wrote about the representations of, and ethical stances towards the Nakba in Israeli Jewish public history education that has been published in TRSE. tandfonline.com/doi/full/10.10…

1/ Excited to share new work with online, learning guy, Alex van Meegen, and Ashok Litwin-Kumar! "Connectivity Structure and Dynamics of Nonlinear Recurrent Neural Networks" analyzes how global, spectral structure of connectivity in real-world networks affects collective dynamics.

Watch my interview with Bianna Golodryga & Zain Asher on CNN about the hostage deal, Netanyahu’s speech, and the response in Israel to the horrifying execution of the 6 hostages by Hamas 👇🏽

I'm looking to hire a post-doc this Fall, specifically for research on Theory-of-Mind, Pragmatics, and ‘Scripted’ Behavior, please see the details below :)