At Center for Research on Foundation Models, we’ve used Levanter to help scale new techniques like: * Sophia: x.com/tengyuma/statu… * Backpacks: x.com/johnhewtt/stat… * Anticipatory Music Transformers: x.com/jwthickstun/st… (co-released today!)

🧑‍🔬LLMs for complex Chemistry reasoning!🧪 Interestingly, we found LLMs (GPT-4) have already encoded lots of ⚗️Chemistry knowledge. 🤔What is really missing is a structured process to elicit the right knowledge, and use the knowledge to perform grounded reasoning. A very

A Design Space for Intelligent and Interactive Writing Assistants #CHI2024 👩🏻✏️🤖 What writing assistants do you use? What else are out there and how do they differ? What do we need to consider when designing new writing assistants? 🔗 arxiv.org/abs/2403.14117 (1/6)

I like to talk about Levanter’s performance, reproducibility, and scalability, but it’s also portable! So portable you can even switch from TPU to GPU in the middle of a run, and then switch back again! github.com/stanford-crfm/…

David Hall has been leading the effort at Center for Research on Foundation Models on developing levanter, a production-grade framework for training foundation models that is legible, scalable, and reproducible. github.com/stanford-crfm/… Here’s why you should try it out for training your next model:

We’ve been using levanter for a number of our research projects, including training music models: x.com/jwthickstun/st…

Calling motivated students interested in pursuing MS/PhD in ML/AI, specifically privacy & generative AI! The research group I'm starting at IIT Madras has openings! Apply by *Mar 31* directly to Dept. of Data Science & AI, IIT Madras or IIT Madras CSE Dept. at research.iitm.ac.in!

Thrilled to announce the 2024 recipients of #KempnerInstitute Research Fellowships: Thomas Fel, Mikail Khona, Bingbin Liu, Isabel Papadimitriou, Noor Sajid, & Aaron Walsman! bit.ly/4aBQ6MS Thomas Fel mikail khona Bingbin Liu Isabel Papadimitriou Noor Sajid Aaron Walsman

Next Wed., April 10 join us for the MIT IDE lunch seminar with guest, Mina Lee on "Writing with Language Models" at 12pm ET. 💻Anyone can join online: bit.ly/ideseminarvirt… 📍Massachusetts Institute of Technology (MIT) + MIT IDE members join in-person: bit.ly/ideseminar410

Final Update: One more magnitude of testing Sophia. We're talking model sizes in the B's, tokens in the T's. Sophia once again wins out. For me at least this is clear evidence that Sophia may be a replacement for Adam even in large scale runs.

I'm excited to share that I will be joining MIT EECS as an assistant professor in Fall 2025! I'll be recruiting PhD students from the December 2024 application pool. Indicate interest if you'd like to work with me on NLP, IR, or ML Systems! Stay tuned for more about my new lab.

Even though I probably spent more time in the lab playing chess than doing research, for some reason they gave me a PhD 🥳🥳

Hooktheory just released ARIA, the most powerful AI MIDI composer to date. ⚡ Aria was built on top of the open-source Anticipatory Music Transformer, then fine tuned on 50k MIDI transcriptions of pop songs. #ai #aimusic audiocipher.com/post/hooktheory

Resources: Code and models: github.com/HazyResearch/p… Blogpost: hazyresearch.stanford.edu/blog/2024-07-0… Paper: arxiv.org/abs/2407.05483 It’s been so much fun to work with the team on this project: Aman Aaryan Singhal Benjamin F Spector Sabri Eyuboglu Jojo Ashish Atri and hazyresearch!! Thank you to

arxiv.org/abs/2407.08351 LM performance on existing benchmarks is highly correlated. How do we build novel benchmarks that reveal previously unknown trends? We propose AutoBencher: it casts benchmark creation as an optimization problem with a novelty term in the objective.

Musicians from the San Francisco Symphony premiered music that I co-composed with an Anticipatory Music Transformer! Here are some thoughts about the process of creating this music and the performance: crfm.stanford.edu/2024/08/06/co-…

My lab will move to NYU Data Science and NYU Courant this Fall! I’m excited to connect with amazing researchers at CILVR and larger ML/NLP community in NYC. I will be recruiting students this cycle at NYU. Happy to be back to the city 🗽on the east coast as well. I had a

During John Thickstun’s post-doc with me, he created the anticipatory music Transformer. This culminated in a violin accompaniment for Für Elise, which premiered at Carol E. Reiley’s SF Symphony SoundBox event in April. Blog post: crfm.stanford.edu/2024/08/06/co-… Some reflections: