I'm hiring experienced ML/NLP researchers at Anthropic this summer!

Our paper on Backpacks has won an Outstanding Paper Award at ACL 2023!

If you're excited about both fascinating learned structure in language models, and designing architectures to enable interpretability while maintaining expressivity, take a read!

backpackmodels.science

How can we build a modular and compositional system that understands 3D scenes? Excited to introduce our #CVPR2024 paper — NS3D: Neuro-Symbolic Grounding of 3D Objects and Relations, w Jiayuan Mao and Jiajun Wu . Check out our poster next week at Tue-AM-249.

How can agents like LLMs become decision-making partners for humans?

💬 Excited to share a new paper + suite of envs for 𝘥𝘦𝘤𝘪𝘴𝘪𝘰𝘯-𝘰𝘳𝘪𝘦𝘯𝘵𝘦𝘥 𝘥𝘪𝘢𝘭𝘰𝘨𝘶𝘦𝘴, where agents + humans collab to solve hard everyday problems. [1/n]

Site: collaborative-dialogue.github.io

We are releasing AlpacaFarm, a simulator enabling everyone to run and study the full RLHF pipeline at a fraction of the time (<24h) and cost (<$200) w/ LLM-simulated annotators. Starting w/ Alpaca, we show RLHF gives big 10+% winrate gains vs davinci003 (crfm.stanford.edu/2023/05/22/alp…)

New work comparing computational models of teaching via *language* or *demonstrations* with Dhara Yu and noahdgoodman!

Happy to share that our paper on teaching generative models how to spell is accepted at #ACL2023 ACL 2023

Simply by making models (both language-only and text-to-image) character-aware, the notorious spelling errors are fixed!
arxiv.org/abs/2212.10562

Introducing 100K Context Windows! We’ve expanded Claude’s context window to 100,000 tokens of text, corresponding to around 75K words. Submit hundreds of pages of materials for Claude to digest and analyze. Conversations with Claude can go on for hours or days.

Gist model checkpoints are now up on Hugging Face. Give it a try and see what prompts you can (or can't) compress!

LLaMA-7B (weight diff only): huggingface.co/jayelm/llama-7…
FLAN-T5-XXL: huggingface.co/jayelm/flan-t5…
Code: github.com/jayelm/gisting

This is probably true for humans too tbh.