Excited to share what I've been working on alongside Philippe Chlenski under Neel Nanda's mentorship: finding interpretable feature circuits in LLMs using a tool called transcoders!
Blogpost: lesswrong.com/posts/YmkjnWtZ…
Code: github.com/jacobdunefsky/…
(1/8)

ML Alignment & Theory Scholars has 1220 summer applicants and enough funding to accept 2.5% (ideally 7%). Accepting donations via existence.org and manifund.org/projects/mats-… at $24k/scholar. Help us support mentors like Neel Nanda, Owain Evans, Ethan Perez, Evan Hubinger, and more!

Great work from my MATS scholars!

Refusal in LLMs is mediated by a single vector - injecting it means harmless statements are refused, ablating it everywhere lets harmful prompts through

We can jailbreak model *weights* by projecting out this direction, no fine tuning needed!

New research post on refusals in LLMs lesswrong.com/posts/jGuXSZgv…

It's a great week for mech interp releases! I'm very excited to try out Anthropic's new recommendations for stable dictionary learning

New Google DeepMind MechInterp work! We introduce Gated SAEs, a Pareto improvement over existing sparse autoencoders.

They find equally good reconstructions with around half as many firing features, while maintaining interpretability (CI 0-13% improvement). Joint w/ Arthur Conmy

astralcodexten.com/p/ye-olde-bay-…