New Preprint. 🔒 In an era where AI's influence is surging, government compliance isn't optional—it's imperative. Our research introduces GUARD, a novel system ensuring LLMs and VLMs are tested for adherence to these critical standards. #AICompliance #GovernmentGuidelines

Can hazardous knowledge be unlearned from LLMs without harming other capabilities? We’re releasing the Weapons of Mass Destruction Proxy (WMDP), a dataset about weaponization, and we create a way to unlearn this knowledge. 📝arxiv.org/abs/2403.03218 🔗wmdp.ai

🚀 Thanks for hosting! Excited to share our latest work on jailbreaking LLMs: 1️⃣ Compliance testing with jailbreak 🧐 arxiv.org/abs/2402.03299 2️⃣ systematic approach to defense 💪 arxiv.org/abs/2401.17263 with Haibo, Andy Zhou, Lapis Labs, and Bo Li; Trustworthy ML Initiative (TrustML)

Language Agent Tree Search 🤖🌲 As LLMs get faster, better, cheaper, developers will be able to compose agentic systems that are able to plan out an entire tree of possible futures, instead of just sequentially planning the next state (e.g. in ReAct). This is crucial for higher

Pleased to announce Language Agent Tree Search was accepted to #ICML2024 !! We propose a general search algorithm for LM agents that effectively navigates the prompt space for agent tasks Check it out here arxiv.org/abs/2310.04406 Great LangChain implementation here

The SWE-agent preprint has finally landed! Check it out at swe-agent.com/paper.pdf

I got an email bringing this paper to my attention that mentioned something was concerning, and the methodology is almost exactly the same as our ICML 2024 work (released Oct 2023), Language Agent Tree Search (arxiv.org/abs/2310.04406), but did not cite us...

Introducing FIRST: Faster Improved Listwise Reranking with Single Token Decoding arxiv.org/pdf/2406.15657 Listwise LLM reranking typically outputs the ranking order as a generation sequence. Instead, we use output logits of the first generated identifier to obtain the ranking.

We present AIR 2024, a unified AI Risk Taxonomy for AI regulation and company policy-guided risk assessment and compliance, jointly with Stanford University's HELM. 📜Blog: virtueai.com/2024/07/27/dec…

Excited to release this work on AI policy! We map out company/government policies into a taxonomy with 314 potentially harmful categories arxiv.org/abs/2406.17864

Leading the Superalignment team is like taking the Defense Against the Dark Arts position.

Excited to feature Tamper-Resistant Safeguards for Open-Weight LLMs from Lapis Labs! Introducing the first safeguards for LLMs that resist fine-tuning attacks, showing the power of tamper-resistance to make open-weight LLMs safer. Rishub Tamirisa is here to answer your questions!

Join us for the invited talk Language Agent Tree Search in the AutoGen community meetup (Do not miss!) Time: August 26, Monday 9am PT. Event link: discord.com/events/1153072… Abstract: ⬇️