📰 Check out our new work on Data Shapley without retraining! Inspired by how privacy accountants track privacy leakage in DPSGD, we track data contributions by accumulating data value scores for each training *step*, in the spirit of divide-and-conquer.

Very cool work led by the incredible Jiachen Wang @ICML; data attribution is an important piece in alignment bc it goes beyond heuristic filtering to actually identify datapoints responsible for teaching the model malicious behavior. This is a good step towards attributing pretrain.

I think there might be many exciting alignment projects that follow up on this work. Months ago, I was trying to think about how to make use of influence functions, but they were too computationally expensive to do the things I wanted to do. But this paper is promising...

The Caltech Division of Engineering & Applied Science Trailblazers Symposium seeks to recognize and provide professional development opportunities for outstanding early career researchers who advance the frontiers of research in engineering and applied science. eastrailblazers.caltech.edu Graduate students and

🎁GPT-4o-mini just drops in to replace GPT-3.5-turbo! Well, how has its 🚨safety refusal capability changed over the past year? 📉GPT-3.5-turbo 0613 (2023) ⮕ 1106 ⮕ 0125 ⮕ GPT-4o-mini 0718📈 On 🥺SORRY-Bench, we outline the change of these models' safety refusal behaviors

Excited to be attending #ICML next week! I will give an oral presentation on our work about the theoretical foundation of Data Shapley for data curation. Happy to discuss data curation, data attribution, and all related topics. Feel free to DM for a coffee chat in Vienna! ☕️

Check out this recent work by my student Bilgehan Sel et al. at #ICML2024! "Algorithm of Thoughts" (AoT) could be a leap forward for how we use large language models (LLMs) to solve problems. 🧠 Ahmad Tawaha Vanshaj Khattar Ruoxi Jia The Sanghani Center at Virginia Tech arxiv.org/abs/2308.10379

Excited to share our latest work that provides a general recipe for advancing SOTA for preference alignment (e.g., DPO) with a simple yet effective data-centric approach—augmenting the preference dataset with rationales! Definitely check it out before aligning your next model:

We're seeking a dedicated student researcher to work on groundbreaking #LLM alignment research this fall. This position requires a minimum 80% appointment, and we strongly encourage working from office. If you're passionate about advancing AI safety, please DM me with your

🚨Wondering how often 🔥Llama-3.1-405B-Instruct refuses to answer potentially unsafe instructions? 👇We outline the percentages of potentially unsafe instructions fulfilled by Llama-3.1 models from SORRY-Bench (sorry-bench.github.io) below. 🔥The new 405B model lies

We at Virtue AI are excited to announce our recent public effort: 🧵[1/3] 🧾Comprehensive Safety Assessment of Llama 3.1 405B: virtueai.com/2024/07/28/saf…

So proud of you! Check out Yi Zeng 曾祎's amazing efforts in technology transfer

🧵[1/5] Introducing AIR 2024: Unifying AI risk categorizations with a shared language to improve AI safety. W/ Kevin Klyman Andy Zhou Yu Yang Minzhou Pan & guidance from Ruoxi Jia Dawn Song Percy Liang Bo Li for kicking off my AI policy research journey 🏦.

Join us tomorrow as we delve into the exciting topic of trustworthy interactive decision-making with foundation models IJCAIconf The Sanghani Center at Virginia Tech Time: 2-5:30 PM Korea Standard Time (GMT+9) Location: 4F-Room 402A @ ICC Jeju docs.google.com/presentation/d…

🤩So honored to receive TWO paper awards ACL 2025, huge shoutout to my amazing collaborators🤩!!! 🏆Best Social Impact Paper🏆: persuasive jailbreaker arxiv.org/abs/2401.06373 🏆Oustanding Paper🏆: persuasive misinformation arxiv.org/abs/2312.09085 #ACL2024

Over the summer, we had a reading group on data attribution. We first covered (a biased selection of) popular data attribution methods and then went through recent papers related to data attribution for generative AI. The recordings can be found at trais-lab.github.io/dattri-reading… 1/