Looking forward to participating and talking at this #ICML2023 workshop on PAC-Bayes and interactive learning. Working on related topics? Consider submitting! The deadline is in on May 31st!

We’re thrilled to announce the first competition on unlearning, as part of the #neurips competition track! Joint work w Fabian Pedregosa, Vincent Dumoulin, Gintare Karolina Dziugaite, Ioannis Mitliagkas, @KurmanjiMeghdad, Peter Triantafillou, Isabelle Guyon and others. Read our blog post!

We are 1.5 months away from the deadline, consider submitting your recent findings in "sparse" learning, data, modality, signal processing. Introducing Conference on Parsimony and Learning (CPAL) 2023 at Hong Kong University youtu.be/pGbjiZOR63I via YouTube

Sharing this fantastic opportunity for high school students interested in STEM! It’s a unique mentoring program enabling young people to engage in research and advanced topics in math. Applications are due in September 🗓️

We're excited to announce the first edition of 🔵🔴 UniReps: the Workshop on Unifying Representations in Neural Models! 🧠 To be held at NeurIPS Conference 2023! SUBMISSION DEADLINE: 4 October Check out our Call for Papers, lineup of speakers and schedule at: unireps.org

Deep learning may be hard, but deep un-learning is even harder. 💪 How do we efficiently remove the influence of specific training examples while maintaining good performance on the remainder? Announcing NeurIPS Unlearning Competition 📢 Submit your best ideas!🏆

Check out this new exciting paper by my colleagues bringing together two hot topics: sparsity and scaling laws! 🔥

CONGRATS to our CPAL Rising Star Awardees!! An incredibly talented and impressive bunch 💯 See who they are and what they do: cpal.cc/rising_stars_a…

It was a great experience to give an in-person talk at #MLinPL to such an active audience with lots of thoughtful questions! Thanks again to the fantastic hosts who made me feel really welcomed, and made sure my visit was really well planned.

🎉 Two of my papers have been accepted this week at #ICLR2024 & #AISTATS! Big thanks and congrats to co-authors Xuxi Chen & Eric Gan, mentors Atlas Wang & Gintare Karolina Dziugaite, and especially my advisor Baharan Mirzasoleiman! 🙏 More details on both papers after the ICML deadline!

We've finally put out a detailed IEEE/ACM paper on Google's multi-year effort to ease the burden of code review with ML. Google engineers now resolve 7.5% of all code review comments with an ML-suggested edit. But the path to that number has been a fun ML and UX journey!

📢Mixtures of Experts unlock parameter scaling for deep RL! Adding MoEs, and in particular Soft MoEs, to value-based deep RL agents results in more parameter-scalable models. Performance keeps increasing as we increase number of experts (green line below)! 1/9

In deep nets, we observe good generalization together with memorization. In this new work, we show that, in stochastic convex optimization, memorization of most of the training data is a necessary feature of optimal learning.

DEADLINE March 29: prepare and submit your application for EEML 2024, Novi Sad, Serbia eeml.eu 🇷🇸. Topics: Basics of ML, Multimodal learning, NLP, Advanced DL architectures, Generative models, AI for Science. Check our stellar speakers! Scholarships available! 🎉

Google presents Mixture-of-Depths Dynamically allocating compute in transformer-based language models Same performance w/ a fraction of the FLOPs per forward pass arxiv.org/abs/2404.02258

How are LLM capabilities affected by pruning? Checkout our ICLR 2025 paper showing that ICL is preserved until high levels of sparsity, in contrast to fact recall which quickly deteriorates. Our analysis reveals which part of the network is more prunable for a given capability.

See u tmrw ICLR 2025 Sess 1 #133! When we down-scale LLMs (e.g.pruning), what happens to their capabilities? We studied complementary skills of memory recall & in-context learning and consistently found that memory recall deteriorates much quicker than ICL when down-scaling.

thrilled this paper was accepted to #ICML2024 , looking forward to chatting about this with many of you in vienna! camera-ready version available at arxiv.org/abs/2402.08609 w/ Johan S. Obando 👍🏽 Ghada Sokar Timon Willi Clare Lyle Jesse Farebrother Jakob Foerster Gintare Karolina Dziugaite & doina precup

We've seen memorization in NNets, despite good generalization. But can we generalize without memorizing? Come hear about our best #ICML2024 paper award work on showing that in stochastic convex opt, optimal learners memorize! Talk today at 11.15am, poster at 11.30am.

Excited to present our spotlight paper on MoEs in RL today at #ICML2024! Me, Johan S. Obando 👍🏽, Pablo Samuel Castro, and Jesse Farebrother are looking forward to chat with you! Poster #1207 Hall C 4-9 at 1:30-3:00 pm