Ever noticed how Pixar adapts movies for international markets? The beloved newscaster in Zootopia is a jaguar in Brazil, a panda in China, a koala in Australia …

While machine translation (MT) has only dealt with language in speech/text thus far, we extend the scope of MT to…

Checkout our work on adapting multitask learning as a tool against worst case group error.

Our modified MTL approach (main task + pre-training auxiliary objective + L1 embedding reg) is competitive against bespoke DRO (Distributionally Robust Optimization) methods

I'm super excited to release DiPaCo, a new kind of mixture of experts, that can scale engineering-wise to data centers across the entire world!

A few words about it in this thread 🧵

Introducing Unified PDE Solvers (UPS), a step towards efficiently building foundation models for PDE solvers (arxiv.org/abs/2403.07187)! Starting from a pretrained LM, UPS tackles diverse spatiotemporal PDEs with SOTA accuracy, using ~20x less data and a single A6000! 🧵[1/x]

For large-scale causal discovery, there's no need to use NOTEARS for its speed. Consider using LiNGAM. We've parallelized it, achieving a 32x speed-up on GPUs.

NOTEARS:
Scalable: ✅
Identifiability guarantees: ❌

AcceleratedLiNGAM:
Scalable: ✅
Identifiability guarantees: ✅

Applications are now open for our AI for Science Master's program 🧪!

Thanks to Google DeepMind there are 40 full scholarships available to students from across Africa 🎓. The cherry on top - 1:1 mentorship by Google DeepMind researchers 🍒

More details: ai.aims.ac.za

Apply to work with Patrick Emedom-Nnamdi !
His startup is up to some cool stuff 🤫 -- so if you're into ML and Healthcare -- please reach out to him !

I've read ~50 SOPs (via STEM for development, my personal network, random email requests, and my recent Twitter form) over the past 2 months, and these are COMMON MISTAKES I've noticed graduate school applicants making quite often:

[A thread]

Graham is amazing !
Come work with him

Checkout this amazing effort by Simran Khanuja ! Labeled data for MT isn’t magicked - you must chose what points to label (under annotation constraints) to help your target language. Simran’s work provides insights about what strategies work and when !

I am running (unopposed?!) for chair of the NAACL board: naacl.org/elections/2024…

My goals:
1. Maintain the high quality of NAACL's content
2. Make sure NAACL welcomes researchers of all backgrounds

If you are a NAACL member, I would appreciate your support; check your email!

The dominant paradigm in deep learning is to build larger-sized models. In recent work, we show that the statistical guarantees of such models do not necessarily suffer as a result of their sheer size.

arxiv.org/abs/2310.03957

🧵

Now accepted to NeurIPS'23! Looking forward to talking to folks in New Orleans 🎉

*⃣ Resource alert for people applying to CS PhD programs this cycle *⃣ cs-sop.org contains >60 example statements of purpose! It's made possible by the many generous submissions from new applicants, and new ones are always welcome! 😊

How much of the value of pre-training comes from merely serving as a *good initialization*? 🤔

We propose a simple, effective, and compute-free initialization method for Transformers (esp. vision) that *mimics* pre-training.

See our #ICML oral at 3pm today (313)! w/Zico Kolter

Amazing, insightful thread by Delip Rao e/σ on understanding motivations of parties of interest behind the different framings of LLM access and capabilities

Congrats Kayo Yin 🇦🇹 Patrick Fernandes !!!!

The Vendi Score: A Diversity Evaluation Metric for Machine Learning

Dan Friedman, Adji Bousso Dieng.

Action editor: Antonio Vergari.

openreview.net/forum?id=g97OH…

#diversity #diverse #similarity