it's important to realize that attention-free architectures might require different methods of interaction (prompting etc) and evaluating them in the same way that we know works for Transformers might not always make sense

FlashAttention is widely used to accelerate Transformers, already making attention 4-8x faster, but has yet to take advantage of modern GPUs. We’re releasing FlashAttention-3: 1.5-2x faster on FP16, up to 740 TFLOPS on H100 (75% util), and FP8 gets close to 1.2 PFLOPS! 1/

we collaborated with Mistral to scale Mamba up on code! I think that different modalities or data formats (e.g. code, byte-level modeling) that have less strong "tokenizations" benefit increasingly more from compressive models such as SSMs

was fun visiting the Mistral AI office!! 🇫🇷🥖

at ICML, presenting Mamba-2 on Wednesday! lmk if anyone wants to catch up

After 7 months on the job market, I am happy to announce: - I joined Ai2 - Professor at Carnegie Mellon University from Fall 2025 - New bitsandbytes maintainer Titus von Koeller My main focus will be to strengthen open-source for real-world problems and bring the best AI to laptops 🧵

I'm very excited to be starting at Brown CS this fall!

Excited to share that I will be joining UCSD CSE as an assistant professor in January 2026! I'll be recruiting PhD students from the 2024 application pool - if you're interested in anything ML Sys/efficiency/etc please reach out & put my name on your application! Until then

To celebrate the launch of Daily's new bots,we built StudyPal, a buddy that helps you learn the most important information from research papers 🥸 Watch how Studypal teaches us the content of our Chief Scientist Albert Gu 's latest Mamba 2 Paper: arxiv.org/abs/2405.21060

distillation.... mmm 🍻 state-of-the-art Mamba models with 1% of the compute, by leveraging pretrained Transformers! key insight: project the (quadratic) attention matrices onto (structured) SSM matrix mixers before end-to-end training led by students Aviv Bick Kevin Li

🤯This study explains my year-long confusion on why #GPT4 leak says OpenAI deployed speculative decoding in their serving last June by Dylan Patel SemiAnalysis because I thought SD is only useful for small batches... Surprisingly speculative decoding can bring more benefits when

We released the #Jamba 1.5 open model family: - 256K #contextwindow - Up to 2.5X faster on #longcontext in its size class - Native support for structured JSON output, function calling, digesting doc objects & generating citations twtr.to/giIEE #AI #LLM #AI21Jamba

Today, we’re unveiling a significant milestone in our journey toward ubiquitous artificial intelligence: AI On-Device. Our team pioneered a radically more efficient architecture for AI with state space models (SSMs). Now, we’ve optimized and deployed them at the edge. We believe

We made distillation and spec decoding work with Mamba (and linear RNNs in general)! Up to 300 tok/sec for 7B🚀. Spec dec is nontrivial as there's no KV cache to backtrack if some tokens aren't accepted, but there's an efficient hardware-aware algo to recompute the SSM states

pretty sure one of these does not belong, but thanks TIME 🤔