trying a new attention pattern with FlexAttention!

Some folks had some confusion about whether FlexAttention worked on H100s. To clarify, FlexAttention runs on H100 with fairly good perf (up to ~40% faster than FA2). However, it is still about 75-85% of the FLOPs of FA3. Check out Driss's thread for more details.

Fun fact, this was actually the first novel mask implemented in FlexAttention from someone not working on FlexAttention!

We’re planning on adding learned bias support for FlexAttention. Unfortunately, this is somewhat nontrivial to do efficiently and generically. So, for flashattention, do you usually care more about 1. good performance or 2. avoiding quadratic memory usage?

to truly appreciate flex attention you need to have tried to make a minor modification to flash attention

Show-o update🔥: 1. We have released training codes on GitHub, including both pre-training and instruction tuning! 🔥 2. Add FlexAttention’s impl for great speed up. Thanks Horace He 🚀 github.com/showlab/Show-o… 3. gradio demo up 🤗 huggingface.co/spaces/showlab… Have fun!

We're excited to announce the release of SGLang v0.3, featuring enhanced performance and extended support for novel architectures! Highlights include: - Up to 7x higher throughput for DeepSeek Multi-Head Latent Attention (MLA) - Up to 1.5x lower latency with torch.compile on