Is there common practice on few-shot examples for text-to-SQL in terms of complexity? Is it better to give an LM progressive examples (easy to hard), full hard, or full easy?

Distilling a strong diffusion model into a much faster one with higher performance is achievable 📖 Read of the day, day 128: « SwiftBrush v2: Make your one-step diffusion model better than its teacher », by Trung Dao et al from VinAI research arxiv.org/pdf/2408.14176 The

Yes, diffusion models can generate video games 📖 Read of the day, day 129: « Diffusion Models are real-time game engines », by Dani Valevski, Yaniv Leviathan, moab.arar, and Shlomi Fruchter from Google Research To watch the mindblowing doom reproduction: gamengen.github.io

Trying to get a blog up Anyone knows good ressources?

Reducing parameters and improving performances of LLMs is possible 📖 Read of the day, day 131: « LLM Pruning and Distillation in Practice: The Minitron Approach », by Sreenivas, Saurav Muralidharan et al from NVIDIA arxiv.org/pdf/2408.11796 The authors apply a strategy to turn a model

Fast, good, open source speech-to-speech is possible 📖 Read of the day, day 132 : Mini Omni: Language Models can hear, talk while thinking in streaming, by Xie Zhifei and Wu from Tsinghua University arxiv.org/pdf/2408.16725 The authors present a framework to make a Large

I'm excited to announce Reflection 70B, the world’s top open-source model. Trained using Reflection-Tuning, a technique developed to enable LLMs to fix their own mistakes. 405B coming next week - we expect it to be the best model in the world. Built w/ Glaive AI. Read on ⬇️:

A search algorithm specific for code generation that does boost performance 📖 Read of the day, day 133: Planning in Natural Language Improves LLM Search For Code Generation, by Evan Wang et al from Scale AI arxiv.org/pdf/2409.03733 Researchers at scale AI made a method to

Pixtral -> Multimodal finally out I guess If it’s multimodal makes sense considering the size, would be SigLip + Nemo

First upload is out 👀

Can’t wait for the benchmark results from Pixtral Wanna know how it compares to current SotA in small-sized VLM I’m thinking that if Mistral released it after so long it should be pretty good, which is why I’m pretty hyped :)

**Frontier AI in your Hands** my live notes from today’s Mistral AI summit ft Jensen Huang and Arthur Mensch and crew here thread emoji

Turns out reflection-tuning was somewhat a good idea Just needed to RL through instead of LoRA

🎉Congrats to OpenAI for releasing o1: - Economics: tylercowen asked o1 basically to write a college essay - Genetics: @catbrownstein asked o1 to help her reason through "n of 1" cases - medical cases that nobody has ever seen - Physics: @mariokrenn6240 used o1 to draft and

o1-mini keeps refusing to try to solve the Riemann Hypothesis on my behalf. Model laziness continues to be a major issue sad ;p

🚨 BREAKING First, live Preliminary LiveBench results for 'Reasoning' show that OpenAI o1-mini massively outperforms Claude Sonnet 3.5 Claude 3.5 Opus soon?

This morning I had my first visceral “🤯” moment with AI for ~2 years 🧵on o1 and cryptic crosswords: My test for new models is a set of cryptic crossword clues that aren’t online (my granny wrote them). Every model so far has been completely useless at them… but o1 gets them

If o1 mini is in fact a distilled version of the o1 or o1 preview, this is huge Feeling like there’s a lot to be explored with llm distillation, especially considering we’ve got really strong large models atm