The interpretation of diffusion as autoregression in the frequency domain seems to be stirring up a lot of thought! (I may or may not have a new blog post in the works 🧐)

real

ok screw it, I'll put my money where my mouth is here. 10k$ bounty and a job offer to anyone who can figure out how to make a Mondrian compress to at least 16x fewer tokens than an equivalent resolution Where's Waldo in a way that generalizes like you'd expect.

I wrote a blogpost "On the speed of ViTs and CNNs". Addresses the following concerns I often hear: - worry about ViTs speed at high resolution. - how high resolution do I need? - is it super important to keep the aspect ratio? I think Yann LeCun might like it too! Link below

Classifier-free guidance is diffusion’s favorite hack for text-to-image generation, but what is it actually doing? We prove that CFG is a kind of predictor-corrector that alternates between denoising & sharpening — i.e., an annealed Langevin dynamics on sharpened distributions.

I’m beyond thrilled to share that our work on using deep learning to compute excited states of molecules is out today in Science Magazine! This is the first time that deep learning has accurately solved some of the hardest problems in quantum physics. science.org/doi/abs/10.112…

Think you understand classifier-free diffusion guidance? Think again! These two papers beg to differ😁 arxiv.org/abs/2406.02507 arxiv.org/abs/2408.09000 Both full of really great insights that question prevailing assumptions. cc Jaakko Lehtinen Arwen Bradley Preetum Nakkiran

It's so much easier to tweet low-effort memes which assert that diffusion is just autoregression in frequency space, than it is to write a blog post about it 🤷 (but I'm doing both!)

I actually like this analogy a lot! (... though IMO it should read "autoregression" instead: a Transformer is an NN architecture, not a modelling paradigm)

New blog post tomorrow 👀🌫️🌈

This post is really nice for intuition - I’ve learned a ton from Sander’s posts over time. But there are a couple of key points worth clarifying about whether “diffusion is *just* spectral autoregression”. TL;DR: I don’t think it is. 1) Autoregression that Sander Dieleman refers

'One can Fourier transform anything—often meaningfully.' -- John Tukey

I think we're just scaling the seed rounds at this point

🚨New blog on Flow Matching 🚨 I wrote it a while ago but decided to not release it since there were many other nicer ones already out. But I decided to publish it anyway. Link below 👇

ayandas.me/blogs/2024-04-…