lucidrains joined the PyTorch team at Meta this June, doing what they do best -- working on open source implementations of important AI work. So proud to have them on the team! (just announcing after realizing we never did)

For too long, users have lived under the software lottery tyranny of fused attention implementations. No longer. Introducing FlexAttention, a new PyTorch API allowing for many attention variants to enjoy fused kernels in a few lines of PyTorch. pytorch.org/blog/flexatten… 1/10

When the Twilight Imperium 4 game has derivatives on if futures contracts of trade goods will be fulfilled by a certain date

Share your FAVORITE PICTURES from the 2024 Olympics In Paris I’ll Start

me trying to act normal on a date:

Very interesting to see differences in feature announcement: - GDM: corpo blog - OAI: weird rumors for a week - Anthropic: dude posts docs and it's out

I wrote a blogpost "On the speed of ViTs and CNNs". Addresses the following concerns I often hear: - worry about ViTs speed at high resolution. - how high resolution do I need? - is it super important to keep the aspect ratio? I think Yann LeCun might like it too! Link below

Doing a quick thread on the backstory here Impulse We realized that putting battery packs in appliances made it possible to greatly exceed the available power you'd get from the wall -- and could crush other stoves at boiling water, getting pans up to temp, etc. BUT

Excited to announce our new work: Critique-out-Loud (CLoud) reward models. CLoud reward models first produce a chain of thought critique of the input before predicting a scalar reward, allowing reward models to reason explicitly instead of implicitly! arxiv.org/abs/2408.11791

My biggest ineffective altruistism: donating to live music on public transit Safe, clean subways with music in New York are such a wonderful experience

Excited to be at PyTorch conference! Come say hi :)

At the height of One Million Checkboxes's popularity I thought I'd been hacked. A few hours later I was tearing up, extraordinarily proud of some brilliant teens. A thread about my favorite story from running OMCB....

everyone working at the forefront of model training should internalize what this graph means for long term cluster scaling.

#NYFW FW24 NVIDIA 🔋✨Tessa

Blazingly fast fashion🔥

set up my internet today

They call it mid training cause the good engineers are on pretraining and the good researchers are on after training

Claude UX is soo much better than anything else (artifacts, projects, etc) But please someone add the ability to mass delete and upload project knowledge (eg for a repo being frequently updated). Would become so elite Alex Albert 🙏🙏

hey NVIDIA now even the press is after you about those graphics drivers