Fused neighborhood attention now supports backward pass! Upgrade today to get all the features and all the speed. Get up to 844%, 385% and 447% improvement in 1D/2D/3D forward+backward pass at the op level. Training w/ spatio-temporal attention should be a breeze now.

44TB! That'd be a respectible size for an image dataset, but just text, that's a LOT of text!

With no changes from Llama2 to Llama3 architecture, the importance of data has never been so clear 🦙
While Idefics2 was heavily advertised, our efforts on data didn't shine ☀️ as much, so here are the 3 most important data updates we did:
1. 🍵 The Cauldron, a compilation of 50…

Not everyone is aware but the fact that llama3 has been added to HF on day one means that right away, you can easily run it on @aws Sagemaker, @microsoft AzureML, Google Cloud Vertex and the Hugging Face hosted solutions (autotrain, spaces, serverless and dedicated endpoints).

Today we release Idefics2 our newest 8B Vison-Language Model!
💪 With only 8B parameters Idefics is one of the strongest open models out there
📋 We used multiple OCR datasets, including PDFA and IDL from Ross Wightman and Pablo Montalvo, and increased resolution up to 980x980 to improve

Introducing Idefics2, the strongest Vision-Language-Model (VLM) < 10B! 🚀 Idefics2 comes with significantly enhanced capabilities in OCR, document understanding, and visual reasoning. 💬📄🖼️

TL;DR;
📚 8B base and instruction variant
🖼️ Image + text inputs ⇒ Text output
📷…

I wrote a blog post on going from not knowing anything about deep learning last year to training state of the art OSS models - vikas.sh/post/how-i-got… .

Hope it helps you.

tldr; read the deep learning book, implemented papers + taught, built open source tools

Did you know that you can now view PDFs hosted on the Hugging Face hub, directly on the hub?

Read here Stas Bekman's Machine Learning Engineering Open Book

“[Open-source AI] is really fundamental because it allows everyone to seize the technology, to diminish the fear of limited understanding or of not being qualified to use AI” - Remi Cadene 🔥🔥
Great piece in Bloomberg Opinion by Parmy Olson on the open-source landscape bloomberg.com/opinion/articl…