This was a super fun interview, really appreciated Tejas Kumar's thoughtful questions!

if you have successfully performed a full fine-tune of Llama 3.1 405B, DM me. working on something pretty special here and want to trade notes.

People think OpenAI charging $2K a month for ChatGPT subscriptions would be crazy, but I would pay $2K a month for a Cursor+Claude subscription if no cheaper alternatives were available. It isn't unreasonable if their new models are way better.

Has anyone had good results using KTO to replace an SFT+DPO pipeline? Seems promising but in practice we're seeing slightly worse results (relative to SFT+DPO) across most benchmarks.

This is strong evidence that Llama 3.2 with multimodal support is close to dropping 😆

We're adding o1-preview and o1-mini-preview to OpenPipe as relabeling models to improve your dataset right now! If you fine-tune a smol model on o1 outputs you'll be able to pick up much of the improved quality without paying any more for inference than before. 🙂

o1 is a huge milestone: we now have the playbook to replicate human-level system-1 and system-2 thinking. o1 reasoning = system 2 small model fine-tuned on o1 outputs = system 1

The playbook every LLM app will run at scale: run the first 1K inputs through o1, fine-tune on outputs, then run the next 1B through llama 8b. System 2 distilled into system 1.

We'll be hosting an SF meetup on September 25th alongside Modal and Airtrain AI! Would love to see you there and chat about real-world AI apps. 🙂

Founder friends: do you promote a writing culture for internal comms at your company? Have you found it to be an unlock? For context OpenPipe is a 6 eng in-person team and internal comms are mostly oral. Works for us but wondering if we're missing something.

I will consider it a great failure if OpenPipe ever shows up on a list of "fastest growing startups by headcount." One of the many places where less is more (see also: lines of code!).

I agree with this! If your information extraction LLM is cheap enough to run you'll get better recall by running it on all 1M posts in the subreddit you care about than by first filtering to the top 50 and running it on those. 🙂

Hacker News has ~38M posts and comments since its creation. Average comment length is 85 tokens Gemini Flash costs $0.075 / 1M tokens ...you can write a script that classifies every HN comment ever using Gemini Flash for... $241. 🤯

excited to nerd out about building AI systems with some of my favorite practitioners, Emmanuel Turlay & Kyle Corbitt -- and hopefully you too! lu.ma/jfsyz0xc