Who is creating a blockchain of test questions, with a 'proof of salience' for maximally separating performance amongst frontier models?

I am starting to formulate my own hypotheses about the inner workings of LLMs. I find myself wanting to have a highly accurate, up-to-date LLM-based AI assistant to converse with about some of my ideas and intuitions, and to dig up relevant papers.
Does someone maintain such a

Alternatively, as the social media enshittification has shown, they may find low-value use cases that suck in and trap users into ultimately giving something of much higher value.
Because web hosting was too hard and a decentralized identity system didn’t exist, most users were

Zuck deciding that Meta should be a better Google than Google is such a fascinating turn.

thinkwert continued to delight after so so many posts

Had a lot of fun Data Council. Thanks again to Pete Soderling for the conf and Tristan Zajonc for being an amazing track host. My talk is now on YT. Tho I’m thinking of renaming it to “What? It’s just a file?! 🤯” instead.

youtu.be/9O2pfXkCDmU?si…

Ok but this is legit amazing

Daytime Addy awards to Ali and Wilhelm

I had to listen to this three times to try and figure out what Ms. Maher is saying. A few things:

1. Afeni Shakur, member of the Black Panther Party, mother of Tupac Shakur, and a legend in her own right, said that she worshipped the written word. In a way all rappers do. To

It is an incredibly safe assumption that models get much cheaper and smarter from here, yet very few founders are acting like it

We now have an open-source model that is beating Claude 3 Opus...

being served at nearly **300 tokens per second** on Groq Inc.

The applications built off of this tech will be nothing short of revolutionary.

Massimo The original Brahmi numeral system, evolved into the Hindu numerals, & that was later copied to form Arabic numeral system, which got carried over to Europe.

Hindu numerals are a set of 10 symbols 1, 2, 3, 4, 5, 6, 7, 8, 9, 0 — that represent numbers in decimal number system.

This fall, I got an e-mail from a guy named JM Simpson who'd read things I'd written about homelessness. He mentioned he was a photojournalist in Olympia and that he'd been photographing the unhoused population there. He wanted to share some work with me . . . (1)

Simpson told me he talks with, gets to know, and asks permission from any person before he takes their photo. This is good ethical practice. But it also leads to photography that feels intimate and humane rather than removed and voyeuristic (3)

Which is to say, his photography avoids reducing the subjects to mere objects of pity, but also avoids romanticizing their lives (6)

I don’t know if AI “understands” human values but it can certainly compute over them in a sensible way. And—Chinese Room philosophical disputes aside—maybe that’s good enough?

See for instance this cool work: meaningalignment.substack.com/p/the-first-mo…

Think of three people you know personally that you look up to/admire

What are the top 2-3 traits you appreciate most about them?

I agree with Andrej Karpathy that smaller models would be welcome as well… What is the best way to build a quality 1B-3B model that could be used in an MoE configuration?
Is it best to quantize and distill or extract from a big existing model, or is it better to train up a small model

Congrats to AI at Meta on Llama 3 release!! 🎉
ai.meta.com/blog/meta-llam…
Notes:

Releasing 8B and 70B (both base and finetuned) models, strong-performing in their model class (but we'll see when the rankings come in @ lmsys.org :))
400B is still training, but already encroaching

How Faithful are RAG Models?

This new paper aims to quantify the tug-of-war between RAG and LLMs' internal prior.

It focuses on GPT-4 and other LLMs on question answering for the analysis.

It finds that providing correct retrieved information fixes most of the model

my big bets for the future:

- the boundaries between mediums will mostly disappear.
- computation will be embedded in other mediums rather than as a stand alone thing.
- the ability to sketch out rough ideas and progressively enhance them will be incredibly important.
- AI will