@aaronsnoswell : To train GPT-3, OpenAI needed over 6550 billion English words of text – about 200x more than the entire English Wikipedia. But this required collecting almost 100x more raw data from the internet, up to 98% of which was then filtered and discarded 🤯 arxiv.org/abs/2005.14165 • TwiDoom

Aaron J. Snoswell, PhD

@aaronsnoswell

+ Follow

Research fellow in computational law @ADMSCentre. Other half of @CSnoswell. Tweets about robotics, machine learning and game development (薛嘉伦).

ID: 48354997

linkhttps://aaronsnoswell.github.io calendar_today18-06-2009 13:51:38

5,5K Tweet

1,1K Takipçi

3,3K Takip Edilen

Aaron J. Snoswell, PhD

@aaronsnoswell

a month ago

To train GPT-3, OpenAI needed over 650 billion English words of text – about 200x more than the entire English Wikipedia. But this required collecting almost 100x more raw data from the internet, up to 98% of which was then filtered and discarded 🤯 arxiv.org/abs/2005.14165

thumb_up_off_alt6

chat_bubble_outline3

repeat0

shareShare