This is amazing.

So I need to convert my llama.c models to llama.cpp, and found this incredible discussion between developers that did it already sometime last year.

This is why Open Source rules....

github.com/ggerganov/llam…

Congrats also to DGDG (dgastonia).

This was the first time I used it, to buy my medallion, and the process was super smooth. Well done!

Got a cool Jester Medallion.
Thanks LightningLad !

I uploaded all my trained models & tokenizers for on-chain llama2.c to HuggingFace.

I can't wait for the floating point update to be released on main net of the Internet Computer, so I can try out the 42M and 110M parameter models that currently hit the instruction limit.

Note…

Past week DFINITY Scalability and Performance team took ICGPT backend LLM for a spin in the new environment demonstrated for image classification earlier. Results came out wonderful. After saving billions of instructions, the generated result was identical.

Truly amazing work…

After hearing lastmjs ∞ describe the vision for Kybra during the ICP HUB townhall, I gave myself a stretch goal. Don't just port llama.cpp to the IC, but also port the python bindings. Because in python is where most of the use is....

Today I tested a beta version of DeVinci which allows you to upload a PDF file from local disk into a vector database that runs side by side with the Llama3 LLM in the browser, all locally on your device.

The DeVinci dApp itself is served from the Internet Computer, fully…

🔥llm.c update: Our single file of 2,000 ~clean lines of C/CUDA code now trains GPT-2 (124M) on GPU at speeds ~matching PyTorch (fp32, no flash attention)
github.com/karpathy/llm.c…

On my A100 I'm seeing 78ms/iter for llm.c and 80ms/iter for PyTorch. Keeping in mind this is fp32,…

Floating point operations are about to get 10x cheaper on the #IC ! #AI

Big progress in scaling my LLM on the IC to multiple concurrent users, thanks to the scalability experts from DFINITY.

This is a great configuration for my case, using 4 LLMs per subnet. Check out how much faster inference is when you go from 1 to 4 LLMs behind a load-balancer…

I needed this badly. Thank you for developing it.