I was wondering if anyone here got Code Llama 70b running or knows of any guides/tutorials on how to do so. I tried setting it up myself with a quantized version, and it was able to load but I think I must have misconfigured it since I only got nonsensical results. One thing I definitely don't understand is the templates, did they change those? Also, if this type of post isn't allowed or is off topic please let me know, I have never posted in this sublemmy before.

you are viewing a single comment's thread
view the rest of the comments

[–] shootwhatsmyname@lemm.ee 2 points 9 months ago* (last edited 9 months ago)

I’m just using Ollama with Ollama WebUI. You’ll have to use the right tag when installing Llama to make sure you get 70b.