LocalLLaMA

2249 readers

1 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 1 year ago

MODERATORS

pax@sh.itjust.works

SkySyrup@sh.itjust.works

noneabove1182@sh.itjust.works

Advice Wanted - Self Hosting Industrial GPUs (kbin.social)

submitted 1 year ago by wagesj45@kbin.social to c/localllama@sh.itjust.works

7 comments fedilink hide all child comments

I just bought a "new" homelab server and am considering adding in some used/refurbished NVIDIA Tesla K80s. They have 24 GB of VRAM and tons of compute power for very cheap if you get them used.

The issue is that these cards run super hot and require extra cooling set ups. I was able to find this fan adapter kit on eBay. But I still worry that if I pop one or two of these bad boys in my server that the fan won't be enough to overcome the raw heat put off by the K80.

Have any of you run this kind of card in a home lab setting? What kind of temps do you get when running models? Would a fan like this actually be enough to cool the thing? I appreciate any insight you guys might have!

you are viewing a single comment's thread
view the rest of the comments

[–] wagesj45@kbin.social 3 points 1 year ago

Thanks for the input. I live in a cold state and all the machines are in the basement, so even in summer it's pretty cool in there. This gives me hope!