this post was submitted on 25 Apr 2024
105 points (100.0% liked)
TechTakes
1552 readers
147 users here now
Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.
This is not debate club. Unless it’s amusing debate.
For actually-good tech, you want our NotAwfulTech community
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
the inputs required to cause this are so basic, I really want to dig in and find out if this is a stupid attempt to make the LLM better at evaluating code (by doing a lazy match on the input for “evaluate” and using the LLM to guess the language) or intern-level bad code in the frameworks that integrate the LLM with the hosting websites. both paths are pretty fucking embarrassing mistakes for supposedly world-class researchers to make, though the first option points to a pretty hilarious amount of cheating going on when LLMs are supposedly evaluating and analyzing code in-model.
I'd argue it's not the job of the AI researchers, I'd say for this it's more on the devs and engineers that built all the support for the AI to bring it to production. So basically the UI, the underlying hardware, OS, VMs etc.
all of the developers I know at AI-related startups identify as researchers, regardless of their actual role
no, let’s not blame unaffiliated systems engineers for this dumb shit, thanks
Oh, yea sorry I forgot AI models actually run in a vacuum and needs no supporting code or infrastructure to make it usable to the average user so it doesn't even need non-AI best security practices! Process isolation? OS hardening? Pfft who needs it
i wouldn't touch the llm stuff with a barge pole unless i was expressly told to do so, and if i've been told to do it, i'd look for another employer (which i'm currently doing, for tangentially-related reasons).
and it's not that i don't care about the llms. i do care very much about them all ending in fiery pit of the deepest of hells.
great thanks