I don't have to update my car in order to drive it. Sucks to be them
this post was submitted on 15 Nov 2023
247 points (98.4% liked)
Technology
59358 readers
6811 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
I thought the Rivian vehicles were still drivable? It was just the infotainment that’s impacted, no?
Imagine paying $90k for a vehicle and you can't even turn on the radio.
That’s always going to be a problem for early adopters of new tech. Plus, since it’s automotive you’d run that risk with the first model year or two after a major redesign too.
If what Rivian has reported is true it’s not really a failing of the equipment but human error. That’s always going to be tough to account for, though there’s likely improvements to be made to their testing methodology before pushing to prod.
I would think they would keep at least 1 of each model/trim of vehicle for testing these things. This leads me to believe one of the following:
- Too tight of deadlines
- Cheap management won't pay for testing time or units
- Culture of pencil whipping
- A bad apple didn't do their job, which should be caught by procedures
I got an offer for software engineering role at Rivian a few years ago. The pay was low.
That's typical of the automotive industry, they don't pay anyone reasonably, they treat the workers like shit, and then they get mad when the workers protest, unionize, or quit. This is how the automotive industry works from the top down, from the highest executive to the lowliest salesman.
Also from an IT/Development perspective, the money is usually meh, but also the technology is usually expected to be cutting edge, while on the backend they're cutting corners and costs, thus undermining that "cutting edge" factor.
I think it's more likely just typical dysfunctional software industry workflows. Companies that actually test their software adequately before deployment are the exception, not the norm. That's different from what you said in your second bullet point because it's not even an issue of cheapness, it's an issue of not actually understanding what the best practices need to be.
I mean... those "typical dysfunctions" are what OP described. You're just describing the general state of the industry, not providing further examples of disfunction.
No, the parent commenter attributed it to management not wanting to spend money. I'm attributing it to management being incompetent instead, which isn't the same thing. Spending even unlimited amounts of money is not sufficient to fix what's wrong.
¿Porque no los dos?
All of the above.
OTA updates are very much a double-edged sword.
Usually it's good, but sometimes you get crap like this or when Tesla downgraded a bunch of Model S batteries when they saw they weren't holding up as well as expected and wouldn't line up with their battery warranty.
E: and it's insane to me that cars don't have snapshots of previously installed firmware/other software. Or like an A/B partition style thing where you can revert if an update goes bad.
I'm a tech enthusiast, but I increasingly am slowing my roll on all updates but security updates. The era of heavily QAing feature updates is ending, which is leading to some frankly annoying issues across the board, from car software, to phone and computer software. Even GPUs don't seem to be immune from occasional bugs in the latest updates.
Companies and orgs really need to dedicate more time to making sure updates are vetted and clean before releasing them. It might cost you some money up front, but it saves you PR, security issues, and hotfixes in the longer run.
in the longer run.
I have yet to work for a company where management cares about this. It's always about what can be sold next week if we rush it enough. Or, more commonly, what was sold weeks ago without any consultation with the technical team.
It's a problem everywhere. Rush to build something, then don't actually finish it, or iron out the bugs before moving on to the next sexy thing. Management wonders why everyone is on the super ancient, rocksolid platform and hasn't really taken the bait on the series of products that have an "agile" development cycle and were dropped after about 5 years.
Guys....my entire team knows the issue. Support your damn products for years. Commit to improvements for a decade, and help people transition to a new product when the old one has to be sunsetted for technological or knowledge worker retirement reasons.
I have a Sync 3 system that filled its self up with files and no longer has free memory. Its a software bug that was fixed, but I'd have to pay to replace the entire unit rather than snag a utility to clear space and upgrade. That is soooo dumb.
But an unaffiliated community created a fix before Ford. Thanks hackers!
Weird. My primitive old Samurai started right up this morning, and it's "infotainment" system (bluetooth boombox bungee-corded to the floor behind the seats) still works.
Maybe if I'd paid $90,000 for it I could be privileged enough to have a broken functionality.
How are you replying from an i386 based pc running windows 3.11?
Slowly
I think while the topic is up it’ll be fun mentioning that the Colorado/Canyon does not have a physical headlight control anymore - in favor of defaulting to Auto and touchscreen controls and the project lead(?) claimed that the system was 100% bug-free.
Also later there was a bug with some OTA update for that model that’ll kill the battery.
Anyways, I bought a 4Runner and immediately called to disable its cellular radio. (dubbed DCM in Toyota-land)
How do you disable the DCM?
There have been some experimentation but the best (IMO) option is hitting the SOS button to call and navigating the phone menu to get to a representative, not emergency services and having them disconnect your car. You may need your VIN, you’ll need to confirm that you do not want connected services and it may take a day or so to take effect. Now, my SOS button doesn’t have the green light and while the radio - according to the infotainment - is still powered, it is no longer connected to the network.
Another way if you don’t plan on using the microphone (like for calls) you can pull the DCM fuse but I prefer the above option.
How though
"According to Rivian the glitch occurred when the wrong build with incorrect security certificates was inadvertently sent out. This error resulted in bricking the infotainment systems of an unknown number of vehicles."
I always thought that 'fat finger' revered to having fingers that are too big for buttons, but I guess it means sending out wrong software versions.
Fat finger in this context typically means that the person knew what they wanted to do, but typed the wrong thing. Their "fat finger" hit the wrong button. Possible that they wanted certificate 123 in the build, but they accidentally typed 133, or something like that.
This. And the guy who did it was fat.
I think it's also worth noting that "security certificates" in this context are almost certainly there for the purpose of locking the system away from modification by the vehicle owner, which means they weren't necessary (and indeed, ought to be prohibited by law) to begin with.
Rivian was hoist on their own consumer-hostile petard.
I don't think so in this context. This is probably more like SSL cert trusting or some private/public key pair.
Based on the fact that the screen is blank, I think @grue was correct. The certificate that tells the system that the infotainment was valid was incorrect, so the infotainment couldn't load. They used the dev-infotainment cert instead of the prod cert.
They did
Rm -rf /
To force remove everything recursively instead of
Rm -fr /
To remove the French language pack. Rookie mistake really.
I know you're being funny here, but....
Dumb-ass managers who thought they could run their embedded software group all Agile and shit, and push stuff out to Production without full testing to meet deadlines, not realizing that they can't just "roll back" this one.
They had 2 files in one directory. One said “good update” the other said “bad update-don’t use”. When they wanted to push out the update they used a tablet and tapped on “good update” but they accidentally touched the wrong one a bit more with the thumb.
Something like this. But maybe they don’t want to say wat really happened.
Between the massive privacy issues and cars getting bricked by automatic OTA updates, I currently see zero reason whatsoever to buy a car made after ~2015.
I just recently upgraded to a 2013 Rav4 and it has all the tech I could ever want. I'm gonna keep it as long as I can.