First driverless car pedestrian death

The Tesla autopilot seems to have a problem recognizing Fire trucks. In June a model X ran without braking into the back of an engine stopped on the 405 in Culver City, CA. This past week another Tesla ran iwithout braking nto a fire Dept maintenance truck in South Jordan, UT (minor injuries in the latter accident).
I can understand the algorithm mistaking the lady with the bike for a tumbleweed, I can't understand how radar could miss a truck stopped at a red light.
 
The Tesla autopilot seems to have a problem recognizing Fire trucks. In June a model X ran without braking into the back of an engine stopped on the 405 in Culver City, CA. This past week another Tesla ran iwithout braking nto a fire Dept maintenance truck in South Jordan, UT (minor injuries in the latter accident).
I can understand the algorithm mistaking the lady with the bike for a tumbleweed, I can't understand how radar could miss a truck stopped at a red light.

Tesla (actually all vehicles with TACC) radar will always ignore stationary objects as it’s not possible to distinguish a car and a road sign using radar. But radar can detect speed well (Doppler effect) so a TACC radar model will contain all of the objects with any velocity relative to ground, and ignore everything else.

When you’re in stop-and-go traffic the car will remember objects that were moving before and are now stopped, but if you happen to come up onto a stopped object out of the blue it will ignore it.

There is a big warning paragraph in the Tesla manual (and apparently Volvo as well) to not solely rely on TACC around stationary vehicles.
 
You'd have thought that a stationary object directly in front of the car, that it's going to drive in to, might be seen as a threat. Even if it were a road sign, you probably still wouldn't want to drive into it...
 
Anyone else see the news that squeaked out...? The car did see her but some brilliant engineer’s code tagged her as a false target.

They haven’t said what the car thought she was or which “stop the car from reacting” logic thread it went down.
Yes, I saw or heard that too. It seems to show that the systems being tested are not artificial intelligence. It sounds like the system is not capable of learning, if the programmers have to code in detailed rules to follow.
 
Last edited:
Anyone else see the news that squeaked out...? The car did see her but some brilliant engineer’s code tagged her as a false target.

They haven’t said what the car thought she was or which “stop the car from reacting” logic thread it went down.
How would you like to be the guy who wrote that section of code?
 
You'd have thought that a stationary object directly in front of the car, that it's going to drive in to, might be seen as a threat. Even if it were a road sign, you probably still wouldn't want to drive into it...

You wouldn’t want it to stop for an overhead road sign though.

TACC radar works in the X and Z plane - not Y. It can’t determine if or how far an object is off the ground.
 
You wouldn’t want it to stop for an overhead road sign though.

TACC radar works in the X and Z plane - not Y. It can’t determine if or how far an object is off the ground.

Seems like quite a deficiency...
 
Anyone else see the news that squeaked out...? The car did see her but some brilliant engineer’s code tagged her as a false target.

They haven’t said what the car thought she was or which “stop the car from reacting” logic thread it went down.

Post 434
 
Is the death somehow more tragic because it was caused by a computer error (if that’s what it was) rather than human error?

No just more stupid and preventable.

A person who try's to lick a rattle snakes death is just as tragic as a small kid who dies from SIDs, however the snake lickers death is far more stupid.
 
Seems like quite a deficiency...

Not really. Radar is just one of the sensors - and predominantly used to determine speed of other vehicles. It has its limits. People think it's magic since radar operates completely outside of human senses, so therefore it can do anything, but it's actually very limited.

All cars have additional sensors to spot other things. Tesla uses cameras and sonar as well. Waymo and Uber uses Lidar.

Currently Tesla only uses 2 out of their cars 8 cameras, so they don't have stereoscopic or surround vision and can't "see" all things. (Don't claim to be able to either).

So in the case of the Tesla crash into the firetruck it would be a camera/software failure, not a radar failure.
 
Last edited:
Anyone else see the news that squeaked out...? The car did see her but some brilliant engineer’s code tagged her as a false target.

They haven’t said what the car thought she was or which “stop the car from reacting” logic thread it went down.

Saw that, but I read that as the press not knowing what they're talking about again. It's possible that this was a coded algorithm, but I doubt it.

I think Uber initially said that it was a "software failure" flagging this as a false positive, and the press reported it as "software bug", and then it eventually turned into "coding bug".

But I doubt that this was coded - Uber uses deep learning. Nobody codes up a deep learning neural net (for that matter, nobody knows what's going on in them either). The failure here would be the dataset that was used to train the neural net probably made the women look more like a plastic bag than a pedestrian.
 
fatal_crash_rate.png
 

XKCD started out good but he has quite often removed important details from his most recent comics which belies that he’s gotten kinda lazy about it all.

The early comics, he wouldn’t have left out the scales on the X axis of those graphs. Or they’d at least be labeled something funny.

Still much love for anyone who came up with “sudo make me a sandwich” but he’s kinda phoning it in now.
 

The story I saw this morning said the emergency braking system had been intentionally disabled because of false positives. Too many emergency braking incidents because of overpasses, shadows, and other things. The data recorder inside the car shows the system apparently recognized the pedestrian, did calculate an intercept and did calculate a braking percentage to stop the vehicle. But the actual "apply the brakes" command was disabled. There was also no warning system to alert the human driver. I'll have to find that story again, but I *thought* it said the drivers were briefed on that limitation to the system.

edit:

I think this is the same story I read:

 
I'm guessing he does know which is the X axis and which is the Y axis, though ;)

Doesn’t matter for the point. He has other comics lately where he’s lazy like that on different axes.

But did you hear the point go ever whooshing as it passed over your head? :)
 
^agree with you on the xkcd stuff, years ago I would check the page religiously, but the last year or so he does seem to have lost some of his "knack" or whatever
 
If your emergency braking algorithm doesn't work you probably should stop road testing until it does.
 
Unfortunately, while the whole Tesla autopilot thing makes for an easy target and a nice news headline, you also will never replace human stupidity. For example, check out this product: https://www.autopilotbuddy.com/ developed by a Tesla driver, so that he can take full advantage of his autopilot

At least in the case of Tesla:
If your emergency braking algorithm doesn't work you probably should stop road testing until it does.
True.. even if the car "sees" stationary objects differently than moving ones (like road signs, etc.) it should be smart enough NOT to careen into them. At the same time though, shouldn't the ultimate "emergency braking algorithm" be the human driver? The marketing of calling it an "autopilot" is definitely suspect, but ultimately this is kind of more of a gimmick than a "real" autopilot. Plus, most people would much rather say "the autopilot caused the car wreck!" vs "I jammed a tennis ball in the steering wheel and was browsing Tinder for 20 minutes and didn't see my car hurtling towards a fire truck"

If I'm shooting an ILS in approach mode on AP but then completely zone out I shouldn't be surprised and attempt to sue Garmin / Cirrus when the plane starts on the missed and I lose all situational awareness, stall, and crash. Nor would I expect to see a headline "AUTOPILOT CAUSES PLANE TO CRASH DURING LANDING" (although maybe Daily Mail or Der Spiegel would run something like that)
 
Glad I live in a state where we don't value "freedom" over the general public being target practice for corporate experiments.
 
At the same time though, shouldn't the ultimate "emergency braking algorithm" be the human driver?

I thought this thread was about the fatality, not Tesla. That’s the other thread. :)

But I’d say Tesla has bigger problems with braking than their automated emergency braking, judging by two major test/review groups saying they couldn’t get a consistent stopping distance out of the cars WITH a driver mashing the brake pedal on a closed track and consistent braking surface.

Adding a human driver doesn’t help in their case.
 
^yikes. That's unsettling.
 
But I’d say Tesla has bigger problems with braking than their automated emergency braking, judging by two major test/review groups saying they couldn’t get a consistent stopping distance out of the cars WITH a driver mashing the brake pedal on a closed track and consistent braking surface.

Adding a human driver doesn’t help in their case.

Had. They fixed that already and rolled out the update to all the vehicles.
 
^yikes. That's unsettling.

It is, but it’s also not something that really affected anybody except if you take the car to a track.

How often do you slam your brake full out so much so that ABS engages on a dry road? Maybe once every ten years? The particular issue specifically was only if you did two ABS full out brakes in a short period of time (ABS calibration got messed up.)

Either way, still bad and messed up that they didn’t catch it before, and a recall level issue, which they did (over the air software update), but unlikely to have affected anyone.
 
Unfortunately, while the whole Tesla autopilot thing makes for an easy target and a nice news headline, you also will never replace human stupidity. For example, check out this product: https://www.autopilotbuddy.com/ developed by a Tesla driver, so that he can take full advantage of his autopilot

The product does disclaim that you still have to be in full control of the vehicle. It's for drivers who don't like the way Tesla detects if you're "paying attention", by detecting periodic torque from the driver against the steering wheel. And there's no legal or technological requirement for that either - for example a Cadillac Supercruise detects "paying attention" using a driver facing camera instead.

The Model 3 also has a driver-facing camera, which they plan to use for that as well (but isn't currently enabled).

These are all proxies though - no piece of technology can truly detect if a driver is paying attention. Even if it detects your eyes point towards the road, you can be mentally checked out. It would be nice when get out of this semi-assisted state of things we're in now to full autonomy, but you have to get through the one to get to the other.

People who want to be texting while driving isn't going to be deterred by either the Tesla or Cadillac system. I don't think this device will make people pay any more or less attention than they do now. You either accept you have to pay attention and do it, or you don't. The autopilotbuddy would likely not change things.
 
Had. They fixed that already and rolled out the update to all the vehicles.

Re-tested by the same third parties, or did they blacklist them for saying negative things about the Great Musk? LOL. I haven’t seen any updates from the third parties.
 
The marketing of calling it an "autopilot" is definitely suspect, but ultimately this is kind of more of a gimmick than a "real" autopilot. Plus, most people would much rather say "the autopilot caused the car wreck!" vs "I jammed a tennis ball in the steering wheel and was browsing Tinder for 20 minutes and didn't see my car hurtling towards a fire truck"

Interestingly enough, the car doesn't use the word "autopilot" anywhere in any of the UI. It has 2 feature names in the car:

1) TACC - this is good old Traffic Aware Cruise Control. Luxury cars have had it for over a decade. Heck, my Ford F350 has it.
2) AutoSteer Beta - this is the new tech that turns the steering wheel to keep the car in the lane.

However, each of the recent crashes that we've become aware of in the media recently was a TACC failure, not an AutoSteer failure. (I actually know of one AutoSteer failure, but it wasn't fatal and wasn't widely reported).

But how come we're only hearing about these now? Is it:

a) TACC has suddenly become less safe over the last 2 years? I doubt it.
-or-
b) The press and the availability heuristic bias? i.e. TACC isn't actually perfectly safe, and has killed hundreds (thousands?) of people since it was introduced. But because it was introduced as a minor incremental technology instead of a sexy new one with a scary sounding name, nobody cared or noticed.

Now that it's widely reported things suddenly things seem a lot scarier, even though nothing really changed risk-wise. Same reason most people are more scared about an airplane falling out of the sky on their house and killing them then they are of dying of a heart attack - availability heuristic bias.
 
Re-tested by the same third parties, or did they blacklist them for saying negative things about the Great Musk? LOL. I haven’t seen any updates from the third parties.

Consumer Reports promised Tuesday that they will retest it, and the fix came out yesterday. Patience young grasshopper.
 
Regardless on whether you hate the man or idolize him, he's not necessarily wrong here

To your point
Now that it's widely reported things suddenly things seem a lot scarier
Screenshot_20180527-170931.png

Granted.. "autopilot" is new tech and scrutiny is warranted.. but this is a good demonstration of the freakout culture..

..this tech could save lives one day from DUI, etc. I like ICE cars and driving (have always preferred proper clutch manual, etc), but the potential positive that come out of technology like this for the average A->B consumer who only cares about getting the cheapest monthly lease and best power train warranty (I'm looming at you Toyota Camry, Chevy Cruze, etc) should not immediately be vilified. That's what I don't understand, why people love to hate it automatically

It would be interesting to see what the fatality rate per autopilot mile driven is and compare that to human mile drive
 
Consumer Reports promised Tuesday that they will retest it, and the fix came out yesterday. Patience young grasshopper.

Wasn’t the original report of the problem by Car and Driver many months ago? CRs test was well after C&Ds test. The C&D editor was just leery of reporting it until CR confirmed it because he knows the PR department at Tesla would cut him off.
 
I remember when people that didn't want to drive themselves in their own car hired a chauffeur to drive for them.......
 
I remember when people that didn't want to drive themselves in their own car hired a chauffeur to drive for them.......

I remember when people enjoyed driving and didn’t need a top to bottom of dash touchscreen to entertain and distract themselves with in cars. Complete with a doodle pad, even... LOL. Everybody needs a car with a doodle pad that has cellular connectivity to send that crap to someone else in email, right? Hahaha.

But then again, most of the vehicles coming out these days all look and drive alike, and the suburban drones mostly got excited about Tesla because they’re different from the CAFE mandated drone purchased, cloned bubble cars.

And even Ford has now given up on those. Why build them when Korea will? Can’t make money competing with that.

Might as well build bubble SUVs for the older crowd willing to go WAY further into debt, and pickup trucks. More margin.

We were walking along a row of modern SUVs in the parking lot at the grocery store today. Jeep Grand Cherokee, Acura whatever-ending-in-X, Audi something-or-other, and three or four other brand names. Nissan and Dodge were also represented for sure. They were parked perfectly in a line in their spots.

You couldn’t tell other than by taillight LED layout which one was which. No wonder nobody can remember the names of the stupid things. Boooooooooring AF.

And yes, even the Ford... it had SLIGHTLY squared off bends which look like their signature corners so you could almost tell it was a Ford.

I know if you lined them up and said “Which one do you want to drive?” I’d yawn and take whatever key was on top of the pile. That’s for sure.
 
Doesn’t matter for the point. He has other comics lately where he’s lazy like that on different axes.

But did you hear the point go ever whooshing as it passed over your head? :)

The point I took away was the irony of someone who doesn't know which axis on a graph is which criticizing a physics grad and ex-NASA researcher for lack of attention to detail :)
 
The point I took away was the irony of someone who doesn't know which axis on a graph is which criticizing a physics grad and ex-NASA researcher for lack of attention to detail :)

Awww think his feelings are hurt that he’s slipping as a cartoonist? Since the other items have literally nothing to do with my comment or his performance as one?
 
Re-tested by the same third parties, or did they blacklist them for saying negative things about the Great Musk? LOL. I haven’t seen any updates from the third parties.

Here you go:

Tesla Model 3 Gets CR Recommendation After Braking Update
https://www.consumerreports.org/car...-gets-cr-recommendation-after-braking-update/


Also cool:

Until now, that type of remote improvement to a car’s basic functionality had been unheard of. “I’ve been at CR for 19 years and tested more than 1,000 cars,” says Jake Fisher, director of auto testing at Consumer Reports, “and I’ve never seen a car that could improve its track performance with an over-the-air update.”
 
Here you go:

Tesla Model 3 Gets CR Recommendation After Braking Update
https://www.consumerreports.org/car...-gets-cr-recommendation-after-braking-update/


Also cool:

Until now, that type of remote improvement to a car’s basic functionality had been unheard of. “I’ve been at CR for 19 years and tested more than 1,000 cars,” says Jake Fisher, director of auto testing at Consumer Reports, “and I’ve never seen a car that could improve its track performance with an over-the-air update.”

Translation: They fixed it like it should have been long before I had to hang it out there and say the popular kids were building total crap, and I need to spooge about the stuff the PR person used as an excuse -- if I ever hope to get another test car here from them, ever again.

:) :) :)
 
Translation: They fixed it like it should have been long before I had to hang it out there and say the popular kids were building total crap, and I need to spooge about the stuff the PR person used as an excuse -- if I ever hope to get another test car here from them, ever again.

:) :) :)

Consumer Reports do not get vehicles from manufacturers or even dealers - otherwise the manufacturer would just give them the best, hand-verified one. They purchase them anonymously. So manufacturers can't cut them off.
 
Back
Top