Uber self driving car accident caused by design flaws

[spartaman64]

Quote

At 1.3 seconds before impact, the self-driving system determined that an emergency braking maneuver was needed to mitigate a collision. According to Uber, emergency braking maneuvers are not enabled while the vehicle is under computer control, to reduce the potential for erratic vehicle behavior. The vehicle operator is relied on to intervene and take action. The system is not designed to alert the operator.

1

Quote

Dashcam footage of the driver looking down at her lap has prompted a lot of speculation that she was looking at a smartphone. But the driver told the NTSB that she was actually looking down at a touchscreen that was used to monitor the self-driving car software.

"The operator is responsible for monitoring diagnostic messages that appear on an interface in the center stack of the vehicle dash and tagging events of interest for subsequent review," the report said.

This is a fail on so many levels. When the software detects something is wrong it doesn't alert the driver and the driver is required to monitor messages that appear on a screen at her lap. She shouldn't be required to do something that distracts from driving and even then why put the screen somewhere that requires the driver to look down? When the news initially broke I thought it was the driver's fault for being distracted by at the time what I thought was her cell phone but now I think its uber's fault and she was just doing what they told her to do.

https://arstechnica.com/cars/2018/05/emergency-brakes-were-disabled-by-ubers-self-driving-software-ntsb-says/

[Froody129]

And now comes a big question surrounding self driving cars: who is responsible? The programmers? The 'driver'? 

[JoostinOnline]

Just now, Froody129 said:

And now comes a big question surrounding self driving cars: who is responsible? The programmers? The 'driver'? 

Not the driver IMO.  Even if she'd been looking (which of course she should have been), there's no way she could react in time.  It was like the biker just appeared out of nowhere.

[spartaman64]

1 minute ago, Froody129 said:

And now comes a big question surrounding self driving cars: who is responsible? The programmers? The 'driver'? 

uber imo and if they decide to punish any programmers for it then its on the company

[19_blackie_73]

there's a reason I like to drive myself - at least I know I fucked up and I am the one to blame.

[Bcat00]

Give it time to mature, nothing comes out perfect on the first try. 

Car designs took decades to get to where we are now, even then its not flawless.

 

I personally love the idea of having AI supported driving, its about time driving became easier.

[mr moose]

48 minutes ago, Bcat00 said:

I personally love the idea of having AI supported driving, its about time driving became easier.

I am a lot more comfortable with AI supported driving rather than driver supported AI. 

 

Maybe in the future when we have better AI and better scanners etc will AI be suitable for today's roads.  Unfortunately it appears to me as if today's technology would only work on tomorrows roads.

[dalekphalm]

1 hour ago, JoostinOnline said:

Not the driver IMO.  Even if she'd been looking (which of course she should have been), there's no way she could react in time.  It was like the biker just appeared out of nowhere.

I initially held the same opinion as you stated here. But it's wrong. The dash cam used by the Uber car is complete garbage. We've seen other dash cam of the exact same location, at a similar time of night, and you can see for blocks.

 

The driver likely could have seen her at least a few seconds before she became "visible" in the footage.

[dalekphalm]

3 minutes ago, huilun02 said:

So who's at fault if a commercial airliner slams into the car ar 500kmh? I'll assume thats what the accident was about until I get a clear explanation of what actually went down. Also the dashcam video from all involved vehicles would be nice.

Woman crossed a multi-lane "parkway-style" divided in-city road at night, outside of designated crossing sections, while walking a bicycle across the road.

 

Dashcam of the Uber self-driving car didn't show the woman until she was about 1.5 seconds from impact. There was a deeply "shadowed" section of the road, and she basically appeared out of darkness. There was a working streetlight nearby.

 

However, the dashcam is of terrible quality, with extreme light/dark contrast issues. Other dashcam footage of the same area, at the same time of day, shows that you can see for blocks, and that there are literally no dark areas at all.

 

So, according to this update, the sensors did indeed detect the woman, and told the car to engage emergency car mode at 1.5s before impact. However, emergency stop mode was disabled by Uber programmers, due to the nature of their system.

[Crunchy Dragon]

1.5 seconds before impact is way too late to avoid an accident. IMO, you would need a good 3-4 seconds to come to a complete stop.

 

Also the human operator should also be paying attention to the road. It's a self-driving car test, you shouldn't expect it to work perfectly the first time.

[dalekphalm]

1 minute ago, Crunchy Dragon said:

1.5 seconds before impact is way too late to avoid an accident. IMO, you would need a good 3-4 seconds to come to a complete stop.

 

Also the human operator should also be paying attention to the road. It's a self-driving car test, you shouldn't expect it to work perfectly the first time.

In the original thread, I actually did the math, working out exactly how much time a human operator would need. 1.5s wouldn't be enough for a human, but would definitely have been enough time to reduce speed of the vehicle to near crawling speeds, if handled entirely by the quick reaction time of the onboard computer.

 

Might not have been enough to stop, but it would be enough to potentially reduce severity of injury, probably preventing death.

[Lady Fitzgerald]

1 minute ago, huilun02 said:

...Also the dashcam video from all involved vehicles would be nice.

The dash cam video has been shown on the news multiple times and I'm sure it can be found online. The video doesn't tell the entire story, though. It does show that the driver would not have had time to react after the pedestrian became visible. What's not shown is that the car's sytem should have been able to detect the pedestrian before the vehicle's headlights lit up the pedestrian enough to be visible nor that  the vehicle deliberately wasn't programmed to respond quickly enough so it could avoid false positive detections.

 

Other than the fact someone died (which is always horrific), what's sad about this accident is that a normal vehicle would not have been able to stop in time yet Uber is taking the full heat for this (never mind that the victim was jaywalking in a poorly lit stretch of roadway that gets little nightime foot traffic; I live nearby and know the area well). Of course, Uber has been found to have other problems, some serious, that need addressing but that wasn't the problem here.

 

Even sadder is Govenor Doug "Doozey" first welcomed Uber with open arms but, to avoid taking heat for that, effectively shut Uber's testing down. No small wonder Uber pulled out of AZ, eliminating 300 jobs.

[Lady Fitzgerald]

37 minutes ago, dalekphalm said:

I initially held the same opinion as you stated here. But it's wrong. The dash cam used by the Uber car is complete garbage. We've seen other dash cam of the exact same location, at a similar time of night, and you can see for blocks.

 

The driver likely could have seen her at least a few seconds before she became "visible" in the footage.

I live near there and I'm familiar with the area at all times day or night. The dash cam was not at fault. Even if the driver had been watching the road, she would not have been able to respond in time. I'm not saying Uber isn't at least partially responsible--their systems could have "seen" the pedestrian sooner than a human could have--but there were other factors involved as well.

[dalekphalm]

20 minutes ago, Lady Fitzgerald said:

The dash cam video has been shown on the news multiple times and I'm sure it can be found online. The video doesn't tell the entire story, though. It does show that the driver would not have had time to react after the pedestrian became visible. What's not shown is that the car's sytem should have been able to detect the pedestrian before the vehicle's headlights lit up the pedestrian enough to be visible nor that  the vehicle deliberately wasn't programmed to respond quickly enough so it could avoid false positive detections.

 

Other than the fact someone died (which is always horrific), what's sad about this accident is that a normal vehicle would not have been able to stop in time yet Uber is taking the full heat for this (never mind that the victim was jaywalking in a poorly lit stretch of roadway that gets little nightime foot traffic; I live nearby and know the area well). Of course, Uber has been found to have other problems, some serious, that need addressing but that wasn't the problem here.

 

Even sadder is Govenor Doug "Doozey" first welcomed Uber with open arms but, to avoid taking heat for that, effectively shut Uber's testing down. No small wonder Uber pulled out of AZ, eliminating 300 jobs.

Take a look at this:

Go to the 8 minute mark. He shows some side-by-side footage a dashcam, vs a different model dashcam.

 

Take a look again at the 9:13 mark, and he shows another comparison - this one includes the original Volvo Uber dashcam video. Specifically, look at the 9:24 mark, where the Uber camera first picks up the pedestrian who was killed. In the alternate dashcam view to the left, you can see clear straight through to the side of the street, and up blocks ahead. You can see the side of the road, and you can entirely see where the "dark space" is.

 

Keep in mind that humans can typically see better at night than most non-IR cameras.

 

12 minutes ago, Lady Fitzgerald said:

I live near there and I'm familiar with the area at all times day or night. The dash cam was not at fault. Even if the driver had been watching the road, she would not have been able to respond in time. I'm not saying Uber isn't at least partially responsible--their systems could have "seen" the pedestrian sooner than a human could have--but there were other factors involved as well.

That's hard to believe based on the available evidence. When the pedestrian was detected by the Dashcam, she was more than halfway across the road, moving pretty slowly.

 

Had the driver been in control and paying attention, avoiding the accident or mitigating the severity of the injury is definitely possible, and maybe even be probable.

EDIT: for clarity, I still don't blame the operator. She was given specific instructions, and the Uber system also malfunctioned. My above conclusion is based on if she been driving under regular/non-autonomous conditions.

[mr moose]

5 minutes ago, dalekphalm said:

Take a look at this:

Go to the 8 minute mark. He shows some side-by-side footage a dashcam, vs a different model dashcam.

 

Take a look again at the 9:13 mark, and he shows another comparison - this one includes the original Volvo Uber dashcam video. Specifically, look at the 9:24 mark, where the Uber camera first picks up the pedestrian who was killed. In the alternate dashcam view to the left, you can see clear straight through to the side of the street, and up blocks ahead. You can see the side of the road, and you can entirely see where the "dark space" is.

 

Keep in mind that humans can typically see better at night than most non-IR cameras.

 

That's hard to believe based on the available evidence. When the pedestrian was detected by the Dashcam, she was more than halfway across the road, moving pretty slowly.

 

Had the driver been in control and paying attention, avoiding the accident or mitigating the severity of the injury is definitely possible, and maybe even be probable.

It really does appear Ubber are trying hard to hide the fact their cars aren't ready for the road.  Regardless of whether that readiness is down to the driver, software or the hardware itself, not ready is not ready.

[ARikozuM]

2 hours ago, Froody129 said:

And now comes a big question surrounding self driving cars: who is responsible? The programmers? The 'driver'? 

The driver should still be responsible. The punishment should be less as there was supposed to be a system for early detection of possible threats, but punishment should still be served. 

[Lady Fitzgerald]

16 minutes ago, dalekphalm said:

Take a look at this:

Go to the 8 minute mark. He shows some side-by-side footage a dashcam, vs a different model dashcam.

 

Take a look again at the 9:13 mark, and he shows another comparison - this one includes the original Volvo Uber dashcam video. Specifically, look at the 9:24 mark, where the Uber camera first picks up the pedestrian who was killed. In the alternate dashcam view to the left, you can see clear straight through to the side of the street, and up blocks ahead. You can see the side of the road, and you can entirely see where the "dark space" is.

 

Keep in mind that humans can typically see better at night than most non-IR cameras.

 

That's hard to believe based on the available evidence. When the pedestrian was detected by the Dashcam, she was more than halfway across the road, moving pretty slowly.

 

Had the driver been in control and paying attention, avoiding the accident or mitigating the severity of the injury is definitely possible, and maybe even be probable.

EDIT: for clarity, I still don't blame the operator. She was given specific instructions, and the Uber system also malfunctioned. My above conclusion is based on if she been driving under regular/non-autonomous conditions.

Again, i have driven though that area at night dozens, if not hundreds of times over several years and you cannot see all that well.

[mr moose]

6 minutes ago, ARikozuM said:

The driver should still be responsible. The punishment should be less as there was supposed to be a system for early detection of possible threats, but punishment should still be served. 

The company should be responsible,  they put a driver in the car and told them what to do, they developed and signed off on the software and they chose the hardware. They put the car on the road and so far I have not seen SOP or evidence it was ignored.  I hate it when companies make scapegoats of single workers, it undermines when a single worker is genuinely at fault. 

[mr moose]

1 minute ago, Lady Fitzgerald said:

Again, i have driven though that area at night dozens, if not hundreds of times over several years and you cannot see all that well.

The hard bit for us to accept is the footage that clearly shows visibility is good.  If a camera can see that clearly then so can the human eye.  It is very hard to dispute the uber footage as either being tampered with or just plain extremely low grade.

[ARikozuM]

Just now, mr moose said:

The company should be responsible,  they put a driver in the car and told them what to do, they developed and signed off on the software and they chose the hardware. They put the car on the road and so far I have not seen SOP or evidence it was ignored.  I hate it when companies make scapegoats of single workers, it undermines when a single worker is genuinely at fault. 

That's a fair point, but it shouldn't absolve the driver of all responsibility. If the company can't back up their claim, they shouldn't make the claim. Problem with this is that they won't compete and will drop out of the competition. 

[Bcat00]

1 hour ago, mr moose said:

I am a lot more comfortable with AI supported driving rather than driver supported AI. 

 

Maybe in the future when we have better AI and better scanners etc will AI be suitable for today's roads.  Unfortunately it appears to me as if today's technology would only work on tomorrows roads.

Not really, the scanners and stuff are fine. It definitely require more tweaking on the programming side that's fore sure, which is why this happened. 

Like for real, whoever decides to not put the break code into the UBER car when it needs to is just plain silly.

[dalekphalm]

3 minutes ago, Lady Fitzgerald said:

Again, i have driven though that area at night dozens, if not hundreds of times over several years and you cannot see all that well.

So, are you saying that you can see worse than the dashcam footage I linked? Because according to that footage, you can see fairly well.

 

Had the woman actually stepped onto the road 1.5s from impact? I would believe you - without hesitation. But she was probably on the road for 15-20s or more.

[mr moose]

Just now, ARikozuM said:

That's a fair point, but it shouldn't absolve the driver of all responsibility. If the company can't back up their claim, they shouldn't make the claim. Problem with this is that they won't compete and will drop out of the competition. 

Is that really a problem?  I mean that's the nature of business, if you can't produce a safe product then you shouldn't be producing at all. (by safe I mean in all aspects, user control/instructions as well as hardware).

 

2 minutes ago, Bcat00 said:

Not really, the scanners and stuff are fine. It definitely require more tweaking on the programming side that's fore sure, which is why this happened. 

Like for real, whoever decides to not put the break code into the UBER car when it needs to is just plain silly.

So not ready.

[Lady Fitzgerald]

6 minutes ago, dalekphalm said:

So, are you saying that you can see worse than the dashcam footage I linked? Because according to that footage, you can see fairly well.

 

Had the woman actually stepped onto the road 1.5s from impact? I would believe you - without hesitation. But she was probably on the road for 15-20s or more.

Yes and no. I have an older camera than can see in the dark better than I can and, other than Prysbiopia (which only affects focus, not night vision, and I wear trifocals for that) so it's not a far stretch to believe some dash cams can see better in the dark than humans.

[Bcat00]

2 minutes ago, mr moose said:

Is that really a problem?  I mean that's the nature of business, if you can't produce a safe product then you shouldn't be producing at all. (by safe I mean in all aspects, user control/instructions as well as hardware).

 

So not ready.

Never said it was ready, i did say it needs more work. The technology is there, the programs not quite.

Give it a few more years before we get a proper AI program that can handle everything.