Automated vehicles susceptible to confusion via graffiti

[WMGroomAK]

Self-driving cars may have another hurdle to overcome before being reliable in the form of reading road signs accurately.  In an interesting article, researchers with the University of Washington have figured out that defacing road signs can pose a threat to a vehicles image recognition and can easily be disguised as graffiti or street art.  By judiciously placing stickers on a stop sign, they were able to get a computer vision algorithm to think it was looking at a speed limit sign instead and were able to confuse the algorithm into thinking a Turn Right sign was a stop sign 67% of the time and an Added Lane sign 33% of the time.  

https://www.engadget.com/2017/08/06/altered-street-signs-confuse-self-driving-cars/

Quote

While car makers and regulators are mostly worried about the possibility of self-driving car hacks, University of Washington researchers are concerned about a more practical threat: defacing street signs. They've learned that it's relatively easy to throw off an autonomous vehicle's image recognition system by strategically using stickers to alter street signs. If attackers know how a car classifies the objects it sees (such as target photos of signs), they can generate stickers that can trick the car into believing a sign really means something else. For instance, the "love/hate" graphics above made a computer vision algorithm believe a stop sign was really a speed limit notice.

 

It's easy to see the potential problems. You could make these stickers using a printer at home, so anyone from dedicated attackers to pranksters could try this. It might lead to a crash the moment someone alters the sign, but it could also produce long-term chaos -- picture your city closing a road until maintenance crews can scrape the stickers off a sign.

 

There are ways to fight this. The research team suggests using contextual information to verify that a sign is accurate. Why would you have a stop sign on the highway, or a high speed limit on a back road? We'd add that local governments could also install signs that use an anti-stick material, or put them out of reach. Whatever happens, something will have to change if passengers are going to trust self-driving cars' sign-reading abilities.

While I don't think this will present any immediate issues, it does bring up that autonomous vehicles should be looking at it's positioning more geographically in order to help determine what a sign might say.  

 

Technical Article: https://arxiv.org/pdf/1707.08945.pdf

 

[TAHIRMIA]

Guess we still have to wait a lot longer for fully self-driving cars.

[alexyy]

That's annoying but I imagine signs designed for use with automated vehicles could overcome this problem but then new signs alone is another issue. 

[HarryNyquist]

I feel like fully autonomous driving will be one of those "future technologies" for a very long time.

[hobobobo]

Does not seem like a big issue, all the signs will just be added to the digital maps and changed the same way they are now+a paper to the relevant agency to amend the data. Its just both lidar and radar are kinda mediocre tech for that and the real thing is the ai which basically tries to imitate fish/bat vision. As soon as it properly gets there, and by properly i mean meeting the gov standarts which will be made and being beyond safer and more efficient (ie irobot style), it will probably be mandated as the only option for a personal vehicle. And then the internet of cars kicks in and it really gets scifi. For the foreseeable future it will be a convenience to save you from your own stupidity and shit reaction time. Full automation of cars without outlawing "manual" ones will just lead to chaos and abuse on some scale

[Sniperfox47]

Before people jump on the hate train for self driving cars, there's a number of factors to keep in mind.

 

They didn't use any neural networks currently in use in a self driving car. Instead they trained their own.

 

They trained it on a relatively small set of data(the LISA sign database), and even then only used a very small amount of that data (500 pictures of each sign type).

 

All of the signs it was trained on were in pristine condition. It was not trained on any vandalized or defaced signs.

 

They used a white box model for designing their stickers, where they had access to the trained model and engineered a sticker based on that. This allowed them to design the threat at high speed. While it may be representative of such a threat being possible (since you can probe the system to verify your threat before implementing it), it would also be *much* harder and longer in a real world attack done by probing the model.

 

They used a smartphone camera with their image recognition model to simulate a car, not any actual in production car model. This means one camera instead of multiple cameras, and a much simpler model for inferencing compared to the models ran by self-driving car supercomputers.(edit: also no radar/lidar or infrared).

 

All in all this research paper indicates that such a threat is possible, but not that it's even a viable threat on existing self-driving cars that have a *much* wider and more diverse training pool.

[ETHREAL1]

You know if EVERY car became fully autonomous there would be no need for road signs when you think of it, they would just know ware they need to stop, but then you could also have a hive mind thing as well but then we would have skynet so...

[mynameisjuan]

Or people could just stop being assholes and not deface others property.

 

But people are assholes so guess no self driving cars anytime soon.

[TetraSky]

The way around this would be to have a truly "smart" city, where street signs and traffic lights can communicate with the autonomous cars wirelessly, so that a car knows what a sign is without even seeing it.

[Bananasplit_00]

Why even bother reading these signs? They should all just be in a database with a digital map. The only signs that should have to be read are things like road work and so on which are temporary 

[WMGroomAK]

1 hour ago, ETHREAL1 said:

You know if EVERY car became fully autonomous there would be no need for road signs when you think of it, they would just know ware they need to stop, but then you could also have a hive mind thing as well but then we would have skynet so...

Living in the environment that I do, I have to confess that full autonomy in vehicles may happen in larger communities, but will probably not be a global deal for a long time...  Mainly I see this when I'm looking at unimproved roads and jeep trails that are used to access various remote sites or roads that suffer from severe weather damage on a yearly basis to the point that even with constant upkeep a driver is needed to assess whether it is safe to drive at speed or slow down for road damage.

[BuckGup]

4 minutes ago, Bananasplit_00 said:

Why even bother reading these signs? They should all just be in a database with a digital map. The only signs that should have to be read are things like road work and so on which are temporary 

Yeah because what if some one hits it and there isn't one their temporarily?

[Cela1]

Just saying: graffiti on signs could also confuse humans, not in this specific way, but they can be altered or obscured anyways... Then someone comes along and cleans it.

[mr moose]

So, this just means autonomous cars will be just as capable of reading signs as most drivers today are.

[DeScruff]

Kinda predicted this...
I imagine it will also be an issue when a sign is old, rusted, and the text is unreadable, which maybe doesn't happen in a urban highly populated area, but in a small neighborhood in 'the middle of nowhere' happens all the time.

As for the "Why would a car need to read signs! they could work with just a database of the signs."  Ill give you a couple of reasons:

• Temporary Signs: - Like Road construction, or a School Bus, or a Traffic cop/crossing guard, that will not be on a database, and can change by the hour.
• Cause GPS map databases can become outdated and take months for such a fix to happen. - It took Google maps ~8 Months to finally realize a road near me no longer exists. The same would probably have happened with the new Stop sign near my house. (Its been there at least 3 months and people still drive past it without stopping, I almost got into an accident cause of that.)
• You can't expect every person to have a car with the most up to date database of road signs. Even if it has to be updated to pass inspections, that can be an update once a year depending on your Country/State's laws.
•  "Your Car is no longer able to receive Database updates." - Please buy a newer car. Like Android phones being unable to update, this will likely happen with cars.

[Mornincupofhate]

Couldn't google just map out all of the stop signs via google maps or something?

[Sniperfox47]

12 minutes ago, Sypran said:

I imagine it will also be an issue when a sign is old, rusted, and the text is unreadable, which maybe doesn't happen in a urban highly populated area, but in a small neighborhood in 'the middle of nowhere' happens all the time.

Google Cloud Vision is actually pretty good at identifying these, and that's not even a dedicated driving neural net. Even when the signs have holes in them it can pick them up pretty well.

 

With content signs like speed limits, turnoffs, and stuff of that nature it's obviously harder but most things can be taken contextually (GPS to determine turns) or sane safeties can be observed (eg. 50 on a city street if not around a school).

 

14 minutes ago, Sypran said:

As for the "Why would a car need to read signs! they could work with just a database of the signs."  Ill give you a couple of reasons:

• Temporary Signs: - Like Road construction, that will not be on a database, and can change by the hour.
• Cause GPS map databases can become outdated and take months for such a fix to happen. - It took Google maps ~8 Months to finally realize a road near me no longer exists. The same would probably have happened with the new Stop sign near my house. (Its been there at least 3 months and people still drive past it without stopping, I almost got into an accident cause of that.)
• You can't expect every person to have a car with the most up to date database of road signs. Even if it has to be updated to pass inspections, that can be an update once a year depending on your Country/State's laws.
•  "Your Car is no longer able to receive Database updates." - Please buy a newer car. Like Android phones being unable to update, this will likely happen with cars.

That's the thing though. Reading signs and a database aren't mutually exclusive. All of the current automated driving systems currently in play build and store a database, typically sharing it with other automated vehicles, based on their own computer vision. Then they use this database in combination with their computer vision to make decisions.

 

For example, if their lidar picks up a sign they've passed before but their cameras can't see what it says, they use their database to infer what the sign is.

 

As self-driving cars become more of a thing, it's likely we'll see standards bodies push for standardization of these so that they can be shared P2P between vehicles in areas the car isn't familiar with. There's no reason a database of known signs and hazards needs to come from the manufacturer.

[Jito463]

4 hours ago, ETHREAL1 said:

You know if EVERY car became fully autonomous

They can take my manual transmission from my burned up corpse in the vehicles fiery husk.  I plan to drive that truck until it dies (or I do).

[spartaman64]

18 hours ago, Sypran said:

Kinda predicted this...
I imagine it will also be an issue when a sign is old, rusted, and the text is unreadable, which maybe doesn't happen in a urban highly populated area, but in a small neighborhood in 'the middle of nowhere' happens all the time.

As for the "Why would a car need to read signs! they could work with just a database of the signs."  Ill give you a couple of reasons:

• Temporary Signs: - Like Road construction, or a School Bus, or a Traffic cop/crossing guard, that will not be on a database, and can change by the hour.
• Cause GPS map databases can become outdated and take months for such a fix to happen. - It took Google maps ~8 Months to finally realize a road near me no longer exists. The same would probably have happened with the new Stop sign near my house. (Its been there at least 3 months and people still drive past it without stopping, I almost got into an accident cause of that.)
• You can't expect every person to have a car with the most up to date database of road signs. Even if it has to be updated to pass inspections, that can be an update once a year depending on your Country/State's laws.
•  "Your Car is no longer able to receive Database updates." - Please buy a newer car. Like Android phones being unable to update, this will likely happen with cars.

google maps is already pretty good at knowing where there is construction and even where there is a crash. yesterday when i was driving back from chicago mid way it tells me there is a 8 min slow down due to construction and asked if i wanted to use a faster route

[Skanky Sylveon]

Good!  Good! (evil laugh)

[Jito463]

5 minutes ago, JayKay3000 said:

I'm told planes can fly themselves, but the pilot is needed in case of emergency.

Yes and no.  A plane still needs a pilot for landing/takeoff.  Also, flying in the air has a much smaller chance of hitting something than when driving on the ground, so it's relatively simple to set a course and just let the plane coast along.

[Shreyas1]

On 8/7/2017 at 11:50 AM, HarryNyquist said:

I feel like fully autonomous driving will be one of those "future technologies" for a very long time.

You mean like flying cars

[SSL]

Not really a significant issue in the long term. As self-driving cars become ubiquitous, I imagine that municipalities will work towards installing navigation aids tailored specifically for autonomous vehicles. Using image recognition to detect human-readable signs seems like a temporary workaround regardless of this hack.

[Satisfoxy]

I have mental problems that make it very difficult for me to drive a car. I want cars that drive themselves so I can actually get places without worry, but nooooo, boohoo we afraid of the robots nation.

 

People going to purposefully put stuff on roadsides because they don't like robot cars.