Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
spartaman64

Researchers trick tesla into going 85 mph autonomously with tape on speed limit sign

Recommended Posts

Posted · Original PosterOP
Quote

Hackers at McAfee Advanced Threat Research conducted the experiment.

They explain what they set out to do in a blog post:

McAfee Advanced Threat Research (ATR) has a specific goal: identify and illuminate a broad spectrum of threats in today’s complex landscape. With model hacking, the study of how adversaries could target and evade artificial intelligence, we have an incredible opportunity to influence the awareness, understanding and development of more secure technologies before they are implemented in a way that has real value to the adversary.

Quote

With that in mind, they decided to target MobilEye’s camera system since it’s deployed in over 40 million vehicles, including Tesla’s first-generating Autopilot vehicles, which were used for this specific experiment.

They decided to try to modify speed limit signs in ways that a human would be able to still know the limit, but the automated system could get confused:

Screen Shot 2020-02-19 at 8.51.16 AMScreen Shot 2020-02-19 at 8.51.10 AM

Quote

Ultimately, they were able to make a Tesla vehicle on Autopilot accelerate by 50 mph over the limit:

The ultimate finding here is that we were able to achieve the original goal. By making a tiny sticker-based modification to our speed limit sign, we were able to cause a targeted misclassification of the MobilEye camera on a Tesla and use it to cause the vehicle to autonomously speed up to 85 mph when reading a 35-mph sign. For safety reasons, the video demonstration shows the speed start to spike and TACC accelerate on its way to 85, but given our test conditions, we apply the brakes well before it reaches target speed. It is worth noting that this is seemingly only possible on the first implementation of TACC when the driver double taps the lever, engaging TACC. If the misclassification is successful, the autopilot engages 100% of the time. This quick demo video shows all these concepts coming together.

Quote

McAfee confirmed that it disclosed its findings to both Tesla and MobilEye before making them public:

McAfee disclosed the findings to Tesla on September 27th, 2019 and MobilEye on October 3rd, 2019. Both vendors indicated interest and were grateful for the research but have not expressed any current plans to address the issue on the existing platform. MobilEye did indicate that the more recent version(s) of the camera system address these use cases.

In previous instances of vulnerabilities being exposed by white-hat hackers, Tesla has been fairly quick to fix them.

source: https://electrek.co/2020/02/19/tesla-autopilot-tricked-accelerate-speed-limit-sign/

 

Idk why im posting so much tesla stuff i blame google news and their algorithms. Some people have pointed out that on a public road where the car can also get the speed limit from gps data this probably won't be as much of a problem. Just another reason why you should pay attention with autopilot on. Also afaik this only affects older teslas.

Link to post
Share on other sites

that first one at a quick glance I'd expect to get most people


Good luck, Have fun, Build PC, and have a last gen console for use once a year. I should answer most of the time between 9 to 3 PST

NightHawk 2.0: R7 2700 @4.0ghz, B450m Steel Legends, H105, 2x8gb Gell EVO 3200, XFX RX 580 8GB, Corsair RM750X, 500 gb 850 evo, 500gb 850 pro and 5tb Toshiba x300

Skunkworks: R5 3500U, 16gb, 250 intel 750, 500gb Adata XPG 6000 lite, Vega 8. HP probook G455R G6

Condor (MC server): 6600K, z170m plus, 16gb corsair vengeance LPX, samsung 750 evo, EVGA BR 450.

Bearcat (F@H box) core 2 duo, 1x4gb EEC DDR2, 250gb WD blue, 9800GTX+, STRIX 660ti, supermicro PSU, dell T3400.

Compute server Rappter(remember to add link) HP DL380G6 2xE5520 24GB ram with 4x146gb 10k drives and 4x300gb 10K drives, running NOTHING can't get anything to work

WIP NAS Spirt Cisco Security Multiservices Platform server e5420 12gb ram, 1x6 1tb raid 6 for plex + Need funding 16+1 2tb raid 6 for mass storage.

PSU Tier List      Motherboard Tier List      How to get PC parts cheap    HP probook 445R G6 review

 

Link to post
Share on other sites
Posted · Original PosterOP
2 minutes ago, handymanshandle said:

Is it gonna start stopping at Sop signs now?

 

if it was more octagonal probably

Link to post
Share on other sites

What happens when you accept data without context.


$potentialspeedlimit = OCR.GetSpeedLimitSign()
If $potentialspeedlimit > 55 and Is.Road("Highway") then $speedlimit = $potentialspeedlimit
 

Heck we can add some additional checks...

$potentialspeedlimit = OCR.GetSpeedLimitSign()
If $potentialspeedlimit > 55 and Is.Road("Highway") and ($potentialspeedlimit - 20)  ≤ $speedlimit then $speedlimit = $potentialspeedlimit


PLEASE QUOTE ME IF YOU ARE REPLYING TO ME
LinusWare Dev | NotCPUCores Dev

Desktop Build: Ryzen 7 1800X @ 4.0GHz, AsRock Fatal1ty X370 Professional Gaming, 32GB Corsair DDR4 @ 3000MHz, RX480 8GB OC, Benq XL2730 1440p 144Hz FS

Retro Build: Intel Pentium III @ 500 MHz, Dell Optiplex G1 Full AT Tower, 768MB SDRAM @ 133MHz, Integrated Graphics, Generic 1024x768 60Hz Monitor


 

Link to post
Share on other sites

is there a situation where the autocar has to refer to the camera for speed limit guidance rather than using map data of the country itself? o_o
 

i'm thinking loss of GPS signal, but even so i'm kinda sure the car can estimate where it's been with velocity sensors and speedometer readouts etc

Link to post
Share on other sites

Can we really count sign vandalism as a problem? I mean, what if it were completely spray painted black? Would a human know how fast they can go in that case?

 

To improve the autopilot Tesla could make it so that if the uncertainty of the sign recognition is above a certain level the car will double check with GPS and a map before accelerating, or go with the lowest of the two speed limits (wouldn't want it to ignore a temporary slow down sign because the map wasn't updated in time).


...is there a question here? ?

sudo chmod -R 000 /*

What is scaling and how does it work? Asus PB287Q unboxing! Console alternatives :D Watch Netflix with Kodi on Arch Linux Sharing folders over the internet using SSH Beginner's Guide To LTT (by iamdarkyoshi)

Sauron'stm Product Scores:

Spoiler

Just a list of my personal scores for some products, in no particular order, with brief comments. I just got the idea to do them so they aren't many for now :)

Don't take these as complete reviews or final truths - they are just my personal impressions on products I may or may not have used, summed up in a couple of sentences and a rough score. All scores take into account the unit's price and time of release, heavily so, therefore don't expect absolute performance to be reflected here.

 

-Lenovo Thinkpad X220 - [8/10]

Spoiler

A durable and reliable machine that is relatively lightweight, has all the hardware it needs to never feel sluggish and has a great IPS matte screen. Downsides are mostly due to its age, most notably the screen resolution of 1366x768 and usb 2.0 ports.

 

-Apple Macbook (2015) - [Garbage -/10]

Spoiler

From my perspective, this product has no redeeming factors given its price and the competition. It is underpowered, overpriced, impractical due to its single port and is made redundant even by Apple's own iPad pro line.

 

-OnePlus X - [7/10]

Spoiler

A good phone for the price. It does everything I (and most people) need without being sluggish and has no particularly bad flaws. The lack of recent software updates and relatively barebones feature kit (most notably the lack of 5GHz wifi, biometric sensors and backlight for the capacitive buttons) prevent it from being exceptional.

 

-Microsoft Surface Book 2 - [Garbage - -/10]

Spoiler

Overpriced and rushed, offers nothing notable compared to the competition, doesn't come with an adequate charger despite the premium price. Worse than the Macbook for not even offering the small plus sides of having macOS. Buy a Razer Blade if you want high performance in a (relatively) light package.

 

-Intel Core i7 2600/k - [9/10]

Spoiler

Quite possibly Intel's best product launch ever. It had all the bleeding edge features of the time, it came with a very significant performance improvement over its predecessor and it had a soldered heatspreader, allowing for efficient cooling and great overclocking. Even the "locked" version could be overclocked through the multiplier within (quite reasonable) limits.

 

-Apple iPad Pro - [5/10]

Spoiler

A pretty good product, sunk by its price (plus the extra cost of the physical keyboard and the pencil). Buy it if you don't mind the Apple tax and are looking for a very light office machine with an excellent digitizer. Particularly good for rich students. Bad for cheap tinkerers like myself.

 

 

Link to post
Share on other sites
Just now, VegetableStu said:

is there a situation where the autocar has to refer to the camera for speed limit guidance rather than using map data of the country itself? o_o

Arguably, always. The signs you actually come across should have precedence over what the map says. With that said, if the car isn't sure of what the sign says and the map says 35 then it makes sense to drive 35 - it's not a crime to drive slower than the limit.


...is there a question here? ?

sudo chmod -R 000 /*

What is scaling and how does it work? Asus PB287Q unboxing! Console alternatives :D Watch Netflix with Kodi on Arch Linux Sharing folders over the internet using SSH Beginner's Guide To LTT (by iamdarkyoshi)

Sauron'stm Product Scores:

Spoiler

Just a list of my personal scores for some products, in no particular order, with brief comments. I just got the idea to do them so they aren't many for now :)

Don't take these as complete reviews or final truths - they are just my personal impressions on products I may or may not have used, summed up in a couple of sentences and a rough score. All scores take into account the unit's price and time of release, heavily so, therefore don't expect absolute performance to be reflected here.

 

-Lenovo Thinkpad X220 - [8/10]

Spoiler

A durable and reliable machine that is relatively lightweight, has all the hardware it needs to never feel sluggish and has a great IPS matte screen. Downsides are mostly due to its age, most notably the screen resolution of 1366x768 and usb 2.0 ports.

 

-Apple Macbook (2015) - [Garbage -/10]

Spoiler

From my perspective, this product has no redeeming factors given its price and the competition. It is underpowered, overpriced, impractical due to its single port and is made redundant even by Apple's own iPad pro line.

 

-OnePlus X - [7/10]

Spoiler

A good phone for the price. It does everything I (and most people) need without being sluggish and has no particularly bad flaws. The lack of recent software updates and relatively barebones feature kit (most notably the lack of 5GHz wifi, biometric sensors and backlight for the capacitive buttons) prevent it from being exceptional.

 

-Microsoft Surface Book 2 - [Garbage - -/10]

Spoiler

Overpriced and rushed, offers nothing notable compared to the competition, doesn't come with an adequate charger despite the premium price. Worse than the Macbook for not even offering the small plus sides of having macOS. Buy a Razer Blade if you want high performance in a (relatively) light package.

 

-Intel Core i7 2600/k - [9/10]

Spoiler

Quite possibly Intel's best product launch ever. It had all the bleeding edge features of the time, it came with a very significant performance improvement over its predecessor and it had a soldered heatspreader, allowing for efficient cooling and great overclocking. Even the "locked" version could be overclocked through the multiplier within (quite reasonable) limits.

 

-Apple iPad Pro - [5/10]

Spoiler

A pretty good product, sunk by its price (plus the extra cost of the physical keyboard and the pencil). Buy it if you don't mind the Apple tax and are looking for a very light office machine with an excellent digitizer. Particularly good for rich students. Bad for cheap tinkerers like myself.

 

 

Link to post
Share on other sites
Just now, Sauron said:

Arguably, always. The signs you actually come across should have precedence over what the map says. With that said, if the car isn't sure of what the sign says and the map says 35 then it makes sense to drive 35 - it's not a crime to drive slower than the limit.

surely google maps (for example) has speed limit data for (almost) every road out there? o_o i mean i'm with you for going to default street speed when unsure

Link to post
Share on other sites
16 minutes ago, VegetableStu said:

surely google maps (for example) has speed limit data for (almost) every road out there?

Yes, but it's not always up to date, particularly when there are temporary changes. On the other hand you can always trust signs (if they're wrong you can contest the fine, not so with GM).


...is there a question here? ?

sudo chmod -R 000 /*

What is scaling and how does it work? Asus PB287Q unboxing! Console alternatives :D Watch Netflix with Kodi on Arch Linux Sharing folders over the internet using SSH Beginner's Guide To LTT (by iamdarkyoshi)

Sauron'stm Product Scores:

Spoiler

Just a list of my personal scores for some products, in no particular order, with brief comments. I just got the idea to do them so they aren't many for now :)

Don't take these as complete reviews or final truths - they are just my personal impressions on products I may or may not have used, summed up in a couple of sentences and a rough score. All scores take into account the unit's price and time of release, heavily so, therefore don't expect absolute performance to be reflected here.

 

-Lenovo Thinkpad X220 - [8/10]

Spoiler

A durable and reliable machine that is relatively lightweight, has all the hardware it needs to never feel sluggish and has a great IPS matte screen. Downsides are mostly due to its age, most notably the screen resolution of 1366x768 and usb 2.0 ports.

 

-Apple Macbook (2015) - [Garbage -/10]

Spoiler

From my perspective, this product has no redeeming factors given its price and the competition. It is underpowered, overpriced, impractical due to its single port and is made redundant even by Apple's own iPad pro line.

 

-OnePlus X - [7/10]

Spoiler

A good phone for the price. It does everything I (and most people) need without being sluggish and has no particularly bad flaws. The lack of recent software updates and relatively barebones feature kit (most notably the lack of 5GHz wifi, biometric sensors and backlight for the capacitive buttons) prevent it from being exceptional.

 

-Microsoft Surface Book 2 - [Garbage - -/10]

Spoiler

Overpriced and rushed, offers nothing notable compared to the competition, doesn't come with an adequate charger despite the premium price. Worse than the Macbook for not even offering the small plus sides of having macOS. Buy a Razer Blade if you want high performance in a (relatively) light package.

 

-Intel Core i7 2600/k - [9/10]

Spoiler

Quite possibly Intel's best product launch ever. It had all the bleeding edge features of the time, it came with a very significant performance improvement over its predecessor and it had a soldered heatspreader, allowing for efficient cooling and great overclocking. Even the "locked" version could be overclocked through the multiplier within (quite reasonable) limits.

 

-Apple iPad Pro - [5/10]

Spoiler

A pretty good product, sunk by its price (plus the extra cost of the physical keyboard and the pencil). Buy it if you don't mind the Apple tax and are looking for a very light office machine with an excellent digitizer. Particularly good for rich students. Bad for cheap tinkerers like myself.

 

 

Link to post
Share on other sites
45 minutes ago, Sauron said:

Arguably, always. The signs you actually come across should have precedence over what the map says. With that said, if the car isn't sure of what the sign says and the map says 35 then it makes sense to drive 35 - it's not a crime to drive slower than the limit.

Yep.

 

As a human driver you should already be somewhat aware of the legal speed limit on a given road. If you saw a 50 sign in the middle of a suburb for example you'd know it probably wasn't correct. The car doesn't have that knowledge so it makes sense to include a fallback in the system. It seems like the obvious solution is to tell the car what speed limit signs actually exist around the world and instruct it to ignore any other numbers it encounters, heck they could include a database of images and have the car cross check the sign against a known image before it increases or decreases speed. The taking a slegdehammer to crack a walnut approach is to simply ask the driver if the sign as accurate before increasing speed.


Main Rig:-

Ryzen 7 3800X | Asus ROG Strix X570-F Gaming | 16GB Team Group Dark Pro 3600Mhz | Samsung 970 Evo 500GB NVMe | Sapphire 5700 XT Pulse | Corsair H115i Platinum | WD Black 1TB | WD Green 4TB | EVGA SuperNOVA G3 650W | Asus TUF GT501 | Samsung C27HG70 1440p 144hz HDR FreeSync 2 | Windows 10 Pro X64 |

 

Server:-

Raspberry Pi 4 Model B running OMV Arrakis and an 8TB Seagate USB 3.0 external HDD

Link to post
Share on other sites
57 minutes ago, VegetableStu said:

surely google maps (for example) has speed limit data for (almost) every road out there? o_o i mean i'm with you for going to default street speed when unsure

There are many roads and state highways in the U.S. that they don't, however, with that said some of those roads you probably wouldn't want to use autopilot on.

Link to post
Share on other sites

Tbh i didnt even think they worked off speed limit signs. 

 

My GPS somehow knows the speed limit on every road im on so i figured tesla would work like that using satilites or something.

 

I wish they would ban these autopilot features on public roads. Innocent people shouldnt be at risk because tesla wants their customers to be their test animals.

 

How many people have been in accidents over autopilot now?

 

Outlaw the shit until its ironed out. 

Link to post
Share on other sites
1 hour ago, Sauron said:

...it's not a crime to drive slower than the limit.

Actually, in some places it is. Some have a posted minimum speed limit ,some may say you have to be going no less than a certain amount below the speed traffic is going, others may nail you for obstructing traffic, driving at an unsafe speed, etc.


Jeannie

 

As long as anyone is oppressed, no one will be safe and free.

One has to be proactive, not reactive, to ensure the safety of one's data so backup your data! And RAID is NOT a backup!

 

Link to post
Share on other sites
1 hour ago, spartaman64 said:

Some people have pointed out that on a public road where the car can also get the speed limit from gps data this probably won't be as much of a problem

my car has this feature, it's a good safety feature but dear god is it annoying.

*you are over the speed limit by *5* kilometers*

I FUCKING KNOW!!!!! IM ON A STREET WITH NO ONE ELSE ON IT, WHO THE FUCK AM I GONNA HIT?!?!?!?!?!??!


*Insert Witty Signature here*

System Config: https://au.pcpartpicker.com/list/yJ2cQV

5U The Waifu (my new in-progress server): https://linustechtips.com/main/topic/1130931-5u-the-waifu-my-new-server/

 

Link to post
Share on other sites
Posted · Original PosterOP
10 minutes ago, RonnieOP said:

Tbh i didnt even think they worked off speed limit signs. 

 

My GPS somehow knows the speed limit on every road im on so i figured tesla would work like that using satilites or something.

 

I wish they would ban these autopilot features on public roads. Innocent people shouldnt be at risk because tesla wants their customers to be their test animals.

 

How many people have been in accidents over autopilot now?

 

Outlaw the shit until its ironed out. 

only the older teslas

Link to post
Share on other sites
5 minutes ago, Salv8 (sam) said:

my car has this feature, it's a good safety feature but dear god is it annoying.

*you are over the speed limit by *5* kilometers*

I FUCKING KNOW!!!!! IM ON A STREET WITH NO ONE ELSE ON IT, WHO THE FUCK AM I GONNA HIT?!?!?!?!?!??!

Yeah. My gps blinks red when i go even 3 miles over the speed limit.

 

Drives me crazy seeing it blink. 

 

I know the speed limits 65mph. But going 75 im holding traffic...stop blinking at me lol

Link to post
Share on other sites

I find it curious how easy A.I can find some tasks that are borderline impossible for humans, but can get floored by others that we find trivial.  

Link to post
Share on other sites

Bugs in this stuff are likely to keep on being found.  They’ll get harder though.  It’s how bugs seem to work, mostly.

This happened to UNIX too back in the day.  The OS is considered insanely stable not because it was bugless but because it’s bugs were found and stomped out.
 People are still finding bugs in intel stuff but they’re getting more and more esoteric.


Life is like a bowl of chocolates: there are all these little crinkly paper cups everywhere.

Link to post
Share on other sites

@spartaman64

 

Stolen from Reddit:
 

Quote

This isn't even a Tesla system. They're using the old MobileEye cars from 2016. MobileEye would need to fix it then Tesla would have to push a software update.

The current generation Tesla AP/FSD uses OpenStreetMap data for speed limits.

This seems very clickbaity in the fact that it doesn't mention Tesla hasn't used MobileEye's AP system in almost 5 years.

 


PLEASE QUOTE ME IF YOU ARE REPLYING TO ME
LinusWare Dev | NotCPUCores Dev

Desktop Build: Ryzen 7 1800X @ 4.0GHz, AsRock Fatal1ty X370 Professional Gaming, 32GB Corsair DDR4 @ 3000MHz, RX480 8GB OC, Benq XL2730 1440p 144Hz FS

Retro Build: Intel Pentium III @ 500 MHz, Dell Optiplex G1 Full AT Tower, 768MB SDRAM @ 133MHz, Integrated Graphics, Generic 1024x768 60Hz Monitor


 

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


×