Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
williamcll

First step to self-awareness - Google AI designs its own chip

Recommended Posts

Posted · Original PosterOP

tpu-19bkw9.max-500x500.PNG

Google's own Artificial intelligence has managed to create it's own Tensor processor after less than a day of training.

Quote

Ideally you want a chip that’s optimized to do today’s AI, not the AI of two to five years ago. Google’s solution: have an AI design the AI chip. “We believe that it is AI itself that will provide the means to shorten the chip design cycle, creating a symbiotic relationship between hardware and AI, with each fueling advances in the other,” they write in a paper describing the work that posted today to Arxiv. “We have already seen that there are algorithms or neural network architectures that… don’t perform as well on existing generations of accelerators, because the accelerators were designed like two years ago, and back then these neural nets didn't exist,” says Azalia Mirhoseini, a senior research scientist at Google. “If we reduce the design cycle, we can bridge the gap.”

 

Mirhoseini and senior software engineer Anna Goldie have come up with a neural network that learn to do a particularly time-consuming part of design called placement. After studying chip designs long enough, it can produce a design for a Google Tensor Processing Unit in less than 24 hours that beats several weeks-worth of design effort by human experts in terms of power, performance, and area. Placement is so complex and time-consuming because it involves placing blocks of logic and memory or clusters of those blocks called macros in such a way that power and performance are maximized and the area of the chip is minimized. Heightening the challenge is the requirement that all this happen while at the same time obeying rules about the density of interconnects. Goldie and Mirhoseini targeted chip placement, because even with today’s advanced tools, it takes a human expert weeks of iteration to produce an acceptable design.

 

Goldie and Mirhoseini modeled chip placement as a reinforcement learning problem. Reinforcement learning systems, unlike typical deep learning, do not train on a large set of labeled data. Instead, they learn by doing, adjusting the parameters in their networks according to a reward signal when they succeed. In this case, the reward was a proxy measure of a combination of power reduction, performance improvement, and area reduction. As a result, the placement-bot becomes better at its task the more designs it does.

The team hopes AI systems like theirs will lead to the design of “more chips in the same time period, and also chips that run faster, use less power, cost less to build, and use less area,” says Goldie.

Source: https://spectrum.ieee.org/tech-talk/semiconductors/design/google-invents-ai-that-learns-a-key-part-of-chip-design

https://arxiv.org/abs/2003.08445

Thoughts: I saw this coming long ago, it might be useful in discovering new routing for data and I could see it saving a lot of time in some scenarios. I don't have a use for NPUs myself so I do wish this could be applied to more consumer processors.


Specs: Motherboard: Asus X470-PLUS TUF gaming (Yes I know it's poor but I wasn't informed) RAM: Corsair VENGEANCE® LPX DDR4 3200Mhz CL16-18-18-36 2x8GB

            CPU: Ryzen 7 2700X @ 4.2Ghz          Case: Antec P8     PSU: G.Storm GS850                        Cooler: Antec K240 with two Noctura Industrial PPC 3000 PWM

            Drives: Samsung 970 EVO plus 250GB, Micron 1100 2TB, Seagate ST4000DM000/1F2168 GPU: EVGA RTX 2080 ti Black edition @ 2Ghz

                                                                                                                             

Link to post
Share on other sites
37 minutes ago, williamcll said:

tpu-19bkw9.max-500x500.PNG

Google's own Statistical analysis system has managed to create it's own Tensor processor after less than a day of maths.

Source: https://spectrum.ieee.org/tech-talk/semiconductors/design/google-invents-ai-that-learns-a-key-part-of-chip-design

https://arxiv.org/abs/2003.08445

Thoughts:

Fixed it. ;)

Link to post
Share on other sites

So Google is doing the 'chicken or egg first' to themselves


Awareness is key. Never enough, even in the face of futility. Speak the truth as if you may never get to say it again. This world is full of ugly. Change it they say. The only way is to reveal the ugly. To change the truth you must first acknowledge it. Never pretend it isn't there. Never bend the knee.

 

Please quote my post in your reply, so that I will be notified and can respond to it. Thanks.

Link to post
Share on other sites
8 minutes ago, huilun02 said:

So Google is doing the 'chicken or egg first' to themselves

 

In this case, Chicken.

 

That makes me hungry, are abstract code concepts edible i wonder?

Link to post
Share on other sites

Oooooo


~New~  BoomBerryPi project !  ~New~


new build log : http://linustechtips.com/main/topic/533392-build-log-the-scrap-simulator-x/?p=7078757 (5 screen flight sim for 620$ CAD)LTT Web Challenge is back ! go here  :  http://linustechtips.com/main/topic/448184-ltt-web-challenge-3-v21/#entry601004

Link to post
Share on other sites
7 hours ago, Den-Fi said:

WarmheartedBlaringBrocketdeer-size_restr

7UMf3h.gif

Up next, it designs its own batteries.

Humans in Matrix ARE the processors, the final script says they are an energy source because the studio feared the audience wouldn't understand it.


Spoiler

Quiet Whirl | CPU: AMD Ryzen 7 3700X Cooler: Noctua NH-D15 Mobo: MSI B450 TOMAHAWK MAX RAM: HyperX Fury RGB 32GB (2x16GB) DDR4 3200 Mhz Graphics card: MSI GeForce RTX 2070 SUPER GAMING X TRIO PSU: Corsair RMx Series RM550x Case: Be quiet! Pure Base 600

 

Buffed HPHP ProBook 430 G4 | CPU: Intel Core i3-7100U RAM: 4GB DDR4 2133Mhz GPU: Intel HD 620 SSD: Some 128GB M.2 SATA

 

Retired:

Melting plastic | Lenovo IdeaPad Z580 | CPU: Intel Core i7-3630QM RAM: 8GB DDR3 GPU: nVidia GeForce GTX 640M HDD: Western Digital 1TB

The Roaring Beast | CPU: Intel Core i5 4690 (BCLK @ 104MHz = 4,05GHz) Cooler: Akasa X3 Motherboard: Gigabyte GA-Z97-D3H RAM: Kingston 16GB DDR3 (2x8GB) Graphics card: Gigabyte GTX 970 4GB (Core: +130MHz, Mem: +230MHz) SSHD: Seagate 1TB SSD: Samsung 850 Evo 500GB HHD: WD Red 4TB PSU: Fractal Design Essence 500W Case: Zalman Z11 Plus

 

Link to post
Share on other sites
46 minutes ago, matrix07012 said:

Humans in Matrix ARE the processors, the final script says they are an energy source because the studio feared the audience wouldn't understand it.

I'm just gonna go with the version of the script that works with the semi-joke I made, lol.

But I do appreciate the insight.

Link to post
Share on other sites

Considering most SoCs are into the billions of transistor range, almost all of it is done by algos designed by the companies already. Google basically is claiming they've managed to make a new algo to improve workflow. 

 

Somehow, we're in 2020 and it feels like 1995 with "and this mundane thing happened on THE INTERNET!" when it comes to "AI". All it is using a neural network to optimize already existing algorithms. Process designed to optimized pre-existing processes optimized All the Things.

Link to post
Share on other sites
2 hours ago, Taf the Ghost said:

Considering most SoCs are into the billions of transistor range, almost all of it is done by algos designed by the companies already. Google basically is claiming they've managed to make a new algo to improve workflow. 

 

Somehow, we're in 2020 and it feels like 1995 with "and this mundane thing happened on THE INTERNET!" when it comes to "AI". All it is using a neural network to optimize already existing algorithms. Process designed to optimized pre-existing processes optimized All the Things.

Sounds like the shit my current employer tries to say when they claim AI is making all these optimizations when it's nothing more than a simple neural network being constrained by human contextual boundaries (don't get me wrong, I expect Google's effort is way better than ours, but still ugh)


LINK-> Kurald Galain:  The Night Eternal 

Top 5820k, 980ti SLI Build in the World*

CPU: i7-5820k // GPU: SLI MSI 980ti Gaming 6G // Cooling: Full Custom WC //  Mobo: ASUS X99 Sabertooth // Ram: 32GB Crucial Ballistic Sport // Boot SSD: Samsung 850 EVO 500GB

Mass SSD: Crucial M500 960GB  // PSU: EVGA Supernova 850G2 // Case: Fractal Design Define S Windowed // OS: Windows 10 // Mouse: Razer Naga Chroma // Keyboard: Corsair k70 Cherry MX Reds

Headset: Senn RS185 // Monitor: ASUS PG348Q // Devices: Note 10+ - Surface Book 2 15"

LINK-> Ainulindale: Music of the Ainur 

Prosumer DYI FreeNAS

CPU: Xeon E3-1231v3  // Cooling: Noctua L9x65 //  Mobo: AsRock E3C224D2I // Ram: 16GB Kingston ECC DDR3-1333

HDDs: 4x HGST Deskstar NAS 3TB  // PSU: EVGA 650GQ // Case: Fractal Design Node 304 // OS: FreeNAS

 

 

 

Link to post
Share on other sites
15 hours ago, RejZoR said:

AMD's Bulldozer chips were also computer generated weren't they? It was faster and cheaper, but we all know how that ended...

Intel's Nehalem and Sandy Bridge parts were as well... Same with Zen and Zen2

Basically everyones' chips use algorithms to lay out critical parts. 

https://en.wikichip.org/wiki/File:amd_zen_core.png <- look at this core. The "mushroomy" stuff was designed mostly by an AI. 

What the humans are doing is providing high level guidance, taking the AI's design, tuning it and then getting it to work in the real world.

As someone who writes "AI" for a living, there's A LOT of tuning to do. 


R9 3900x; 64GB RAM | RTX 2080 | 1.5TB Optane P4800x

1TB ADATA XPG Pro 8200 SSD | 2TB Micron 1100 SSD
HD800 + SCHIIT VALI | Topre Realforce Keyboard

Link to post
Share on other sites

Comments Feels GIF - Find & Share on GIPHY

 

That is all!


My Folding Stats Join the fight against COVID-19 with FOLDING! 

 

The only true wisdom is in knowing you know nothing. - Socrates
 

Please put as much effort into your question as you expect me to put into answering it. 

 

  • CPU
    Ryzen 7 1700 3GHz 8-Core Processor @ 3.7Ghz
  • Motherboard
    GA-AX370-GAMING 5
  • RAM
    DOMINATOR Platinum 16GB (2 x 8GB) @ 3400mhz
  • GPU
    Aorus GTX 1080 Waterforce
  • Case
    Cooler Master - MasterCase H500P
  • Storage
    Western Digital Black 250GB, Seagate BarraCuda 1TB x2
  • PSU
    EVGA Supernova 1000w 
  • Display(s)
    BenQ - XL2430(144hz), Dell 24" portrait
  • Cooling
    MasterLiquid Lite 240
Link to post
Share on other sites
3 hours ago, Curufinwe_wins said:

Sounds like the shit my current employer tries to say when they claim AI is making all these optimizations when it's nothing more than a simple neural network being constrained by human contextual boundaries (don't get me wrong, I expect Google's effort is way better than ours, but still ugh)

For as hard as actual underlying technology is to comprehend to most, the reality is that all a neural network system can never do is optimize to the criteria inputted and the assumptions made when getting there. The results are always "surface-level" and very "numb", but they are optimized. In the case of placing transistors (which is one of the few places in the world of Human Experience where billions of iterations are actually needed), it's actually quite useful to optimize well over that many iterations. The real power of automation has always been to iterate at a rate beyond what a human can do.

 

That being said, AI is a massive, overblown bubble. Spreadsheets & Databases have been extremely powerful since being introduced, but it still takes a lot of skill to get anything useful out of them. AI will be even worse. Unless you have a task that can be iterated on millions of times like SoC development, the actual utility will always be very limited.

Link to post
Share on other sites
4 hours ago, comander said:

Intel's Nehalem and Sandy Bridge parts were as well... Same with Zen and Zen2

Basically everyones' chips use algorithms to lay out critical parts. 

https://en.wikichip.org/wiki/File:amd_zen_core.png <- look at this core. The "mushroomy" stuff was designed mostly by an AI. 

What the humans are doing is providing high level guidance, taking the AI's design, tuning it and then getting it to work in the real world.

As someone who writes "AI" for a living, there's A LOT of tuning to do. 

So, it's not really "Ai", it's just layout coordinator that stuffs required stuff into as small space possible. "Ai". Just throw it at anything. I guess I'm eating an Ai sardines right now...

Link to post
Share on other sites
1 hour ago, RejZoR said:

So, it's not really "Ai", it's just layout coordinator that stuffs required stuff into as small space possible. "Ai". Just throw it at anything. I guess I'm eating an Ai sardines right now...

They seem to have put in the work to optimize another step that used to require a lot more of "hands-on" work. If their criteria was set properly. It'll still need to be reviewed by the engineers that would have needed to do weeks of silicon implementation work. So, it might, eventually, be a solid time saver. (Still took nearly a day on what is likely a near super computer.)

 

Humorously, to me, it looks like they used the data they got from OpenAI playing Dota 2 for how to attack the problem. Also quite funny is I think OpenAI took ages longer to get good, lol.

Link to post
Share on other sites
9 hours ago, RejZoR said:

So, it's not really "Ai", it's just layout coordinator that stuffs required stuff into as small space possible. "Ai". Just throw it at anything. I guess I'm eating an Ai sardines right now...

At some level AI is just more automated algorthmic logic. 

And yeah, algorithmic optimization has been a HUGE part of CPU design for a while. 

The stuff going into Bulldozer probably wouldn't qualify as AI to most people FWIW. The stuff going into Google's chips does - they're using reinforcement learning which is basically the oldschool (2008ish) stuff on steroids. This is the same class of techniques used for AlphaStar and AlphaGo (though implementation is obviously different). 


R9 3900x; 64GB RAM | RTX 2080 | 1.5TB Optane P4800x

1TB ADATA XPG Pro 8200 SSD | 2TB Micron 1100 SSD
HD800 + SCHIIT VALI | Topre Realforce Keyboard

Link to post
Share on other sites
7 hours ago, VegetableStu said:

"strange, the traces look like a game's battle map..."

I wouldn't doubt they pick all of the Cheesy Meta heroes as well.

 

The actual OpenAI Dota 2 project was interesting, mostly as because after millions of hours of playing Dota 2, the AI basically just ran at you and all it did was out-execute a human team. Its Borg-like unrelentingness is what allowed it to do so well.

Link to post
Share on other sites
32 minutes ago, Taf the Ghost said:

I wouldn't doubt they pick all of the Cheesy Meta heroes as well.

 

The actual OpenAI Dota 2 project was interesting, mostly as because after millions of hours of playing Dota 2, the AI basically just ran at you and all it did was out-execute a human team. Its Borg-like unrelentingness is what allowed it to do so well.

Which is why the SC2 results were WAY more impressive. Even after the additional limitations placed on Alphastar. It plays pretty darn cool legit games. It can be tricked, eventually, sometimes, but there are actual lessons being learned that have already been incorporated into proplay.


LINK-> Kurald Galain:  The Night Eternal 

Top 5820k, 980ti SLI Build in the World*

CPU: i7-5820k // GPU: SLI MSI 980ti Gaming 6G // Cooling: Full Custom WC //  Mobo: ASUS X99 Sabertooth // Ram: 32GB Crucial Ballistic Sport // Boot SSD: Samsung 850 EVO 500GB

Mass SSD: Crucial M500 960GB  // PSU: EVGA Supernova 850G2 // Case: Fractal Design Define S Windowed // OS: Windows 10 // Mouse: Razer Naga Chroma // Keyboard: Corsair k70 Cherry MX Reds

Headset: Senn RS185 // Monitor: ASUS PG348Q // Devices: Note 10+ - Surface Book 2 15"

LINK-> Ainulindale: Music of the Ainur 

Prosumer DYI FreeNAS

CPU: Xeon E3-1231v3  // Cooling: Noctua L9x65 //  Mobo: AsRock E3C224D2I // Ram: 16GB Kingston ECC DDR3-1333

HDDs: 4x HGST Deskstar NAS 3TB  // PSU: EVGA 650GQ // Case: Fractal Design Node 304 // OS: FreeNAS

 

 

 

Link to post
Share on other sites
6 minutes ago, Curufinwe_wins said:

Which is why the SC2 results were WAY more impressive. Even after the additional limitations placed on Alphastar. It plays pretty darn cool legit games. It can be tricked, eventually, sometimes, but there are actual lessons being learned that have already been incorporated into proplay.

The AlphaStar stuff was definitely interesting, though the fact that the AI could keep a recallable database of builds that the opponent could do, it made scouting and the AI's ability to execute always have a pretty sizable advantage.  Though, SC2 is a little weird in that there were already unbeatable AIs by I think 2011, since you can run at over 30k APM within the game engine. 

Link to post
Share on other sites
10 minutes ago, Taf the Ghost said:

The AlphaStar stuff was definitely interesting, though the fact that the AI could keep a recallable database of builds that the opponent could do, it made scouting and the AI's ability to execute always have a pretty sizable advantage.  Though, SC2 is a little weird in that there were already unbeatable AIs by I think 2011, since you can run at over 30k APM within the game engine. 

I don't think I'd agree with those assessments on the whole, in fact one of Alphastars apparent weaknesses (while still quite good) was to just really bizarre play. (The one map TLO won was by basically forcing Alphastar into a cyclical paralysis position until he was able to bring overwhelming force to bear)

 

Plus the later variants capped APM both spike and average to levels well below the best players and even incorporated limits to the speed and frequency by which screen could jump around (after all otherwise the fact that a computer could theoretically take in a screens worth of information almost instantly would be a signficant advantage).

 

Though yes, between games of perfect information, and unlimited control, there were hard to beat AIs in the past (they tended to be fairly trivial to exploit, but if you just let them get to where their strength was... good luck)

 

 

Anyways, not really relevant this discussion. If you are interested more, BeastyQt has some great commentary/analysis of later alphastar games on his youtube channel. Not just for experts of the game or such.


LINK-> Kurald Galain:  The Night Eternal 

Top 5820k, 980ti SLI Build in the World*

CPU: i7-5820k // GPU: SLI MSI 980ti Gaming 6G // Cooling: Full Custom WC //  Mobo: ASUS X99 Sabertooth // Ram: 32GB Crucial Ballistic Sport // Boot SSD: Samsung 850 EVO 500GB

Mass SSD: Crucial M500 960GB  // PSU: EVGA Supernova 850G2 // Case: Fractal Design Define S Windowed // OS: Windows 10 // Mouse: Razer Naga Chroma // Keyboard: Corsair k70 Cherry MX Reds

Headset: Senn RS185 // Monitor: ASUS PG348Q // Devices: Note 10+ - Surface Book 2 15"

LINK-> Ainulindale: Music of the Ainur 

Prosumer DYI FreeNAS

CPU: Xeon E3-1231v3  // Cooling: Noctua L9x65 //  Mobo: AsRock E3C224D2I // Ram: 16GB Kingston ECC DDR3-1333

HDDs: 4x HGST Deskstar NAS 3TB  // PSU: EVGA 650GQ // Case: Fractal Design Node 304 // OS: FreeNAS

 

 

 

Link to post
Share on other sites
38 minutes ago, Curufinwe_wins said:

I don't think I'd agree with those assessments on the whole, in fact one of Alphastars apparent weaknesses (while still quite good) was to just really bizarre play. (The one map TLO won was by basically forcing Alphastar into a cyclical paralysis position until he was able to bring overwhelming force to bear)

 

Plus the later variants capped APM both spike and average to levels well below the best players and even incorporated limits to the speed and frequency by which screen could jump around (after all otherwise the fact that a computer could theoretically take in a screens worth of information almost instantly would be a signficant advantage).

 

Though yes, between games of perfect information, and unlimited control, there were hard to beat AIs in the past (they tended to be fairly trivial to exploit, but if you just let them get to where their strength was... good luck)

 

 

Anyways, not really relevant this discussion. If you are interested more, BeastyQt has some great commentary/analysis of later alphastar games on his youtube channel. Not just for experts of the game or such.

Yeah. You can always make a bigger hammer, and a big hammer always hits hard. Difference being, a person can often work around and find new exploits to said hammer (in this example, the hammer is AI ;) ).

Some AI try, through trial and error, to overcome such obstacles. But often perform worse than a human in intelligence, but better in number of iterations (a server farm of compute will give millions of iterations in the time a person can think of a few responses). It's again, not *really* "better intelligence" as the real life counter would be a city block of people working on a problem (example would be NASA getting the astrunoughts back in Apollo 13).

 

So really, when someone says "this AI is better" what they mean is "we spent more money/we used more resources/we have more people/we used more math/we have more compute/we outsourced". :P

Link to post
Share on other sites

I didn't know Terminator was based on a true story that takes place into our future.


Mobo: Z97 MSI Gaming 7 / CPU: i5-4690k@4.5GHz 1.23v / GPU: EVGA GTX 1070 / RAM: 8GB DDR3 1600MHz@CL9 1.5v / PSU: Corsair CX500M / Case: NZXT 410 / Monitor: 1080p IPS Acer R240HY bidx

Link to post
Share on other sites
On 3/30/2020 at 12:31 PM, TechyBen said:

So really, when someone says "this AI is better" what they mean is "we spent more money/we used more resources/we have more people/we used more math/we have more compute/we outsourced". :P

While throwing compute at problems definitely matters, it's not all that. 

HOW something is computed often matters more than the quantity of compute tossed at a problem (Quick-sort on a 1000 number list is MUCH faster than BOGO sort on the same list with 10,000 the IOPs)

In a lot of cases the exact methods used matter a TON. Here and today, if pressed I could make an image classifier in a month for some sort of problem using my desktop (let's assume all of my relevant data is already gathered). A decade ago, it would've taken a team of PhDs with super computers a lot longer to come up with worse results. In many instances, people end up standing on the shoulder's of giants and reusing techniques (or pretrained neural networks). The people at DeepMind are in some sense those giants. They're REALLY on the cutting edge.


R9 3900x; 64GB RAM | RTX 2080 | 1.5TB Optane P4800x

1TB ADATA XPG Pro 8200 SSD | 2TB Micron 1100 SSD
HD800 + SCHIIT VALI | Topre Realforce Keyboard

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


×