Google's AlphaZero Chess AI takes only 4 hours of learning to beat world's best computer chess engine

[Phil9943]

Google's Alphazero AI program took only 4 hours to teach itself chess well enough to beat the current world champion of computer chess, a chess engine named Stockfish. In a landmark event for computer chess, Alphazero tied 72 and won 28 of the 100 games it played against Stockfish 8. Alphazero was developed by Deepmind, a company acquired by Google in 2014, after defeating the world champion in Go with its own AlphaGo program back in October.

 

Deepmind released a paper detailing the results of their program, in which 10 of the 100 games were completely published and many statistics were shared. One image I found particularly interesting showed the frequency of openings played as the program became smarter over the 4 hours. (See Attached image) 

 

The image was taken from a much more deep analysis on chess.com. I will link their article but will credit Ars Technica as an original source.

 

Most chess engines that are popular today use a brute-force method of move calculation, a method that was almost always hindered by hardware, not necessarily by how they were programmed. However, Alphazero uses a different method of finding the best move. It uses a common AI technique known as enforcement learning. Many Grandmasters, who usually use engines such as Stockfish or Komodo as preparation tools, are eager for the program to become publicly available. GM Larry Kaufman, head of the Komodo engine, said he hopes that the program will effective on more than just Google's own servers.

 

The original paper can be read here: https://arxiv.org/pdf/1712.01815.pdf

 

Original Article (from Ars Technica): https://arstechnica.com/gaming/2017/12/deepmind-ai-needs-mere-4-hours-of-self-training-to-become-a-chess-overlord/

10 published games of the 100 (for all the chess lovers): http://www.chessgames.com/perl/chess.pl?tid=91944&pid=125070

Chess.com analysis: https://www.chess.com/news/view/google-s-alphazero-destroys-stockfish-in-100-game-match

[2FA]

Something Deepmind also recently published was that providing the AI with zero initial training data resulted in exceptionally better play in Go. The AI that was trained this way beat AlphaGo which conversely had training data from Go masters.

[Phil9943]

Any hardcore chess fans on this forum? I think it's kind of interesting how little king's pawn gets played. They didn't even include it in the paper, and it's a move a lot of beginners tend to find, and you still see a lot of it when GM's play each other. It seems to have decided that Queen's gambit and English are good openings, but the English doesn't make much sense to me.

 

I'm not claiming I'm good actually good at chess, but just from a basic principle perspective, queens pawn and ruy lopez make more sense to me.

[Phil9943]

4 minutes ago, DeadEyePsycho said:

Something Deepmind also recently published was that providing the AI with zero initial training data resulted in exceptionally better play in Go. The AI that was trained this way beat AlphaGo which conversely had training data from Go masters.

Yeah, I think it's pretty cool how they didn't give Alphazero ANY kind of an opening table. GM Kaufmann pointed out how it kind of had a "superpowered" opening table of sorts after letting AI do it's thing. I'm not sure how Komodo and Stockfish create their opening table, but from the way Kaufmann was talking, it didn't seem like they do it like this. Maybe they just do a lot of chessbase research, but idk.

[Phil9943]

Anyone notice the spike in the French? Also, there seemed to be a lot of Caro-kann, but then it figured something out after about 7 hours. Maybe it noticed some pretty big flaws that us humans are just overlooking. Sort of similar thing looks like it happened with the Ruy Lopez. Maybe the AI can teach us a few things about opening theory we haven't figured out yet

[spartaman64]

3 hours ago, DeadEyePsycho said:

Something Deepmind also recently published was that providing the AI with zero initial training data resulted in exceptionally better play in Go. The AI that was trained this way beat AlphaGo which conversely had training data from Go masters.

as i thought training AI based off of human games is only holding them back

[spartaman64]

3 hours ago, Phil9943 said:

Any hardcore chess fans on this forum? I think it's kind of interesting how little king's pawn gets played. They didn't even include it in the paper, and it's a move a lot of beginners tend to find, and you still see a lot of it when GM's play each other. It seems to have decided that Queen's gambit and English are good openings, but the English doesn't make much sense to me.

 

I'm not claiming I'm good actually good at chess, but just from a basic principle perspective, queens pawn and ruy lopez make more sense to me.

to me the english allows for a more active queenside pawn structure instead of having the pawn stuck behind the knight. and later when you play pawn b3 you have a nice pawn chain and you can plan to put a knight on d5 which is a nice square

 

and chess engines are not severely affected by the lack of open or end tables for a while now

[Taf the Ghost]

7 hours ago, DeadEyePsycho said:

Something Deepmind also recently published was that providing the AI with zero initial training data resulted in exceptionally better play in Go. The AI that was trained this way beat AlphaGo which conversely had training data from Go masters.

It's the same reason Machine AIs will develop unique languages when "talking" to each other. Humans have a generalized interaction with the world. A chessboard is no different than a checkers board, but for a machine it is utterly different. 

 

In the case of AlphaGo, the computer could absorb the 10,000s of failed attempts within a potential branch upon which it was able to beat a Go master. You see a version of this in the business world, actually. A new company can take deep risks that an established firm can't. If the new firm can make the technology or design work, they suddenly can compete with the established firm(s). In the case of a Machine Learning approach to rules-based games, it's just a hyper-fast version of this. A human can't absorb losing that much, as they're bound to their time investment, so they'll optimize to their local win conditions. (Or quit because it's not worth the time.)

7 hours ago, Phil9943 said:

Yeah, I think it's pretty cool how they didn't give Alphazero ANY kind of an opening table. GM Kaufmann pointed out how it kind of had a "superpowered" opening table of sorts after letting AI do it's thing. I'm not sure how Komodo and Stockfish create their opening table, but from the way Kaufmann was talking, it didn't seem like they do it like this. Maybe they just do a lot of chessbase research, but idk.

 

7 hours ago, Phil9943 said:

Anyone notice the spike in the French? Also, there seemed to be a lot of Caro-kann, but then it figured something out after about 7 hours. Maybe it noticed some pretty big flaws that us humans are just overlooking. Sort of similar thing looks like it happened with the Ruy Lopez. Maybe the AI can teach us a few things about opening theory we haven't figured out yet

Strategic value of positions will change as the AI learns how far those positions allow it to go further within the game. For Humans, it's very possible that the types of moves that would be needed would be such a tight range it wouldn't work well. There's also the issue that the Humans are less predictable in action, so in a human vs human scenario, you can have to re-evaluate your end-game scenarios when the other person messes up. Computer vs Computer doesn't make mistakes in the same way.

3 hours ago, spartaman64 said:

as i thought training AI based off of human games is only holding them back

High-level Human games aren't isolated. They're two people with tendencies that have studied each other, thus they'll skew their own approaches in a way as to beat their opponent.  A Human is trying to beat another Human; an AI is trying to optimize for a victory state. It's actually quite a different approach. 

[sof006]

Thats pretty damn amazing.. and scary. I wonder how far away we are from more advanced AI.

[Matu20]

I really love reading headlines about this, but the math itself is too much for me to get hooked.