David Silver hinted that DeepMind is done with Starcraft in a BBC news article saying “the lab may rest now” and that they have “completed the Starcraft challenge”.

I thought this was a little disappointing since the skill level Alphastar reached on ladder was not enough to beat professional players. I think we all wanted a real nice showdown between the human champion and the robot, right? That’d been pretty cool.

The Nature paper had a nice graph depicting Alphastar’s MMR which is basically Blizzard’s version of elo rating. The Protoss agent had reached an MMR of ~6200 and the aggregate of all three races was 6030 iirc. The graph also had MMR’s of Alphastar’s opponents and information on whether the agent won or lost.

Basically Alphastar had lost all but 2 games against players who had higher than 6200 MMR. On ladder, it could not beat the professionals.

The agent from January was estimated to have been over 7000 MMR. I figured it’d be nice to estimate how well this newest agent would have fared against Mana. Right now, MaNa’s MMR is ~6700.

So I looked at the EU ladder, found someone with an MMR of ~6200, popped him and MaNa into Aligulac (sc2 database) and let it estimate some odds. MaNa had ~75% chance of winning a Best of 5, and his 6200 MMR opponent had less than 1% chance of beating MaNa 5-0.

At this point I became convinced that DeepMind was throwing in the towel on sc2 because the cost of further improving Alphastar was too high to justify the publicity they were getting from the project. The team looked to be moving on to different things and the showmatch vs the world champion had been cancelled.

But then something absolutely baffling happened which I don’t think anyone saw coming.

Blizzcon was this weekend. With little to no fanfare DeepMind had brought Alphastar with them and let Blizzcon visitors play against it. Serral, one of the best players in the world, had just finished top 4 in the biggest tournament of the year wandered to the arcade and played a few games against the bot. Serral’s MMR is over 7000.

He lost 0-3 to the Protoss agent. These games were not televised. All we have is some blurry smartphone footage. https://mobile.twitter.com/LiquidTLO/status/1190779241564000256

I don’t get it. If Alphastar was this strong why didn’t DeepMind let it play more on ladder and get a higher ranking? Why didn’t they organize a showmatch or something? They dropped the ball pretty hard on this one. This is so confusing to me.

First they beat two professional players but were hit with a huge, imo warranted backlash due to the APM controversy.

Then they produced agents under more proper mechanical limitations and the agents turned out to be much weaker than the previous version.

Finally, they beat the best player In the world, seemingly accidentally while no one was looking.

From PR standpoint, could this have gone any worse for Deepmind?