January 13, 2015

Stockfish headbangers: Twin duel SF5 vs SF121014

Just getting back from my job late in the evening, i went directly to the refrigerator to check whether the twin duel has ended well. Yes, it's done. At least a first stage of the experiment delivers logical results, opposing my first attempt to find out which Stockfish version plays better chess.

Komodo 8 won last TCEC on a super PC with 2 Xeon cpu's. However, the Android port is somehow weaker than Stockfish, as suggested not only by my own experiments but also some others. Lower nodes/second, faster time controls and 32-bit Android environment work all against the reptiloid and we have nothing but Stockfish 5 and Stockfish 121014 to battle for an humble mobie crown.

I've got the two versions extracted from Droidfish 1.54 and 1.56, both Arm7 and i686 versions to play on Galaxy Note II and simultaneously on Asus ME173X.

Asus' Intel 3745D cpu is in general slightly faster than Exynos 4412 but when it comes to chess, it seems it depends. According to 3 minutes on inital position tests, 121014 hits 800 Knps vs SF5 reaching 718 Knps on Exynos, while on Intel they all come get stuck at 770 Knps. This means Intel is supposed to rip out about 10 ELO of difference from Arm7 to Intel and should favour SF5 more than on Exynos.

After 64 games played both sides with 32 TCEC-6 superfinal openings, i've obtained two similar results. They are close enough and both matches favour 121014 as the strongest Android engine to date.

The result on Galaxy Note II is:
Stockfish 121014: 35½ (+16 =39  -9) 55.5% +38 ELO
Stockfish 5     : 28½ ( +9 =39 -16) 44.5% -38 ELO

The summary on Intel 3745D is:
Stockfish 121014: 36½ (+17 =39  -8) 55.5% +49 ELO
Stockfish 5     : 27½ ( +8 =39 -17) 44.5% -49 ELO

Given that the error margin after 64 games is still 53 ELO, i must decide to continue with TCEC-6 Stage-4 openings to be sure. Then 96 more games will extend the duel.

Regarding my December-2014 experiment where SF5 won against 121014, even after comparing the log records, i see no traces of error on my side and both engines seem to have gone as deep as possible. Therefore, the only different parameter which remained is the different binary of 121014. Unfortunately i'm not able to comment anything about such reversed result.

Hopefully, the current duel is based on the latest compiles. So, i rather focus more on the latest and optimize it with more samples.

Gone fishing. Bye.

No comments:

Post a Comment