HAL9000

HAL9000
"It just isn't conceivable that you can design a program strong enough to beat players like me."

January 31, 2016

RAPIDROID RELOADED: February-2016 with Stockfish 7, finally

Stockfish 7 finally comes in and replaces its predecessor 121215 on top of Rapidroid. Nothing unexpected here since Komodo 9.3 had lost the leadership in January list, after a shocking regression of some forty ELOs which i assumed to be specific to Android, in contrast to Windows, where 9.3 and 9.2 perform almost the same. 
This build of Stockfish just added some more gap over Komodo. I wish i could revert back to Komodo 9.2 but it's technically impossible for any device on which 9.3 is installed once.

The unnecessary complementary info of this release is that i've dropped Robocide and Virutor from the list of CCRL anchors as their Android builds play better than Windows counterparts. I won't be looking for an explanation of this. The other 40 engines remaining are still enough to keep a good balance between two operating systems. Cutting off the extreme deviations should be healty.

The impact of dropping two engines is about +12 ELO for the whole list. The average gap of Android engines vs their targets calculated from CCRL is now around 35 ELO which is acceptable for a list with 300 games per engine. 

I hope that March-2016 release will be fun with Fritz, Gull and Ivanhoe. They may disturb the peace in the top segment as they are expected to reach 3060, 3050 and 2950 ELO respectively.

The highlights of this release:
* UPDATED Crafty 24.1 to 25.0 arm7
* UPDATED ExChess 7.71 to 7.88 arm7
* UPDATED Phalanx XXIV arm5 to arm7
* UPDATED RedQueen 1.1.97 to 1.1.98 arm7
* UPDATED Stockfish 121215 to 7

The list is calibrated using CCRL 40/4 ratings of 40 engines, converted to Exynos 4412, assuming that the latter performs 65 and 110 ELO lower than 32 and 64 bit engines of CCRL respectively.

Targets for next release:
* Introduce Fritz 14, Gull 3 and Ivanhoe 9.47c Beta

BAYES RATINGS AFTER 16561 GAMES PLAYED BY 125 PROGRAMS
Rnk Name                     O/S T  Elo   +   - gam sco oppo dra
001 Stockfish 7              A32 4 3349  41  39 300 85% 3068 29%
002 Komodo 9.3               A32 4 3294  38  36 300 78% 3072 30%
003 Critter 1.6a             A32 4 3150  33  33 308 59% 3086 41%
004 Sting SF 4.8.4 JA        A32 4 3119  34  34 304 58% 3054 36%
005 Firenzina 2.4.1 xTreme   A32 4 3116  33  32 304 54% 3085 46%
006 BlackMamba 2.0           A32 4 3083  32  33 306 51% 3082 44%
007 Rybka 2.3.2a mp          W64 4 3076 109 108  26 54% 3049 38%
008 Texel 1.05               A32 4 3036  34  34 306 50% 3044 32%
009 Senpai 1.0               A32 4 3024  34  34 316 56% 2981 31%
010 DeepSaros ver.2.3f       A32 4 3018  35  35 302 53% 3000 29%
011 Hiarcs 13.71             IOS 2 2983 121 114  26 65% 2874 23%
012 RobboLito 0.085e4l       A32 1 2971  34  34 306 51% 2970 34%
013 Cheng 4.39               A32 4 2951  33  33 312 47% 2983 36%
014 Shredder 1.7.0           IOS 2 2919 114 113  24 54% 2902 42%
015 Hakkapeliitta 3.0        A32 1 2900  34  34 312 50% 2911 29%
016 Gaviota v1.0-d           A32 4 2899  34  34 312 48% 2920 29%
017 Scorpio_2.7.7.JA_xb.arm7 A32 4 2898  37  37 278 58% 2837 28%
018 Arasan 18.2              A32 4 2892  34  34 306 46% 2925 30%
019 ExChess_7.88b.JA_xb.arm7 A32 4 2883  34  34 300 46% 2924 37%
020 Grapefruit 1.0           A32 4 2853  32  32 318 48% 2867 40%
021 Cyclone 3.4              A32 4 2840  32  32 316 50% 2840 37%
022 DiscoCheck 5.2.1         A32 1 2835  33  33 324 47% 2858 33%
023 Deep Saros 0.9           A32 4 2812  32  32 318 47% 2830 37%
024 Toga II 3.0              A32 1 2808  32  32 314 47% 2831 37%
025 Deuterium v14.3.34.130   A32 1 2776  32  32 322 50% 2774 38%
026 Bobcat 6.4b              A32 1 2765  33  33 320 49% 2771 28%
027 Doch32 1.3.4 JA          A32 1 2760  32  32 318 49% 2769 35%
028 Chess Pro 2016.02        IOS 2 2757 111 113  22 45% 2786 55%
029 Fruit Reloaded 2.1       A32 1 2756  32  32 298 52% 2747 40%
030 GNU Chess 5.60           A32 1 2752  33  33 312 50% 2750 29%
031 Murka 3                  A32 1 2749  31  31 330 51% 2742 36%
032 Crafty_25.0.JA_xb.arm7   A32 1 2741  33  33 318 51% 2734 31%
033 IvanHoe 9.46b            A32 4 2725  33  33 312 46% 2748 31%
034 The King 3.50 x64        W64 1 2718  49  50 132 42% 2768 35%
035 Strelka 5                A32 1 2713  33  33 318 53% 2689 31%
036 TheMadPrune 1.7.04       A32 1 2700  34  34 288 47% 2720 32%
037 RedQueen 1.1.98          A32 4 2666  33  33 320 51% 2661 30%
038 CNVCS 1.2.0              IOS 2 2660 102 102  30 50% 2654 40%
039 Tucano_5.00.JA_xb        A32 1 2656  36  36 252 47% 2679 34%
040 Rodent 1.7 build 1       A32 1 2653  33  33 320 49% 2661 30%
041 Mini Rodent 1.0          A32 1 2630  33  33 294 46% 2654 35%
042 Rhetoric 1.4.1           A32 1 2615  33  33 320 50% 2613 28%
043 Bison 15.1               A32 1 2608  33  33 320 50% 2613 29%
044 Chess Genius 4.0.00      IOS 2 2578 195 255   8 13% 2830 25%
045 Alfil 12.10              A32 1 2576  33  33 314 48% 2586 29%
046 Gull 1.2 JA              A32 1 2566  33  33 316 51% 2559 28%
047 Daydreamer 1.75 JA       A32 1 2554  32  32 314 51% 2550 36%
048 Cheese 1.7               A32 1 2544  33  34 308 44% 2588 30%
049 Rotor 0.8                A32 1 2541  33  33 316 45% 2585 29%
050 Fridolin 2.00            A32 4 2521  33  32 306 53% 2501 34%
051 Chess Genius 2.6.4       A32 1 2517 224 239   4 38% 2565 75%
052 Glaurung Mainz           A32 1 2504  41  41 208 44% 2543 27%
053 GarboChess 3             A32 1 2504  34  34 314 46% 2532 25%
054 Sloppy_0.23.JA_xb        A32 1 2483  34  34 292 52% 2466 33%
055 Danasah_5.07.JA_xb       A32 1 2483  35  35 290 53% 2456 28%
056 BBChess 1.3b JA          A32 4 2476  33  33 314 51% 2470 29%
057 Maverick 1.5 arm         A32 1 2473  33  33 318 57% 2420 29%
058 Dirty_030411.JA_xb       A32 1 2470  36  36 280 52% 2454 26%
059 Phalanx_XXIV.JA_xb.arm7  A32 1 2466  36  36 286 49% 2474 16%
060 Pawny_1.0.JA_uci2xb      A32 1 2432  34  34 300 53% 2407 30%
061 GreKo_12.5.JA_xb         A32 1 2429  35  35 296 54% 2398 23%
062 Pepito v1.59             A32 1 2413  33  33 312 48% 2427 33%
063 BetsabeII_1.47.JA_xb     A32 1 2402  36  36 294 50% 2403 20%
064 Ifrit_m1.8.JA_uci2xb     A32 1 2376  34  34 296 55% 2338 28%
065 Diablo 0.5.1b JA         A32 1 2347  34  34 314 52% 2331 24%
066 zurichess geneva         A32 1 2347  52  52 138 49% 2354 25%
067 Typhoon_1.0.r358.JA_xb   A32 1 2345  34  34 306 54% 2314 25%
068 Olithink_5.3.2.JA_xb     A32 1 2327  35  35 304 52% 2311 21%
069 Amy_0.8.JA_xb            A32 1 2297  35  35 320 48% 2312 21%
070 Myrddin_0.86.JA_xb       A32 1 2281  36  36 302 48% 2291 21%
071 TJchess 1.1U             A32 1 2270  33  34 330 48% 2281 24%
072 Bitfoot 150922.JA        A32 1 2270  35  35 330 58% 2204 15%
073 Natwarlal_0.14.JA_xb     A32 1 2269  34  34 312 52% 2250 23%
074 MangoPaola_1.1.JA_xb     A32 1 2256  35  35 312 50% 2246 20%
075 Sungorus 1.4 JA          A32 1 2235  34  34 306 47% 2254 24%
076 KmtChess_1.21.JA_xb      A32 1 2187  35  35 318 51% 2183 22%
077 Rattate_Nosferatu.JA_xb  A32 1 2177  35  35 318 48% 2189 17%
078 NGplay_9.86.JA_xb        A32 1 2172  34  34 314 50% 2168 23%
079 Scidlet_2.61b2.JA_xb     A32 1 2170  36  36 318 53% 2141 13%
080 Resp_0.19.JA_xb          A32 1 2141  34  34 330 51% 2133 18%
081 Clubfoot 150907.JA       A32 1 2118  37  36 320 63% 2008 14%
082 Floyd 0.7 JA             A32 1 2083  37  36 326 56% 2024 12%
083 DanChess_1.04.JA_xb      A32 1 2079  36  36 310 50% 2081 17%
084 Kurt 0.9.2.2 JA          A32 1 2037  34  35 330 47% 2065 18%
085 Robocide 28.12.14.JA     A32 1 2027  33  33 354 50% 2021 18%
086 Witz_Alpha21.JA_xb       A32 1 2008  35  35 312 47% 2031 20%
087 Woodpecker_2.11.JA_xb    A32 1 1983  36  36 306 50% 1983 15%
088 Knightcap_3.7F.JA_xb     A32 1 1974  36  36 292 52% 1956 17%
089 AdroitChess0.4 JA        A32 1 1963  36  36 318 47% 1984 16%
090 BikJump v1.8             A32 1 1960  33  34 340 46% 1989 22%
091 Sjeng_1.12.JA_xb         A32 1 1937  36  36 298 50% 1939 16%
092 Gunborg_1.39.JA_uci2xb   A32 1 1933  39  38 276 61% 1845 20%
093 Leonidas_r83.JA_xb       A32 1 1930  36  36 300 48% 1950 19%
094 ZCT-0.3.2500             A32 1 1909  36  37 310 42% 1979 12%
095 Faile_1.44.JA_xb         A32 1 1902  35  35 288 47% 1924 28%
096 Samchess_JA_xb           A32 1 1895  37  37 296 43% 1955 17%
097 Mephisto Roma Turbo      W64 1 1892  79  83  56 37% 1994 16%
098 Cilian_4.14.JA_xb        A32 1 1888  34  34 310 51% 1873 26%
099 Ecce rev. 508            A32 1 1851  36  36 306 44% 1902 16%
100 Sayuri 2015.10.01        A32 4 1847  37  36 312 55% 1788 14%
101 Colchess_8.0.JA_xb       A32 1 1813  38  38 258 51% 1793 24%
102 Smash 1.03 JA            A32 1 1811  37  37 318 49% 1808 12%
103 Surprise_4.3.b13.JA_xb   A32 1 1809  50  50 158 49% 1814 16%
104 Claudia v. 0.5           A32 1 1807  54  54 150 51% 1797 13%
105 Zzzzzz_3.5.1.JA_xb       A32 1 1722  38  38 248 50% 1696 31%
106 Hoichess_0.12.1.JA_xb    A32 1 1708  36  37 300 48% 1711 19%
107 Chenard_2015.08.15.JA_xb A32 1 1696  43  44 246 47% 1687  9%
108 Kitteneitor_060513.JA_xb A32 1 1684  39  39 238 46% 1683 34%
109 Tscp_1.8.1.AB_xb         A32 1 1681  40  41 250 45% 1701 17%
110 Jester_0.84.JA_xb        A32 1 1677  43  43 232 50% 1634 13%
111 Rocinante 2.0 JA         A32 1 1675  38  38 298 51% 1623 18%
112 Colossus 4.0 100X        C64 1 1674 237 208  10 80% 1297 20%
113 Pulse 1.5-cpp            A32 1 1619  37  38 302 53% 1533 29%
114 Mephisto Roma 68020 UCI  W64 1 1613 124 134  20 35% 1715 30%
115 VIRUTOR CHESS 1.1.1      A32 1 1486  42  43 302 51% 1432 10%
116 Superpawn b108 JA        A32 1 1449  49  49 198 53% 1419 19%
117 K2 v.075                 A32 1 1437  54  53 192 57% 1372  6%
118 Chess for Android        A32 1 1437  43  43 295 52% 1374 13%
119 Chess Titans             W64 1 1336 212 204   7 57% 1306 29%
120 Trappy_Beowulf_2.0.JA_xb A32 1 1196  46  47 290 37% 1315  9%
121 Colossus 4.0             C64 1 1170 170 186  14 36% 1253 14%
122 Byak 8.10.14.JA          A32 1 1119  49  51 212 24% 1370 18%
123 Xadreco_5.7.JA_xb        A32 1 1014  56  61 206 14% 1390  9%
124 Novag Secondo            TTC 1  942 277  23   6 42%  964 17%
125 OliveChess 0.2.7         A32 1  576 379-338 170  0% 1503  0%

Rapidroid test platform:
* GT-N7100 1.6 * 4 + 256MB hash: All Android progs
* GT-N5105 1.6 * 4 + 256MB hash: All Android progs
* Codegen Novatab 1.4 * 4 + 256MB: Single thread Android progs,
* Polypad 1010IPS tablet 1.61 * 2 + 128MB: Single thread Android progs
* HTC Diam 528Mhz, 16MB hash: Windows Mobile
* i7 M620 2.67 Ghz dual + Arena 3.5 + 2GB hash: Windows 64
* iPhone5S A7 1.3 Ghz * 2: iOS progs
* DosBox 1.74: DOS progs
* WinVICE 2.24: Commodore-64 progs
* Messtiny UCI adapters or CB-Emu2014: Mephisto progs
* Openings: 20 ply from Adam Hair or 16 ply from TCEC, no Q exchange, +0.15 to +0.40 eval by Stockfish and Komodo, depth 20 minimum, played twice both sides
* Repeating openings and twin games not allowed between two programs
* Tablebases and pondering off
* Time control: 10 to 30 sec/move or 600+0 to 1800+5 or closest known by both programs.

7 comments:

Dariusz Domagała said...

Gurcan, thank you for your work!

It is unfortunate that critter is not developed.
I'm glad to see Phalanx. It's engine with very interesting style of play. Since many years, from time to time I playing against in long game and analyzing sharp positions.

Unfortunately, the Phalanx does not support the UCI protocol on android. Analyses would be easier in a good application.
Maybe Jim Ablett could extend this engine in the UCI support ?

Regards, Dariusz

vadlm said...

Добавьте пожалуйста в рейтинг Senior Chess

Archie Shipp said...

I am currently running a 10 game tournament with Gull 3 and Ivanhoe 9.47c beta versus about a dozen of the top free engines (Robolitto .085e41 to Stockfish 7) So far, Gull has come in around 3150 and is an equal to Critter 1.6. Still working on Ivanhoe. Won't be testing Fritz as it is commercial.

Gurcan Uckardes said...

Archie, thanks for the feedback. Rapidroid March-2016 will include Gull, Fritz and Ivanhoe. On Exynos 4412, Fritz and Gull are both close to 3100, still a little bit lower than Critter. Ivanhoe is very similar to Robbolito, close to 3000. They all played 300+ games already. I think Gull could perform even better. My sixth sense tells me there's a little bug in JA builds making the SMP code less efficient than expected. It doesn't use all available cores all the time, simply unstable...

Gurcan Uckardes said...

@Vadlm: Senior Chess shoukd be around 1900-2100. I will test it with 20 manual games whenever possible.

Archie Shipp said...
This comment has been removed by the author.
Archie Shipp said...

Here is the data on Gull 3 and Ivanhoe 9.47c beta. Versus 11 other top engines, 10 games per match, 110 games per engine; Gull scored 60% and has a rough ELO of 3159. Ivanhoe scored 40% with a rough ELO of 3019. (Sidenote: Senpai 1.0 got seriously worked over by the rest of the engines in this list. Not a worthy performance from a supposedly 3000+ engine.)

Engine ELO Stockfish Sting Gull Fire Critter DSaros Robbolito Komodo Texel BMamba Ivanhoe Senpai Percentage ELO + / - Perfromance
Stockfish_7.0.JA_uci.arm7 3349 xxx 7.5 8 7.5 7 7.5 9.5 8.5 9 9.5 9.5 10 0.85 301 3390
Sting_4.84.JA_uci.arm5 3119 2.5 xxx 5.5 5 6.5 5 7.5 7.5 6.5 6.5 8.5 8 0.63 92 3181
Gull_3.JA_uci.arm7 2 4.5 xxx 4 5 8 7 5.5 7 8.5 7.5 7.5 0.60 70 3159
Firenzina_2.4.1_uci.arm5 3116 2.5 5 6 xxx 3.5 5 6.5 7.5 8 5 7 8.5 0.59 63 3152
Critter_1.6a_uci.arm5 3150 3 3.5 5 6.5 xxx 6 5.5 5 6 6.5 7.5 9.5 0.58 56 3145
Deep Saros_2.3f.JA_uci.arm7 3018 2.5 5 2 5 4 xxx 4.5 6.5 4 7 7 8 0.50 0 3089
Robbolito_.085e41_uci.arm5 2971 0.5 2.5 3 3.5 4.5 5.5 xxx 5 8 6 4 7.5 0.45 -35 3054
Komodo_3.AB_uci.arm7 3022 1.5 2.5 4.5 2.5 5 3.5 5 xxx 7.5 4.5 6 7 0.45 -35 3054
Texel_1.05_uci.arm7 3036 1 3.5 3 2 4 6 2 2.5 xxx 5.5 4.5 9.5 0.40 -70 3019
BlackMamba_2.0_uci.arm7 3083 0.5 3.5 1.5 5 3.5 3 4 5.5 4.5 xxx 5 7.5 0.40 -70 3019
Ivanhoe_9.47c.beta_uci.arm 5 0.5 1.5 2.5 3 2.5 3 6 4 5.5 5 xxx 10 0.40 -70 3019
Senpai_1.0.JA_uci.arm7 3024 0 2 2.5 1.5 0.5 2 2.5 3 0.5 2.5 0 xxx 0.15 -301 2788