The promise is kept on time. So, you may come taste the mobile insanity that nobody else has gone into but me. Despite neverending technical issues which caused me a lot of deleted, replayed, adjudicated games and interrupted tourneys, i like the way things progress. Ugh n' buff!
The loss of tempo may be blamed on countless new engines and versions which kept appearing in march. I prefered to introduce them into the pool on the run, rather than rigorously testing separately before introduction. As expected, some of them bounced back, causing headache.
After all, thanks to a debug and replay session done at the end of Round-3, the resulting PGN database is clean, free of termination errors, therefore good for ELO calculation.
Changes were:
* 104 engines increased to 120, only Gunborg 1.0 not yet evaluated,
* CPW Engine excluded because of "ponder always on" issue,
* Komodo 3 stopped after Doch and Beekay came in, enough Komodos...
* First non-Android member Colossus 4.0 C64 introduced with 2 versions (:-))
* Rockchip 3188 introduced as new processor capable to simulate Exynos 4412.
To do's are:
* Introduce Hiarcs on iOS5,
* Introduce Shredder Android,
* Introduce Chess Genius Android,
* Introduce 2 new RK3188 devices, for 5 Androids in total.
Targets for next release expected in May-2015 are:
* Error margins below +/-60 ELO from 89,
* Number of games standard deviation reduced to 16% from %20,
* Standard deviation of score% reduced to 9% from %13,
* Average ELO change +/-50 from 87.
As seen above, accuracy indicators are not healthy yet. Quite normal... They will need about 100 games per engine to start speaking seriously.
If one states 3300+ is too exaggerated for Android, i'd admit it could be true for the moment, in this very early steps. In parallel to increased number of samples, i expect the list will get compressed, the way it happened in previous Rapidroid experiment.
BAYES RATINGS AFTER 3112 GAMES PLAYED BY 120 PROGRAMS
Rnk Name O/S T Elo + - ga sco oppo dra
1 Stockfish 6 A32 4 3339 91 82 54 82% 3099 31%
2 Komodo 8 A32 4 3305 88 81 54 77% 3103 31%
3 Firenzina 2.4.1 xTreme A32 4 3176 78 76 54 58% 3117 39%
4 Sting SF 4.8.4 JA A32 4 3136 80 78 54 60% 3055 31%
5 BlackMamba 2.0 A32 4 3123 74 74 54 50% 3123 52%
6 Critter 1.6a A32 4 3122 74 74 54 50% 3123 48%
7 BeeKay A32 4 3118 75 76 54 49% 3123 46%
8 Texel 1.05 A32 4 3092 76 77 54 45% 3126 39%
9 Komodo32 3 AB A32 1 3041 92 91 36 53% 3024 50%
10 cheng4 0.38 A32 4 3006 81 80 54 56% 2970 30%
11 RobboLito 0.085e4l A32 1 2998 79 76 56 61% 2925 39%
12 DeepSaros ver.2.3f A32 1 2976 80 80 54 51% 2973 28%
13 Senpai 1.0 A32 4 2926 76 76 56 50% 2933 43%
14 Arasan 17.5 A32 4 2905 76 75 56 55% 2871 32%
15 Gaviota v1.0-d A32 4 2904 78 76 54 59% 2831 37%
16 Fruit reloaded 2.1 A32 1 2824 80 85 56 30% 2976 29%
17 Grapefruit 1.0 A32 4 2822 78 78 54 48% 2840 30%
18 Cyclone 3.4 A32 1 2818 75 78 56 33% 2944 48%
19 Doch32 1.3.4 JA A32 1 2815 73 75 56 40% 2881 41%
20 Toga II 3.0 A32 1 2802 78 79 54 44% 2842 30%
21 Deep Saros 0.9 A32 4 2751 82 83 54 44% 2784 22%
22 DiscoCheck 5.2.1 A32 1 2751 84 81 50 63% 2663 30%
23 Hakkapeliitta dev 120115 A32 1 2730 82 82 52 48% 2741 27%
24 Deuterium v14.3.34.130 A32 1 2699 80 79 50 55% 2669 34%
25 Murka 3 ARM UCI A32 1 2698 77 77 54 50% 2687 41%
26 IvanHoe 9.46b A32 4 2693 80 81 52 46% 2720 31%
27 Scorpio_2.7.7.JA_xb A32 1 2691 91 85 52 74% 2497 25%
28 RedQueen 1.1.97 A32 4 2689 80 79 52 57% 2639 33%
29 Rhetoric 1.4.1 A32 1 2634 81 79 54 58% 2563 28%
30 Rodent 1.7 build 1 A32 1 2618 80 81 52 44% 2647 31%
31 Crafty_24.1.JA_xb A32 1 2601 84 83 54 54% 2566 15%
32 Gull 1.2 JA A32 1 2594 78 80 52 39% 2650 37%
33 GNU Chess 5.50-32 A32 1 2563 83 81 54 57% 2509 22%
34 TheMadPrune 1.7.04 A32 4 2540 95 94 42 51% 2530 12%
35 Tucano_5.00.JA_xb A32 1 2512 83 80 52 62% 2427 31%
36 Bison 15.1 A32 1 2468 82 84 54 43% 2519 19%
37 Alfil 12.10 A32 1 2468 81 80 58 53% 2437 17%
38 Rotor 0.8 A32 1 2431 80 82 56 41% 2499 21%
39 Daydreamer 1.75 JA A32 1 2429 77 77 58 49% 2441 29%
40 Dirty_030411.JA_xb A32 1 2372 77 78 54 47% 2394 31%
41 Vajolet 2.03 A32 1 2372 75 76 58 47% 2393 26%
42 Danasah_5.07.JA_xb A32 1 2353 77 78 54 44% 2396 33%
43 BBChess 1.3b JA A32 4 2349 84 90 54 28% 2532 19%
44 Octochess arm r.6556 A32 1 2335 75 75 56 50% 2340 32%
45 Phalanx_XXIV.JA_xb A32 1 2322 82 80 54 58% 2260 20%
46 Sloppy_0.23.JA_xb A32 1 2319 78 79 52 46% 2350 31%
47 Pepito v1.59 A32 1 2309 79 80 56 46% 2343 20%
48 GarboChess 3 A32 1 2302 82 81 56 56% 2255 20%
49 GreKo_12.5.JA_xb A32 1 2299 81 80 54 54% 2263 22%
50 DanasahZ_0.4.JA_xb A32 1 2283 82 82 50 49% 2285 30%
51 Glaurung Mainz A32 1 2236 111 125 36 25% 2458 6%
52 Ifrit_M18.JA_uci2xb A32 1 2232 86 88 52 40% 2316 15%
53 Jazz Orchestra 8.40 A32 4 2228 89 84 56 71% 2032 23%
54 BetsabeII_1.47.JA_xb A32 1 2215 79 77 56 61% 2143 25%
55 Typhoon_1.0.r358.JA_xb A32 1 2172 76 75 56 54% 2148 32%
56 Amy_0.8.JA_xb A32 1 2166 79 78 60 54% 2133 15%
57 Diablo 0.5.1b JA A32 1 2155 77 75 62 57% 2098 21%
58 Pawny_1.0.JA_uci2xb A32 1 2136 76 76 60 49% 2136 22%
59 Sungorus 1.4 JA A32 1 2119 77 76 60 58% 2061 22%
60 Natwarlal_0.14.JA_xb A32 1 2114 80 80 54 50% 2111 22%
61 Simplex 0.9.8 A32 1 2083 78 78 62 53% 2033 13%
62 MangoPaola_1.1.JA_xb A32 1 2079 80 78 58 58% 2019 16%
63 Olithink_5.3.2.JA_xb A32 1 2076 77 77 60 52% 2065 17%
64 Myrddin_0.86.JA_xb A32 1 2074 84 81 54 62% 1981 20%
65 TJchess 1.1U A32 1 2070 76 77 60 47% 2069 27%
66 Rattate_Nosferatu.JA_xb A32 1 2062 78 80 62 44% 2106 11%
67 Maverick 0.60 arm A32 1 2051 75 75 62 48% 2063 23%
68 NGplay_9.86.JA_xb A32 1 2041 85 84 48 52% 2031 29%
69 Scidlet_2.61b2.JA_xb A32 1 2023 82 81 54 55% 1986 20%
70 Ges_1.34.JA_xb A32 1 2005 116 105 34 72% 1841 15%
71 Gk_0.90.JA_xb A32 1 1996 122 121 26 56% 1914 19%
72 KmtChess_1.21.JA_xb A32 1 1983 86 86 46 54% 1950 30%
73 Resp_0.19.JA_xb A32 1 1978 79 79 58 50% 1958 21%
74 DanChess_1.04.JA_xb A32 1 1977 80 84 60 38% 2072 10%
75 Kurt 0.9.2.2 JA A32 1 1909 80 81 58 47% 1921 17%
76 Sjaak II 1.0 A32 1 1870 88 83 60 74% 1650 18%
77 Samchess_JA_xb A32 1 1868 88 91 46 39% 1954 17%
78 robocide A32 1 1865 83 81 62 60% 1778 13%
79 Faile_1.44.JA_xb A32 1 1864 92 88 44 60% 1790 16%
80 Romichess_p3k.JA_xb A32 1 1860 84 90 54 31% 1995 19%
81 Surprise_4.3.b13.JA_xb A32 1 1851 126 117 30 65% 1709 1%
82 Fairymax_4.8L.AB_xb A32 1 1834 104 107 32 39% 1928 28%
83 Knightcap_3.7F.JA_xb A32 1 1834 92 95 42 42% 1903 21%
84 Witz_Alpha21.JA_xb A32 1 1823 91 88 48 64% 1713 19%
85 ZCT-0.3.2500 A32 1 1820 81 82 58 46% 1859 9%
86 BikJump v1.8 A32 1 1805 78 80 58 43% 1860 17%
87 Woodpecker_2.11.JA_xb A32 1 1774 89 92 44 40% 1850 20%
88 Clubfoot r2293 JA A32 1 1771 94 100 46 34% 1886 20%
89 Cilian_4.14.JA_xb A32 1 1751 77 77 58 47% 1777 24%
90 AdroitChess0.4 JA A32 1 1751 80 82 58 44% 1791 16%
91 Sjeng_1.12.JA_xb A32 1 1742 91 90 48 56% 1682 17%
92 Leonidas_r83.JA_xb A32 1 1742 87 89 46 45% 1777 15%
93 Ecce rev. 508 A32 1 1736 81 82 58 45% 1778 7%
94 smash A32 1 1731 85 82 62 64% 1593 11%
95 Umax_4.8W_xb A32 1 1715 96 103 36 32% 1856 25%
96 Colchess_8.0.JA_xb A32 1 1686 85 86 50 47% 1706 22%
97 Bismark_1.1 A32 1 1651 83 80 62 65% 1523 19%
98 Claudia v. 0.5 A32 1 1556 113 120 30 37% 1655 20%
99 Sayuri 2015.03.02 A32 4 1533 83 83 60 51% 1534 15%
100 Zzzzzz_3.5.1.JA_xb A32 1 1532 84 85 50 47% 1553 30%
101 zurichess A32 1 1519 110 103 36 72% 1312 28%
102 Jester_0.84.JA_xb A32 1 1485 93 90 48 67% 1315 25%
103 Tscp_1.8.1.AB_xb A32 1 1482 90 91 48 51% 1452 23%
104 Rocinante 2.0 JA A32 1 1476 76 76 66 58% 1391 27%
105 Chenard_2014.07.11.JA_xb A32 1 1445 97 96 48 58% 1350 17%
106 Kitteneitor_060513.JA_xb A32 1 1443 84 84 50 49% 1433 38%
107 Pulse 1.5-cpp A32 1 1414 72 72 70 60% 1303 37%
108 VIRUTOR CHESS 1.1.1 A32 1 1375 81 78 68 68% 1188 22%
109 Colossus 4.0 100X C64 1 1370 287 249 6 83% 1058 33%
110 Chess for Android A32 1 1325 82 80 68 63% 1192 18%
111 Carnivor_1.0.JA_xb A32 1 1260 76 78 68 46% 1285 31%
112 Evolchess_0.6.JA_xb A32 1 1241 88 93 50 35% 1354 34%
113 Trappy_Beowulf_2.0.JA_xb A32 1 1181 90 94 66 35% 1340 9%
114 Toledo_Nano_2010.JA_xb A32 1 1147 117 125 36 24% 1457 19%
115 K2 v.071 A32 1 1105 125 136 36 18% 1513 19%
116 Colossus 4.0 C64 1 1044 278 236 6 50% 1058 0%
117 Byak 8.10.14.JA A32 1 987 105 121 44 19% 1264 7%
118 Xadreco_5.7.JA_xb A32 1 937 117 136 40 14% 1277 8%
119 Joanna_2002.JA_xb A32 1 858 126 98 48 10% 1286 8%
120 OliveChess 0.2.7 A32 1 772 145 20 52 5% 1290 6%
Rapidroid test platform:
* GT-N7100 4*1.7Ghz, 256MB hash: All Android progs,
* Codegen Novatab 4*1.4Ghz, 256MB hash: All Android progs,
* Polypad 1010IPS 2*1.61Ghz, 128MB hash: Android progs single core,
* HTC Diam 528Mhz, 16MB hash: Windows Mobile,
* i7 M620 2*2.67 Ghz, Arena 3.5, 2GB hash: Windows 64
* iPhone5S A7 2*1.3Ghz: iOS programs
* DosBox 1.74: DOS programs,
* WinVICE 2.24: Commodore-64,
* Messtiny UCI adapters or CB-Emu2014: Mephisto programs,
* Openings: 20 ply by A.Hair or 16 ply from TCEC, no Q exchange, +0.15 to +0.40 eval by Stockfish & Komodo @ D20, played twice both colors,
* Repeating openings and twin games not allowed between two programs,
* Tablebases and pondering off,
* Time control: 10" to 30"/move or 600+0 to 1800+5 or closest.
Pls put komodo 9 also in,
ReplyDeleteYes, indeed. It's in the schedule now.
ReplyDelete