At 30% of the long run, Stockfish 5 leads again as expected.
There are two issues which remain to be fixed whenever possible:
1) Distortion caused by versions of the same program > I guess Stockfish 5 has +20 ELO thanks to the games played against other fishes.
2) Overall calibration vs other reference rating lists > I guess the list may need +150 ELO but i need to validate this with a few (!) manual games vs tabletop computers and also Windows programs.
BLITZOID RANKING / 05-JUL-2014
12188 GAMES PLAYED BY 92 PROGRAMS
COMPUTED WITH BAYESELO, OFFSET: 2117
Ra Name Elo + - Gam Sco Oppo Dra
1 Stockfish 5 3015 39 38 268 78% 2803 31%
2 Stockfish DD 2974 37 36 268 74% 2803 39%
3 Stockfish 3 2903 33 33 296 60% 2831 37%
4 Stockfish 4 2901 33 32 296 60% 2831 46%
5 Stockfish 2.3.1 2869 33 32 294 55% 2822 45%
6 Critter 1.4 2858 33 33 294 54% 2823 40%
7 Critter 1.6a 2853 33 33 294 54% 2822 43%
8 BlackMamba 2.0 2785 38 37 280 64% 2651 33%
9 Critter 1.2 2785 36 36 290 58% 2708 33%
10 Stockfish 2.0 2780 35 35 292 50% 2772 32%
11 RobboLito 0.085e4l 2722 35 35 282 46% 2746 35%
12 Komodo32 2.03 JA 2703 35 35 288 47% 2725 31%
13 RobboLito 0.085g3l 2684 36 36 280 52% 2657 33%
14 Senpai 1.0 2668 35 36 284 43% 2716 30%
15 Komodo32 3 AB 2665 36 36 280 48% 2673 28%
16 Texel 1.04 2607 39 39 266 53% 2577 21%
17 Gaviota v1.0 2582 36 36 280 56% 2534 30%
18 Komodo32 1.3 JA 2579 37 38 278 46% 2608 26%
19 Texel 1.03 2558 39 39 264 49% 2563 23%
20 Toga II 3.0 2503 37 37 282 50% 2513 24%
21 IvanHoe 9.46b 2483 38 38 274 52% 2468 22%
22 Arasan 15.2 JA 2420 38 38 274 49% 2432 20%
23 Gaviota v0.86 2404 38 38 268 48% 2428 24%
24 Toga II 1.4.1SE 2388 37 37 276 43% 2451 27%
25 Toga II 2.0 JA 2375 36 36 272 49% 2393 26%
26 DiscoCheck 3.7.1 2361 37 37 272 47% 2391 24%
27 Texel 1.01 2352 37 37 280 44% 2406 23%
28 Arasan 13.4 2327 36 36 270 50% 2335 26%
29 DiscoCheck 4.0.1 2299 36 36 274 48% 2313 27%
30 Arasan 14.0.1 2286 36 36 276 44% 2330 28%
31 Crafty_23.4.JA_xb 2286 36 36 272 50% 2290 23%
32 GNU Chess 5.50-32 2276 35 36 272 47% 2302 29%
33 gaviota v0.84 2262 37 36 268 51% 2261 24%
34 Crafty_23.5.JA_xb 2254 37 37 272 46% 2297 19%
35 Alfil 12.10 w32 2246 36 36 266 51% 2238 25%
36 RedQueen 1.1.2 2244 37 37 274 45% 2285 20%
37 Rhetoric 1.4 2242 38 38 240 47% 2263 27%
38 Rodent 1.00 2235 36 36 264 52% 2215 31%
39 RedQueen 1.1.3 TCEC 2233 38 39 270 42% 2301 18%
40 Rotor 0.7a 2214 35 36 270 50% 2219 27%
41 Rodent 0.18.0 2210 36 36 264 56% 2166 30%
42 cheng3 1.07 JA 2182 37 37 264 48% 2191 20%
43 Rotor 0.8 2176 37 37 262 50% 2180 25%
44 GarboChess 3 2172 37 37 266 56% 2128 23%
45 Scorpio_2.7.JA_xb 2167 37 37 264 51% 2159 22%
46 Daydreamer 1.75 JA 2164 36 36 268 48% 2176 28%
47 gaviota v0.83 2140 37 37 262 47% 2165 22%
48 Sloppy_0.23.JA_xb 2127 36 36 264 48% 2140 27%
49 Pepito v1.59 2091 37 37 260 49% 2096 21%
50 Danasah_4.88.JA_xb 2079 37 37 262 47% 2098 24%
51 GNU Chess 6.0.2 2066 36 36 260 48% 2081 28%
52 DanasahZ_0.4.JA_xb 2064 36 37 258 48% 2075 26%
53 Tucano_1.04.AB_xb 2063 37 37 260 50% 2061 20%
54 DoubleCheck 2.7 2055 38 38 260 50% 2050 15%
55 DoubleCheck 2.6 JA 2051 38 38 260 49% 2057 17%
56 Danasah_5.06.JA_xb 2032 37 37 256 56% 1982 27%
57 Danasah_4.66.JA_xb 2013 38 38 258 52% 1990 20%
58 BetsabeII_1.30.JA_xb 2006 38 38 260 50% 2000 16%
59 Typhoon_1.0.r358.JA_xb 2002 37 37 260 51% 1995 20%
60 Diablo 0.5.1b JA 1994 37 37 260 53% 1966 24%
61 Olithink_5.3.2.JA_xb 1976 39 39 260 54% 1945 16%
62 GreKo_9.8.AB_uci 1962 37 37 260 49% 1965 23%
63 Greko_8.2_uci 1961 37 37 256 52% 1943 27%
64 GreKo_9.0.JA_uci 1957 37 37 260 53% 1932 20%
65 GreKo_10.0.JA_xb 1940 37 37 260 50% 1936 23%
66 Phalanx_XXIII.JA_xb 1939 39 39 260 47% 1959 13%
67 Sungorus 1.4 JA 1898 39 39 260 50% 1892 16%
68 BetsabeII_1.22.JA_xb 1852 40 39 256 55% 1805 14%
69 TJchess 1.1U 1844 39 39 260 51% 1822 22%
70 Myrddin_0.86.JA_xb 1835 39 39 260 55% 1788 16%
71 DoubleCheck 2.3 1816 40 40 260 51% 1801 16%
72 Natwarlal_0.14.JA_xb 1814 40 40 260 51% 1798 13%
73 KmtChess_1.21.JA_xb 1809 39 39 260 49% 1812 19%
74 Jazz 6.40 JA 1783 39 39 260 45% 1810 22%
75 Scidlet_2.61b2.JA_xb 1758 40 40 260 51% 1741 16%
76 Jazz v444 JA 1736 39 39 260 50% 1732 21%
77 Jazz v5.01 JA 1720 39 39 260 52% 1704 20%
78 Sjeng_1.12.JA_xb 1598 41 41 258 52% 1573 12%
79 BikJump v1.8 1571 41 41 258 51% 1555 14%
80 AdroitChess0.4 JA 1540 42 43 254 48% 1554 11%
81 Leonidas_r83.JA_xb 1497 42 42 254 50% 1484 19%
82 AdroitChess 0.3 1486 43 44 250 47% 1493 15%
83 ZCT-0.3.2500 1475 45 45 248 52% 1439 9%
84 Sjaak_4.68.JA_xb 1467 45 45 244 58% 1369 11%
85 BikJump v2.1P 1464 43 44 250 48% 1467 14%
86 Tscp_1.8.1.AB_xb 1372 45 45 244 50% 1355 11%
87 Zzzzzz_3.5.1.JA_xb 1339 44 44 242 48% 1351 18%
88 Rocinante 2.0 JA 1287 45 46 238 51% 1275 11%
89 VIRUTOR CHESS 1.1.4 1140 46 47 234 38% 1242 12%
90 VIRUTOR CHESS 1.1.1 1135 46 47 234 38% 1243 11%
91 Chess for Android 971 52 55 234 21% 1264 8%
92 Simplex 0.9.8 783 68 49 234 8% 1288 2%
Blitzoid test platform:
* Samsung Galaxy Note II @ 1.6 Ghz without downscaling
* 64MB hash tables where selectable
* 4 cpu threads where selectable
* Own books disabled and replaced by Silver Opening Suite positions (15 of 50 played so far)
* Opening positions played twice with different colors
* Tablebases and pondering off
* GUI: Aart Bik's Chess for Android
* Time control: 5 sec/move
Very interesting rating list. I like testing and watching when engines play against yourself. Unfortunately, I can not find some engines, eg. Black Mamba 2.0 for android (it's impossible to download form author site). Can you share all android engines in one arch (zip, rar) ? Best regards from Poland!
ReplyDeleteThanks for your comment. I will soon prepare a file repository on box.com to let interested people get the engines.
ReplyDelete