November 8, 2014

How many more ELO's for double time?

This is not a new question indeed. One would google it and easily find countless topics.

Well, the question being numeric, the answers are countless too. If you would force me to give a unique answer, i could only say "More..."

Frankly, the ELO gain by doubling the thinking time of a chess engine seems to depend on various factors. The most decisive one, according to programmers, is the search depth which itself increases with more thinking time. Due to continously widening search tree, the increase however, is not linear, rather reversely exponential. This means the increase is significant in lower depths but tends to diminish beyond a treshold which is different from an engine to another.

Simply speaking with fictive numbers for a typical average engine may be:
1 min/game: 2500 ELO
2 min/game: 2600 ELO
4 min/game: 2650 ELO
8 min/game: 2675 ELO
16 min/game: 2688 ELO
32 min/game: 2694 ELO
64 min/game: 2697 ELO
128 min/game: 2699 ELO
256 min/game: 2700 ELO

It was just an example to visualize things. Without any doubt, steps and gains vary in real cases but always in similar trendline, quickly up, slowly closer to the maximum and bump!

Many results from the huge chess history suggest that doubling the thinking time would deliver as high as +200 ELO at lower depths and after "diminishing returns" along deeper plies, falls as low as +50 ELO starting from 20 ply depth. I've just taken a very rough average of hundreds of results i've been reading during years. If you may find different results out of range, no problem. They are already part of the database as rare cases.

Recently i had to conduct my own modest experiment, not because i wanted to add another silly test to the internet database but to my own purpose, i've had to make sure whether one of my Android engines is really using 1 second extra per move or not. A great opportunity to save the world, isn't it?

Discocheck is an abandoned engine after v5.2 and it had no updated compile for Android, only v4.3 available. I didn't use it in my previous 1 sec/move bullet tournaments because it's PV's were always reporting 2 sec/move time usage, for no reason. And when i raised to TC to 5 secs, it used 6 sec, 11 secs vs 10 secs, 16 vs 15 and so on. Always 1 second more than the setting. Weird stuff... Was it a safety overhead or real time usage for searching moves?

I've just had the idea to make this compile play with the other compile of the same version made by Jim Ablett. The latter used to fully respect the time usage but intermittently disconnected from the GUI which makes it impossible to include in tournaments. But in a head-to-head duel, this might be manageable.

I was succesfull in installing two different compiles of the same version under Chess for Android by renaming each under different suffixes and they were able to run simultaneously in a match.
They should tie but double time usage mattered.

At the end, a test of 100 games resulted in 66-34 score in favor of the 2 seconds version vs 1 second version. Once again, the world is saved thanks to my valuable effort and two fantastic findings can now be shared publicly with the humanity:

1) Doubling the time from 1 sec/move to 2 sec/move increases the ELO by 115 when experimented with Discocheck 4.3 engine under Android. Elo difference formula is 400*log(66/34).

2) Discocheck 4.3 compile by Aart Bik is still not eligible for time controlled tournaments under 15 seconds/move.

Beyond all above blah blah from the center of an insane dedication, more importantly, it feels good to see the pratice meets the theory. Kind of a statistical enjoyment...

No comments:

Post a Comment