Kinda-Sorta Basic Benchmarking in Java
ryans-mba:wargame ryanrampersad$ ./run.sh
======= WarGame 2.1 =======
Number of threads: 4
Variation magnitude: -4 [0.01%]
Prime time: 60 seconds
Maximum tests: 100
Update frequency: 1 fps
Speed: 6.0738 (g/ms) - Test #19 at 0 seconds
Games completed: 1,287,878
Tests completed: 19
Speed improvement: -6.06%
from 6.4425 (g/ms) to 6.0741 (g/ms)
Final confidence range:
6.0751 (g/ms) > 6.0741 (g/ms) > 6.0739 (g/ms)
Final rate: 0.1646 (ms/g)
Final speed: 6.0741 (g/ms)
Final score: 6
To run the WarGame on not-Windows:
To run the WarGame on Windows:
To run the WarGame with options on not-Windows:
java -cp "bin" wargame/App
To compile to WarGame on not-Windows:
javac -d "bin" -cp "source/wargame/" source/wargame/*.java
This will run like so:
- Priming phase: running 60 seconds worth of games
- Testing phase: check if the speed is within 00.01% since the last test
- Retesting phase: if the speed is not within the range, check in a few seconds
- Results phase: display the results
The WarGame program has five primary settings:
- Threads: by default, the number of logical cores available
- Variation magnitude: the change allowed during the testing phase, as a percentage. By default, 00.01%.
For example, if the speed at the beginning of the testing phase is 5.4465, then the confidence range would be
5.5518 (g/ms) > 5.4465 (g/ms) > 5.4419 (g/ms)
- Prime time: by default, 60 seconds
Prime time allows the program to reach a period of stability before testing begins.
- Maximum tests: by default, 100 tests
- Update frequency: by default, 1 fps
Set the number of screen updates per second on progress. 30 frames per second would be realtime.
- 2.1: Saturday, January 11th, 2014
- 18.104.22.168: Thursday, January 17th, 2012 – initial release
The WarGame has various modes: Help, Short and Interactive.
java App s
This mode will run the WarGame program with 1% confidence and a 10 second priming phase.
java App i
This mode will run the WarGame program with user specified settings, those being threads, confidence and prime time.
The Help mode is self-explanatory, this page provides the same contents and more.
2.1 Update Notes
The 2.1 update will round to a specific number, which is based on speed, or 1/rate. Because this is a kinda-sorta bench mark, this is fine. While the whole number is shown, the raw numbers are still given as well.
For example, in the output log above, the computer has a final speed of 6.0741 (g/ms). The final score is simply
round(final speed), which is simply 6. If the final speed were above 6.5, it would have scored 7. It is that simple and very arbitrary, but the value is an easy number to compare to others.
There are techincal updates as well:
- Printing: instead of printing as frequently as possible, the screen is only updated once per second with the default values
- Code: the code has been rewritten quite substantially to make better use of a class architecture
- Code: attempts at handling the ctrl+c case of escaping
- Results: a URL is provided to pre-fill results into our Google Doc
The WarGame was designed to scale well on different types of computers, assuming they could run Java in the first place. There are multiple aspects that control the scaling system.
The original WarGame was always set to 1 million games of war. While this was decent for my computer at the time, on a slow laptop, the program could take over twenty minutes. On a fast machine (like a 10 thousand dollar dual-Xeon server), the 1 million game process could be over in just forty seconds. To fix this, the war game will run until a condition is met, regardless of how long it takes to actually get there - it could be twenty minutes or it could be just under two minutes. The solution to this problem is the variation magnitude. This is the percentage change between one instant's speed and another instant's speed. If less change than the variation magnitude, then the program has found an equilibrium on the machine and it is probably safe to say that is as fast as it can go.
After the priming time, which allows the JVM to allocate as much memory as it can for this program, to get the processors up to full usage and to max out thermal conditions, the tests begin. The testing phase is simple - it compares the speed to the last test's speed within in the bounds of the variation magnitude. If the test is successful, that is, the speeds have a difference of less than the magnitude, then the program ends. Otherwise, the tests continue in this same way until it is successful. The testing period has a delay added to it so that fast computers that have a high head room can be accurately tested while slower computers can be completed faster.
When I was in tenth grade, I spent a week or so coding the original WarGame — a simulator for playing war. The sim was fine, but my favorite past time was running it on various computers with a million games to see how long it would end up taking. In 2012, I built a new computer with an incredibly fast processor and I needed something better to test it than my original WarGame. The original WarGame tested to a user specified value (usually one million in my case) and it only used a single thread.
Thus, the WarGame 2 was made. The second generation WarGame is the same at its core -- the game of War simulating two people playing cards. But the advancements are in the details: multiple threads play simultaneously (or whatever is an approximate of that) and a testing method that scales based on hardware capabilities. Threading makes full use cores available while the scaling method used allows for some degree of precision and consistency between runs. A faster computer will run more games for longer to ensure that its speed isn't fooling anyone.
WarGame 2.1 Results
Please run the WarGame 2.1 and use the link given to you to submit your data.
WarGame 22.214.171.124 Results
These results are from my own tests (as of WG2-126.96.36.199) on a few computers I have access to and results from others. To submit you results, you must email them to me.