This code is simply "measuring" (if you can call it that) the JIT warmup time.
By the time the code is optimized the first loop is probably almost done.
Also have to look out for that fact that it can often be smart enough to see that the loops accomplish nothing, and remove them entirely.
If you want any kind of meaningful data out of a Java microbenchmark you're pretty much going to have to use JMH.http://openjdk.java.net/projects/code-tools/jmh/http://java-performance.info/jmh/
Do you have any opinion on using this KeyValue class for things like storing Entity X/Y which would be constantly read/written to? I imagine it would be more optimized in that case if I were to encapsulate the x/y coordinates in an Object like a Point or something as well.
HashMaps are pretty bad for cache locality, good ol' ArrayList has your back 99% of the time. And you know what they say about premature optimization. Have you attempted to time ArrayList<Point> etc. vs. Map? I'll bet List is faster.
It's also almost guaranteed that you don't need every last drop of perf anyway. Don't think too hard about it until you need to.