Java-Gaming.org    
Featured games (79)
games approved by the League of Dukes
Games in Showcase (477)
Games in Android Showcase (109)
games submitted by our members
Games in WIP (536)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  for Jeff.... microbenchmarking.  (Read 3559 times)
0 Members and 1 Guest are viewing this topic.
Offline JasonB

Junior Member





« Posted 2004-01-08 21:14:34 »

This is for Jeff, since I know how much he !!!adores!!! microbenchmarking....

http://www.osnews.com/story.php?news_id=5602


Grin
Offline swpalmer

JGO Coder




Where's the Kaboom?


« Reply #1 - Posted 2004-01-08 22:56:49 »

Actually  it is great to see Java doing so well and his tests seemed reasonably fair given the that they are micro-benchmarks.

And his ultimate conclusion is what we all know... that speed is no longer an issue for choosing C or Java.

Now to find a way to improve Java's trig score...

Offline Jeff

JGO Coder




Got any cats?


« Reply #2 - Posted 2004-01-09 06:32:01 »

Not a bad article, he clearly understands the issues...

"Designing good, helpful benchmarks is fiendishly difficult. This fact led me to keep the scope of this benchmark quite limited.  ...  I didn't test string manipulation, graphics, object creation and management (for object oriented languages), complex data structures, network access, database access, or any of the countless other things that go on in any non-trivial program. "

So he knows that his benchmark is far from the whoel performance story.

Its not that I hate Microbnechmarks, its that I hate their misuse, which is prevelant.  I've used Microbenchmarks to help ferret out performance issues, you just need to understand what you are and aren't measuring.  And in a complex rum-time optimizing environment like the VM that often means needing access to the VM authors to interpret what you see.  (An advantage I had writing the Performance book that I realize most people don't :/ )

Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline JasonB

Junior Member





« Reply #3 - Posted 2004-01-09 06:47:28 »

any clues on the bad trig score?
Offline Jeff

JGO Coder




Got any cats?


« Reply #4 - Posted 2004-01-09 06:50:57 »

Havent dug into the benchmark.  My first wild guess would be that there is some built in trig support on the intel chip that doesn't support Java's rigorous FP definition.

Just out of curiosity,  who does game using the supplied trig functions?

Maybe this is dating me and prcoessors have goitten so good this isn't an issue but back when Iw as doing 2D games we always did a trig table.

Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Offline Mark Thornton

Senior Member





« Reply #5 - Posted 2004-01-09 07:12:12 »

Quote
My first wild guess would be that there is some built in trig support on the intel chip that doesn't support Java's rigorous FP definition.


No prizes, but that is the problem. When reducing 'large' arguments Intel use a 66bit version of pi which is inadequate to maintain the precision required by Java. They also restrict the domain of the trig functions to rather less than the full range of double. Java requires that Math.sin(1.0e300) produce a result accurate to within 1 lsb.

http://developer.java.sun.com/developer/bugParade/bugs/4345903.html
Offline Abuse

JGO Coder


Medals: 11


falling into the abyss of reality


« Reply #6 - Posted 2004-01-09 11:54:46 »

RFE for a FastMath class perhaps? Wink
(unless ofcourse thats what the bug report linked above is.... still havnt got my java.sun.com account sorted Sad)

Make Elite IV:Dangerous happen! Pledge your backing at KICKSTARTER here! https://dl.dropbox.com/u/54785909/EliteIVsmaller.png
Offline grayarea

Junior Newbie





« Reply #7 - Posted 2004-01-09 17:42:03 »

Looking at the previous thread "amazing java.lang.math performance" at http://www.JavaGaming.org/cgi-bin/JGNetForums/YaBB.cgi?board=Tuning;action=display;num=1063822912, I see that even with table lookups and fast versions of the functions there was a 3x improvement in performance. That doesn't explain the more than 10x difference between Microsoft's implementation of trig functions and Java 1.4.2, does it?
Offline JasonB

Junior Member





« Reply #8 - Posted 2004-01-09 18:12:14 »

Another one (this time on AMD Athlon XP/MP):

http://fails.org/benchmark.html
Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Reply #9 - Posted 2004-01-09 22:08:06 »

Wasn't there some sort of switch in the Sun JRE to use a less exact but much faster variant of Math?

Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline nonnus29

Senior Member




Giving Java a second chance after ludumdare fiasco


« Reply #10 - Posted 2004-01-09 23:09:25 »

Quote

Just out of curiosity,  who does game using the supplied trig functions?


I'm using lang.Math trig functions in my software renderer but i'm not using many trig ops in the main loop, just in the matrix class.  I used both the lang.Math functions and a lookup table in a mode7 rendering applet  and I didn't see any performance difference between the two.  Both methods gave the same frame rate.
Offline Jeff

JGO Coder




Got any cats?


« Reply #11 - Posted 2004-01-10 00:44:55 »

Quote
Wasn't there some sort of switch in the Sun JRE to use a less exact but much faster variant of Math?


Your thinkign of the "strictfp" loophole. Its a very very tiny loophole. Just enough to allow the Intel fused multiply and add.


Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Offline Mark Thornton

Senior Member





« Reply #12 - Posted 2004-01-10 11:25:04 »

Quote


Your thinkign of the "strictfp" loophole. Its a very very tiny loophole. Just enough to allow the Intel fused multiply and add.


Intel x86 processors don't have a fused multiply and add (the IA64/Itanium does). These fused instructions maintain additional precision which is not permitted by non strictfp java. The relaxtion which accompanied strictfp only allows additional exponent range during intermediate calculations. This permits the use of the Intel 80 bit reals for temporaries (provided the FPU is running in the mode which restricts the precision to that of normal double/float as appropriate).

There was a JSR (84) which proposed further relaxation to allow the use of fused multiply/add instructions as found in PowerPC for example. Unfortunately this was withdrawn apparently due to problems setting up the expert group.
http://www.jcp.org/en/jsr/detail?id=84
Offline Jeff

JGO Coder




Got any cats?


« Reply #13 - Posted 2004-01-11 02:11:16 »

Quote

Intel x86 processors don't have a fused multiply and add (the IA64/Itanium does). These fused instructions maintain additional precision which is not permitted by non strictfp java. The relaxtion which accompanied strictfp only allows additional exponent range during intermediate calculations. This permits the use of the Intel 80 bit reals for temporaries (provided the FPU is running in the mode which restricts the precision to that of normal double/float as appropriate).

There was a JSR (84) which proposed further relaxation to allow the use of fused multiply/add instructions as found in PowerPC for example. Unfortunately this was withdrawn apparently due to problems setting up the expert group.
http://www.jcp.org/en/jsr/detail?id=84


Okie I'll cop to being confused  by all the various register level processor issues.  Not my area of expertise Smiley

Thanks for the clarification on whats what.





Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Offline renanse

Junior Member




Intelligence is light to a dark world.


« Reply #14 - Posted 2004-01-13 15:16:40 »

If you look at his Benchmark.java source code, you'll notice he simply goes from 1->10million and computes Math.sin, etc. for each increment of 1.  If we clamped the range so it fell between 0 and 2PI, would that not return the same values?  IE, alter his loop to look like:
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
        double clampedI = 0.0;
        while (i < trigMax)
        {
            clampedI = i%(2*Math.PI);
            sine = Math.sin(clampedI);
            cosine = Math.cos(clampedI);
            tangent = Math.tan(clampedI);
            logarithm = Math.log(i);
            squareRoot = Math.sqrt(i);
            i++;
        }


Perhaps I'm off, (was up very late last night)...  But, if the math is accurate, this gives a very nice performance boost.  (from 65s -> 13s on my P4 2.0G machine)

Fire away.  Smiley

Renanse  (ruh-NON-say)
Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Reply #15 - Posted 2004-01-13 19:58:51 »

Yeah, but that's not really relevant, is it? You're changing the program for just java, which is cheating. So you'd have to change it for all languages and measure again on all languages.

Offline Jeff

JGO Coder




Got any cats?


« Reply #16 - Posted 2004-01-14 00:31:29 »

Its a good program optimization..  the kind oif algorythmic optimization that in the real-world often does the most good for your program.

But as a comparison, I agree, you would need to do the same to the C code.

Just points again to the difference between benchmarking and writing real code.

Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Offline renanse

Junior Member




Intelligence is light to a dark world.


« Reply #17 - Posted 2004-01-14 05:41:00 »

Yep, I agree it'd have to be done across the board.  But still, if the optimization brought c++'s trig score from 3.5secs -> 0secs (giving best case) and Java's trig score from 57secs -> 15secs (the speedup I saw), you'd still end up with c++'s total score being ~ 45 and Java's about 60 (or slightly better than, or at least comparable to C#)

Renanse  (ruh-NON-say)
Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Reply #18 - Posted 2004-01-14 06:47:35 »

Quote
(or slightly better than, or at least comparable to C#)


If you wouldn't update the C# benchmark too, maybe, maybe not. But you have to. Again, apples to apples and all... In any way you'd have to measure again instead of just speculate.
As I see it, java's bad trig score is because of java's spec being more demanding than the other language's specs towards trig precision.
Although the benchmark is just a benchmark and not real-life code, it still demonstrates this difference.
You can't change that by changing the benchmark although I agree that if changing the benchmark to more real life code will make the results closer together demonstrates you must never draw too much conclusions from benchmark results.
For real performance, you might have to use look-up tables anyway although a less precise but fast alternative would be of course very welcome.

Offline princec

JGO Kernel


Medals: 343
Projects: 3
Exp: 16 years


Eh? Who? What? ... Me?


« Reply #19 - Posted 2004-01-14 07:49:46 »

The most worrying aspect of the benchmark that's circulating the net at the moment is the C# to Java performance, not the C/C++ comparison. I'd like to know how M$ have gotten some operations so fast compared to Java.

Cas Smiley

Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Reply #20 - Posted 2004-01-14 08:28:41 »

What's so worrying? As I see it, C# performs worse than java, except for trigonometry.

Offline Matzon

JGO Knight


Medals: 19
Projects: 1


I'm gonna wring your pants!


« Reply #21 - Posted 2004-01-14 08:32:40 »

yes, and if C# & java is "the same thing", how come java is slower than C# ?

Offline princec

JGO Kernel


Medals: 343
Projects: 3
Exp: 16 years


Eh? Who? What? ... Me?


« Reply #22 - Posted 2004-01-14 08:37:29 »

Ah, there was another benchmark floating around which I was referring to that shows apples->apples Java and C# along with about 100 other mostly obscure languages. The C# code generation's good.

It's mainly a bummer if you're doing trig of course Smiley And games programmers do tend to use those trig functions rather a lot...

Cas Smiley

Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Reply #23 - Posted 2004-01-14 08:37:38 »

Quote
yes, and if C# & java is "the same thing", how come java is slower than C# ?


You're losing me. Is it?  Roll Eyes
Discounting trig, scores in this benchmark are
java: 46.1
C#: 61.2

Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Reply #24 - Posted 2004-01-14 08:39:40 »

Quote
Ah, there was another benchmark floating around


Googled around but didn't find it. You have a link?

Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Reply #25 - Posted 2004-01-14 08:56:25 »

You mean the windows::developer benchmark?

quote from the summary:
Quote

Java is the worst performer in boxing and memory, is good on exceptions,
but doesn&#8217;t have templates or RAII in any guise; it&#8217;s not
looking too rosy really&#8212;I think C#/.NET has the measure of it, at
least as far as the Windows platform goes.


and
Quote
C# is significantly faster (3&#8211;4 times) at memory
allocation than Java.


which is worrying I suppose. The benchmark mentioned in this thread didn't measure memory allocation.

Offline Matzon

JGO Knight


Medals: 19
Projects: 1


I'm gonna wring your pants!


« Reply #26 - Posted 2004-01-14 10:22:30 »

Quote


You're losing me. Is it?  Roll Eyes
Discounting trig, scores in this benchmark are
java: 46.1
C#: 61.2

I was talking about trig!
C#: 4.1
Java: 57.1

C# is 12 times faster than Java!

Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Reply #27 - Posted 2004-01-14 11:36:10 »

I created a trig lookup table. It's only 4 times as fast as java.lang.Math with float accuracy on the client. The strange this is when I use the server, performance severely degrades: with the table more (almost 4x as slow) as java.lang.math (takes 'only' 30% more time).
Maybe 'microbenchmarking problem' (which the 'osnews' benchmark also suffers from)? Or a server bug? Hard to tell...  :-/

Offline Jeff

JGO Coder




Got any cats?


« Reply #28 - Posted 2004-01-14 22:26:06 »

Quote
You mean the windows::developer benchmark?


To paraphrase one of my favorite Mark Twain quotes:

"There are three kinds of lies.  Lies, damn lies, and benchmarks."

When i talk I tell the people in the room not to trust anyones benchmarks.  Don't trust those with a  vested interest, because they all cheat (so called "benchmarketing").

Don't naively trust third party benchmarks because you don't know how much they really knew about the black art of benchmarking (usually its next to nothing.)

Its just like statistics.  The numbers mean nothing until they are interpreted.  Interpretation is prone to error.  Look at the WHOLE study and use your own brain to decide if the methodology really warrants the conclusions.  And then do your OWN tests with your OWN apps, because they are the final performance measure you really care about.

"50% of the US highschool graduates graduated in the bottom half of their class!" -- "How to lie with statistics"

Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Pages: [1]
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

CogWheelz (18 views)
2014-07-30 21:08:39

Riven (25 views)
2014-07-29 18:09:19

Riven (15 views)
2014-07-29 18:08:52

Dwinin (12 views)
2014-07-29 10:59:34

E.R. Fleming (33 views)
2014-07-29 03:07:13

E.R. Fleming (12 views)
2014-07-29 03:06:25

pw (43 views)
2014-07-24 01:59:36

Riven (43 views)
2014-07-23 21:16:32

Riven (30 views)
2014-07-23 21:07:15

Riven (31 views)
2014-07-23 20:56:16
List of Learning Resources
by SilverTiger
2014-07-31 18:29:50

List of Learning Resources
by SilverTiger
2014-07-31 18:26:06

List of Learning Resources
by SilverTiger
2014-07-31 13:54:12

HotSpot Options
by dleskov
2014-07-08 03:59:08

Java and Game Development Tutorials
by SwordsMiner
2014-06-14 00:58:24

Java and Game Development Tutorials
by SwordsMiner
2014-06-14 00:47:22

How do I start Java Game Development?
by ra4king
2014-05-17 11:13:37

HotSpot Options
by Roquen
2014-05-15 09:59:54
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!