Java-Gaming.org    
Featured games (81)
games approved by the League of Dukes
Games in Showcase (499)
Games in Android Showcase (118)
games submitted by our members
Games in WIP (567)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1] 2
  ignore  |  Print  
  Small micro benchmark for Java vs C++: howto?  (Read 4391 times)
0 Members and 1 Guest are viewing this topic.
Offline Preston

Senior Member


Medals: 4



« Posted 2004-01-12 07:34:35 »

Since (micro-) benchmarking is a science on its own, I thought let's ask here.

Say a person not knowing anything about the two languages asks you to do a comparable benchmark in Java and in C++, because he doesn't trust any external quotes (just what he sees). He'd like to start a C++ exe and a Java Jar/Bat file on the same PCs (several actually) to see the differences.

It should be a small and easy to implementable game related benchmark, including computing operations (needed for simulation) and such like.

Are there any benchmarks sources doing something comparable? If not: how to write some short one which doesn't break any of the many "don't do this ever in micro-benchmarking" rules...?

(Some days ago a new benchmark doing similar things have been quoted here in the forum, comparing 9 languages. It's nice basically. Aside the trigonometry numbers for Java 1.4.3 - when does this problem being solved?)

I know such questions tend to sound very silly. However, just imagine for a second you would have been asked to do this exactly: what would you answer/do ? :-)  Thanks.
Online princec

JGO Kernel


Medals: 386
Projects: 3
Exp: 16 years


Eh? Who? What? ... Me?


« Reply #1 - Posted 2004-01-12 12:56:16 »

Well, microbenchmarking really isn't a science, and the best answer is not to microbenchmark but macrobenchmark and profile a real problem.

Cas Smiley

Offline crystalsquid

Junior Member




... Boing ...


« Reply #2 - Posted 2004-01-12 12:56:28 »

Quote
what would you answer/do ?


Slap them, with a large fish.

If they persist:
Get them to detail what they expect the average frame to have to do.
i.e:
a) 15% complex float maths (physics)
b) 10% traversing bizrre linked data structures
c) 20% Game logic (mainly ifs/some integer flag stuff)
d) 5% sound & effects
e) 50% drawing sprites in software
(or e) - rendering sprites using whatever API you want)

then make some simple test cases that test the critical aspects of each task:

a) do some complex maths: a matrix by matrix multiply, a convert to quaternion, a vector normalise, and a 5 elemnt taylor series expansion.

b) build a linked list of derived classes and call a virtual function on each. Then delete the whole thing.

c) Generate a random number, then run through a series of if statements testing each bit, some statements using &&, some ||, etc. And a 8 case switch statement.

d) Play 8 sound effects, each one starting and stopping. And stream a large sample as well.

e) This one is mainly an uncached memory read/write to array problem, so make a large array (640x480, but 1D), and write to random sections (32 x 32byte lines to simulate a sprite - each line of the sprite would cause a cache miss). The trick is to get around 2x screen overdraw for realism.

Then build a test case that runs each test (while sound is going on, as sound is more of a %CPU usage as opposed to time taken).

Time for AT LEAST 1 MINUTE.

Write up the results, then slap the person some more for wasting your time when they end up doing something completely different to what you tested anyway (which they will do).

- Dom

PS: If they are serious about writing the games to make money, you must consider that you will have to use Java 1.1 or you will lose 75% of your visitors. If using C++, use OpenGL for graphics, as this is much nicer to port.
PPS: Unless you really want the run-in-a-web browser (which I do for our demo versions) use C++. Being able to use pointers, Intel compilers, BoundsChecker & Debuggers will save you much pain for a large game project. Dealing with hardware wierdness is nothing compared to dealing with Java VM version wierdness. Plus, a real publisher (CD's in boxes type) will not take a Java version seriously. Sad but true.
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline blahblahblahh

JGO Coder


Medals: 1


http://t-machine.org


« Reply #3 - Posted 2004-01-12 13:25:21 »

Quote


Slap them, with a large fish.

If they persist:
Get them to detail what they expect the average frame to have to do.


(snip excellent advice: get a requirements list off them, and then implement it).

I suggest you also implement if all within Unit (C/C++) and JUnit (Java). JUnit is a bit pants these days (ONLY because of the lack of tools that a "real" testing suite would come with as standard), but... there are some good "add-ons" for JUnit which do a lot of this.

e.g. I recently spotted an add-on which effortlessly times all your tests, presenting results individually as well as in aggregate.

e.g. I have seen (although I can't find it any more Sad ) a nice simple visualization tool that automatically finds and runs all your JUnit tests, and presents a summary in the format of a typical "report". But you could pretty easily write a small app to auto-translate JUnit output to an HTML file

Couple of advantages to doing it with (J)Unit:

  • It doesn't mandate much, but at least provides some nice consistency and formalism to your tests
  • the Unit stuff is so widely used in commercial programming that any new staff have a very high chance of already having used it (and I'd hope that the people you're trying to convince have used it at some time before in their lives; should make it slightly easier for them to read, trust, and fiddle with your source to satisfy themselves)
  • Gives your team an example of Unit tests before you even start the project (it's easier to get people to write them if they've already written / used some)
  • It's a good start - you could encourage people to add extra tests (because Unit makes adding tests very simple; it picks up new tests automatically without having to be reconfigured/recompiled each time) for any new things they are suspicious about


Quote

Time for AT LEAST 1 MINUTE.


I'd say at least one hour.

Quote

Write up the results, then slap the person some more for wasting your time when they end up doing something completely different to what you tested anyway (which they will do).


Export your results (or save them directly) as CSV. Then everyone can open them in Excel. This is handy, because Excel can instantly and effortlessly produce all manner of graphs, statistical analysis, and it's damn easy to sort stuff Smiley. I hate Excel, but basic stats analysis takes < 30 seconds in excel (plus loading time Sad ), so I confess to using it quite often Smiley.

Quote

PS: If they are serious about writing the games to make money, you must consider that you will have to use Java 1.1 or you will lose 75% of your visitors. If using C++, use OpenGL for graphics, as this is much nicer to port.


Ahh, I'm one of the disagree-ers here...I say "mandate 1.4 and be damned".

Bear in mind, crystalsquid, he's talking about a 2-year dev time. In that period, the installed base of 1.4+ is going to sky-rocket, largely because of the pre-installs on all Dell and HP machines (IIRC; yes?). Simultaneously, there will be more upgrades/replacements of old versions simply because 1.1 is now removed (IIRC ?) from windows, forcing more people to download a sun version.

Also, and this may be a very small thing, java is now so old people are starting to trust it more Smiley. My own brother, who refused to install java since his last re-format of windows (oh, the joy!) because an old JVM (1.2.x JDK) added crap to his computer and didn't install properly, has finally acceded; like Flash and Shockwave, the longer Java is around, the longer the world goes without "Java virus destroys world" stories, the less suspicious joe bloggs home user becomes.

But I liked your list of things to test, it's a very handy starting point.

malloc will be first against the wall when the revolution comes...
Offline blahblahblahh

JGO Coder


Medals: 1


http://t-machine.org


« Reply #4 - Posted 2004-01-12 13:29:08 »

Quote
Since (micro-) benchmarking is a science on its own, I thought let's ask here.
...
Are there any benchmarks sources doing something comparable? If not: how to write some short one which doesn't break any of the many "don't do this ever in micro-benchmarking" rules...?


Also, if you do go the Unit route, if you are willing to put the code somewhere public (perhaps on JGO?) I would contribute some mini benchmarks too; there've been a couple of times in the last 12 months when I wanted something similar (although usually to test 3rd party libraries in conjunction with games dev, rather than test java vs. something else) but knew I'd never have the time / effort to write anything comprehensive enough to be worth it.

If you got something started, so I could just come along and knock up e.g. some simple NIO tests...

malloc will be first against the wall when the revolution comes...
Offline Preston

Senior Member


Medals: 4



« Reply #5 - Posted 2004-01-12 14:23:39 »

Many thanks for the good input, guys. I'll have to re-read your articles a few times and think about them.

Well, I think I'll start with the ... fish. ;-)

JUnit - never used but will start reading an idiot's guide to JUnit.

Yes, indeed, I forgot to mention the project would be using 3d, so some 3d API will be there and no software sprites this time, luckily.
Blabla is right: it's for a larger game, too, so no Applet, plus Java 1.5 should well be ready then. Java 1.7 maybe? ;-)  Size doesn't matter: it will be on CD (in case the publisher will let us - I think I'll take a copy of Chrome game with us, plus a big fish).

Quote
PPS: Unless you really want the run-in-a-web browser (which I do for our demo versions) use C++. Being able to use pointers, Intel compilers, BoundsChecker & Debuggers will save you much pain for a large game project. Dealing with hardware wierdness is nothing compared to dealing with Java VM version wierdness

1) C++ sucks ;-) , 2) pointer sucks ;-) , 3) Intel's compiler is cool but too many Win32 games can't use it because they're contaminated with M$ libs/etc which won't compile on Intel's compiler, sigh (ISO C++ standard? Not for Win32), 4) hardware and software (Windoze) weirdness is such a pain, not again thanks, and since it would be using one JVM version: nice. 4) Distributed compiling is a nice idea but not handy at all.
Etc blabla. Anyway thanks, Crystalsquid, for the nice benchmark framework ideas!

So, a lot of work. Maybe for nothing. Well, no risk no fun. Still Java is really nice.
Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Reply #6 - Posted 2004-01-12 15:16:50 »

Quote
Unless you really want the run-in-a-web browser (which I do for our demo versions) use C++. Being able to use pointers, Intel compilers, BoundsChecker & Debuggers will save you much pain for a large game project. Dealing with hardware wierdness is nothing compared to dealing with Java VM version wierdness.

I didn't come across Java VM version weirdness that much. And God knows how much I cursed those C/C++ typical hard to track down bugs.
Preston's argument is true; if you ship on CD, you will ship with 1 JVM. You aren't trying to compile your C++ code with 20 compilers, are you? (shaky anology, I know  Wink)
For me, java is a way to move forward in games development, even if it's not all perfect for it yet. Like java's progressing the last couple of years, it's getting there.

Quote
PS: If they are serious about writing the games to make money, you must consider that you will have to use Java 1.1 or you will lose 75% of your visitors.

I'm a disagree-er too. Ban 1.1! 1.4 at least.

Quote

Plus, a real publisher (CD's in boxes type) will not take a Java version seriously. Sad but true.

If you'd say to them "hey, look at my cool java game" you're probably right. I'd say, show them your stuff and tell em it was java when you sold your first million Grin

Erik

Offline crystalsquid

Junior Member




... Boing ...


« Reply #7 - Posted 2004-01-12 15:47:54 »

Quote
I didn't come across Java VM version weirdness that much


Oh - I have Tongue Using BufferStrategy causing it not to work on Macs, playing samples using 1.1 interface runs REAL slow on 1.4, etc. etc. mind you, we also had a 'destructor in base class not virtual' causing a bizarre memory leak just yesterday in our C++, so not much between them really I suppose.

Quote
I'm a disagree-er too. Ban 1.1! 1.4 at least.


It would be nice, but from putting our games on the web we found the 75% of people did not have 1.4, and would not upgrade (not surprising given the 15MB or whatever it is at the moment). Having said that - if there are lots of CD based games that people install which ends up with them installing a newer Java version, then this can only be a good thing. I know many people only upgrade their DirectX when they buy a game that installs it for them.

Quote
If you'd say to them "hey, look at my cool java game" you're probably right. I'd say, show them your stuff and tell em it was java when you sold your first million  


yeah, I disagreed with myself in the thread in the business forum. I used to advise my old company about some purchases (it was a very big publisher/developer) and to be honest, we didn't look to close at what technology they were using if they had a decent demo of the game running. If it meets the golden rule - put CD in, install, press 1 button to play, then they don't really care. If the install process is too complicated you will have issues though.

I am a die-hard C++ fan (well, assembler really), and my main issues with Java do stem from having to use 1.1 at the moment to ensure enough people can play our demos. (Grr!)

- Dom

(Bring on the inline bytecode syntax!)
Offline Preston

Senior Member


Medals: 4



« Reply #8 - Posted 2004-01-12 16:30:15 »

Quote
Well, microbenchmarking really isn't a science, and the best answer is not to microbenchmark but macrobenchmark and profile a real problem.

Well, yes, that's right. :-)
If I understand Jeff correctly, then maybe I should have said: "Understanding what you're microbenchmarking is a science" or such.
Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Reply #9 - Posted 2004-01-12 21:06:37 »

Quote
Oh - I have  Using BufferStrategy causing it not to work on Macs, playing samples using 1.1 interface runs REAL slow on 1.4, etc. etc. mind you, we also had a 'destructor in base class not virtual' causing a bizarre memory leak just yesterday in our C++, so not much between them really I suppose.


Of course there are problems between java versions. To be fair, 1.4 on mac is fairly young; fullscreen on windows didn't work flawlessly in the 1st versions. The 1.1 sound problem was a bug which is probably solved  in the meantime (haven't checked though). But sure, there are problems and sometimes really frustrating ones.
You have strange version issues between C compilers as well. I've seen some really weird workarounds to optimizer bugs for example.
The thing is, with java when you arrived at the point you come across problems on macs and linux versions, you might not even be halfway the windows version alone using C++.
Might be because I'm a pretty shitty C programmer these days  Grin but I can program a hell of a lot faster in java with fewer bugs.

Quote
It would be nice, but from putting our games on the web we found the 75% of people did not have 1.4, and would not upgrade (not surprising given the 15MB or whatever it is at the moment).


I agree in such situations you have to just do the most practical thing. But besides that, I think it's really worth it on the long run to push new java versions anyway. I've said it before and I'll say it again: Make your games the large JRE download worthwile. (Not that I'm saying your stuff isn't! Just talking in general)
Sticking to 1.1 will surely kill java for games eventually. It's all up to us developers.

Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Reply #10 - Posted 2004-01-12 21:10:07 »

Quote

(Bring on the inline bytecode syntax!)


Could be a good idea. Might help the endless masking and such in my CPU emulators. Make an enhancement request  Smiley

Offline swpalmer

JGO Coder


Exp: 12 years


Where's the Kaboom?


« Reply #11 - Posted 2004-01-12 21:24:26 »

No - don't add byte code stuff to Java.. use one of the many other things that will produce Java bytecode.  Don't pollute the Java language... use an appropriate language.
There are things like BCEL  (Byte Code Engineering Library ?) and other tools that you can use for these niche applications.

Offline Raghar

Junior Member




Ue ni taete 'ru hitomi ni kono mi wa dou utsuru


« Reply #12 - Posted 2004-01-12 23:39:30 »

I know I didn't see a DirectX 3.0 game for ages. So what the idea of outdated beta of Java?
If you make users lazy and expecting you'd do job for them, you'd be sorry for it quckly.
I think that best idea is to say: "You should download last version of JVM form here, or here. Current best version is xxxx" And add binaries compiled to one version  backwards to alow user one year of time for updating.
It's just about forcing users to do "uncommon things", like regular update of theirs software. (Like antivirus, firewall, and so on) Actually number of e-mail viruses each mouth shows that "JOE" average isn't aware about
his necessity of such things.

Java 15-44MB/year. AV 3MB/month. FW 5MB/ 3months.
Average DL of Dial up user at least 350-600 MB/month.
Conclusion If someone don't have updated JRE it's
1. Problem with lazy user.
2. User that didn't know what he should do.
3. User that is scared with doing new things.
solutions:
1. Force him as mercilessly as creator of DirectX. You might have some problems, but it's better in the long run.
2. Web pages for dummies is often best solution, just try to write it simple and provide user with step by step instructions. Don't forget to add a warning. "you should read all text on this page, don't forget to scroll down". I know it's silly, but...
3. Words "computers are changing and you need to upgrade to get new (faster) features", or "some programs are demanding to upgrade your OS, we just need for you to upgrade your (Sun, or IBM) Java VM. "
(I actually thing that 3. problem comes with age. Users are used to old DOS, NIX or whatever and whe they should be doing something they they didn't did before they try to move it to sometime later. Aka scream from linux boards: "DO ME A FAVOR AND UPDATE CLIB(don't recall proper name). It will solve all of your problem without any stupid hacks. You know it's freely avilable here right?")

It's just about thinking of reasonable, or completely unreasonable, but nicely sounding excuse.  (More stable programs, and less necessary restarts excuse works well.)

As for microbenchmarking. If you know some methods from your game you could use them with real word data. However, it will be a big problem to rewrite them in C++ and ASM with high enough precission.

Why is Jeff so Huh?, when someone would start to talk about benchmarks?


Offline Jeff

JGO Coder




Got any cats?


« Reply #13 - Posted 2004-01-13 00:33:10 »

Gee I agree with CS  on everything except the PS which I have strong disagreements with...


Quote

PS: If they are serious about writing the games to make money, you must consider that you will have to use Java 1.1 or you will lose 75% of your visitors.


I question the scientific accuracy of the 75% figure.  More generally this is only an issue at all if you are targeting the web Applet market as it exists right now.  If you are targeting either Java Web Start or Java Applications it a  non-issue.  I

f you are targeting Applets a year or two from now the situation is likely to be radically different as MSFT is due to STOP shipping 1.1 soon and we already have comittments from the PC vendors to ship real VMs with their systems that shoudl cover 50%+ of all new PCs.

(About the only reason to target Applets IMO is to reach the causal market today, for all other markets the other approaches are much more flexible.)

Quote

If using C++, use OpenGL for graphics, as this is much nicer to port.


As Bioware discovered, just usiong OpenGL however doesnt gaurnatee yo ua fast or easy port of C/C++ code.

Quote

PPS: Unless you really want the run-in-a-web browser (which I do for our demo versions) use C++. Being able to use pointers, Intel compilers, BoundsChecker & Debuggers will save you much pain for a large game project.


Absolutely 100% disagree.  A competant Java coder puts out lkess bug-ridden code in anywhere between 2 and times the speed of C/C++ coding.  I really need to scan that sheet of statistics from CGDC two years ago and post it.  All the Java developers reported:

90%+ of their time was spent in development, less then 10% in Q&A.

Porting times to Mac, Linux and Solaris that ranged from 3 hours to 2 weeks for pure Java, and less then a month (PC to Mac) for Java ontop of custom graphics and Io bindings.

Over-all development times unheard of  in the game industry.  (ex:  Shawn's entire FullSail 3D game engine took less then 2 man years.  Based on that engine he brought an initial FPS to demo playability in a month.  He did a complete playable fighting game in less then 3 months, as I recall.)

Quote

Dealing with hardware wierdness is nothing compared to dealing with Java VM version wierdness. Plus, a real publisher (CD's in boxes type) will not take a Java version seriously. Sad but true.


Again untrue.  Thre have been about a dozen preofessionally, in a box, published games partly or fully in Java  in recent years.   Roboforge (99.9% Java, 3 lines of C code) made it to a box.  Star Wars Galaxies used Java for its scripting, as did VTM (both published y major publishers in a  box.)  Law and Order (100% Java) was not only published bya amjor publisher, but so successful that the publisher published a second game on the same Java engine.

Will you have to answer a few questions about "why Java?" probably. But it HAS been done successfully already.


Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Offline Jeff

JGO Coder




Got any cats?


« Reply #14 - Posted 2004-01-13 00:45:32 »

Quote
Why is Jeff so Huh?, when someone would start to talk about benchmarks?




NOT benchamrks. Benchmarks are vital to performance coding.

MICRObenchmarks.  Microbenchmarks are often misleading and more often then not useless.   The problem is that microbenchmarks don't act like real code so the question they end up answering 99% of the time is NOT "how will real code in this environment perform" but only "how does THIS BENCHMARK in this environment perform."

Modern execution environments such as Hotspot are very complex and are designed to analyze what your progam is doing and attack it in the best manner for that problem.   Sicne the MB is a different problem, you get different results.

To write a good benchmark you need to really need to understand a number of things:

(1) What is your target app really going to be doing, in total?
(2) How will this environment optimize that work.
(3) How does this benchmark behave>
(4) In what ways are the optimizations that will be done for this benchmark similar or dissimilar to what will happen with native code.

A good simple example is that microbenchmarks are commonly written to dive into a single loop and sit ther for the whole run.  Ths is VERY different from an app which will be calling other functions and different loops all over the place.

As it happens, those two conditions are treated differently by Hotspot because one (the MB) calls for on-stack code replacement while apps geneally don't.  Just recently here on JGO we stumbled across a bug in Hotspot's on-stack replacement that led the bunch of us here  on a  wild goose chase for about a week-- all based on a micro-benchmark.

I was the first full-time hire on Sun's JDK performance tunign team and worked heavily on the tuning of JDK1.3.  I've got a ton of war-stories in this area.  Remember how slow Java2D was in 1.2?  That was the DIRECT result of micro-benchmarking leading the 2D team to tunign the WRONg things.  Come to one of my periodic talks at conferences on performance tunign Java and you will get to hear the whole story.

The number of microbenchmarks I've seen at this point probably number in the hundreds.  The number I've seen that actually produced information useful to an app developer can be counted on two hands.

(Baring getting to a talk, or even if you do, I'd also recommend Steve and my book for an easy relatively short read that will likely open your eyes in a lot of ways: http:java.sun.com/docs/books/performance )

To sum it up, the prooblem with most (micro)benchmarks is that the authors have no idea what it is they are really measuring.

To paraphrase the NRA

Benchmarks don't fool programmers.
Programmers fool themselves by mis-using benchmarks Smiley

Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Offline Jeff

JGO Coder




Got any cats?


« Reply #15 - Posted 2004-01-13 00:58:12 »

So to answer the original question.

A microbenchmark is about 90% likely to provide meaningless results.

Whether those results are unreasonably good (such as Java optimizing out the entire benchmark, which has happened mor then once) or unreasonably bad (such as the cost-of-interfaces microbenchmark recently that ended up tripping over an optimzier bug that did not remove all redundant null checks in casting in the on-stack replacement case) is anyone's guess.

If you want small benchmarks, then find chunks of reasonably seperable pieces of functionality of your app that you are sure will be significant and can implement and test indpendantly.  Be careful of getting too small or it can again become fairly meaningless (ex: poly fill rate which at this point totally depends on the graphics card and not at all on the processor.)

Some good examples of things to benchmark might be  pathfinding algorithyms, fft, etc.

Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Reply #16 - Posted 2004-01-14 12:13:09 »

FWIW, I think some good 'macro benchmarks' would be helpful.
I mean, one micro benchmark is posted on the internet after the other. Here everybody screams either "evil microbenchmark" or "see? java is really fast!" (depending on the result  Wink)

Online princec

JGO Kernel


Medals: 386
Projects: 3
Exp: 16 years


Eh? Who? What? ... Me?


« Reply #17 - Posted 2004-01-14 14:21:48 »

Actually that microbenchmark is a rather good one because it rather highlights a glaring performance deficiency in one small area. Individually, little deficiencies like this don't make a big difference to the big picture but if you just happen to write the application that strings a few of them together (like, say, you're writing a 3D game...) it all adds up to a crappy framerate. So perhaps someone should post a "bug" on the trig score?

Cas Smiley

Offline Mark Thornton

Senior Member





« Reply #18 - Posted 2004-01-14 16:41:01 »

Quote
So perhaps someone should post a "bug" on the trig score?

The current trig score is the result of fixing a bug (I posted the link earlier). To get faster trig you have to relax the specification or create a new FastMath class with a relaxed spec (anything goes?). I suspect that this would be resisted because many would use it even when they don't actually have a performance problem at all. Alternatively ask Intel to improve the accuracy of their trig methods to meet the Java requirements.
Online princec

JGO Kernel


Medals: 386
Projects: 3
Exp: 16 years


Eh? Who? What? ... Me?


« Reply #19 - Posted 2004-01-14 20:06:45 »

That's a head in the sand if ever I saw one. We don't much care about VMs that don't actually accurately reflect the common hardware out there today. If Intel is the lowest common denominator then it should be specified to match Intel's standards. No-one cares about a standard which doesn't work well on 95% of the world's hardware.

Cas Smiley

Offline Jeff

JGO Coder




Got any cats?


« Reply #20 - Posted 2004-01-14 21:14:32 »

"Work well" is a value judgement, cas.

Java trig works VERY well for the original goals of java-- that the same code produce exactly the same out put on any system.  This is very important to those apps that want consistency and accuracy over speed (eg the scientific apps that Java math was originally targeted at.)

What you want, a fast and sloppy trig, wasn't the design spec.

There are some ways being talked about now inside of Sun about how to reduce the JNI over-head.  No promises yet but if that happens then it'll be reasonable for you to call a C function to do your fast trig...

Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Offline Jeff

JGO Coder




Got any cats?


« Reply #21 - Posted 2004-01-14 21:19:41 »

Quote
Actually that microbenchmark is a rather good one because it rather highlights a glaring performance deficiency in one small area.


Sure, which is why I say microbenchmarks are useful IF you understand what you are measuring and how it will relate to your actual app.  They can also teach you things about whats going on in the VM that can be very useful IF you have the knowledge base to properly interpret them. (Which generally means familiarity with the intricacies of the VM in question and logic behind its design.)

But they rarely reflect real code so interpreted naively, which is what is almost always done, they tend to mean less thyen nothing and lead peopelo to improper conclusions.

Keep in mind one other factor, unless you are doing an incredibly simple program I **gaurantee** you you DON'T know what your bottlenecks will really be.  Real code execution is far too complex for our puny minds to statically analyze effectively.  So there's no real good way to predict the relationship of that MB to your over-all code performance.  This is why Profilers were invented.

Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Online princec

JGO Kernel


Medals: 386
Projects: 3
Exp: 16 years


Eh? Who? What? ... Me?


« Reply #22 - Posted 2004-01-15 07:31:03 »

It's all very well making a spec that mandates accurate trig but if 95% of the worlds processors do slightly less accurate trig that's good enough for everyone else where's the sense in it??? The Java spec ought to be amended to lower the accuracy to the Intel FPU.

...but hang on, isn't this what all that strictfp stuff was about?

Cas Smiley

Offline Mark Thornton

Senior Member





« Reply #23 - Posted 2004-01-15 08:10:13 »

Quote
...but hang on, isn't this what all that strictfp stuff was about?

Under strictfp calculations must at all times use only the precision and range of double (or float). Relaxed code is permitted to use the extended exponent range that intel processors can provide (but NOT the extended precision) using the 80 bit real form.

A related change was the addition of the StrictMath class which is currently defined to produce results identical to a specified math library (fdlibm). The spec for the Math class was then relaxed very slightly: "a larger error bound of 1 or 2 ulps is allowed for certain methods". Unfortunately the trig operations on Intel processors substantially exceed this error limit for some input values (especially values near Math.PI).

I'm not in favour of further relaxing the spec for the Math class, but perhaps another class could be added with a deliberately insulting name like BadMath which offered no accuracy guarantees but did offer the prospect of speed. The mischievous name might discourage inappropriate use of the class.
Offline Orangy Tang

JGO Kernel


Medals: 56
Projects: 11


Monkey for a head


« Reply #24 - Posted 2004-01-15 08:37:47 »

BadMath.PI == 3? Shocked

[ TriangularPixels.com - Play Growth Spurt, Rescue Squad and Snowman Village ] [ Rebirth - game resource library ]
Offline blahblahblahh

JGO Coder


Medals: 1


http://t-machine.org


« Reply #25 - Posted 2004-01-15 09:25:45 »

Quote
BadMath.PI == 3? Shocked


Spoken like a *true* games developer...
Grin

malloc will be first against the wall when the revolution comes...
Offline Jeff

JGO Coder




Got any cats?


« Reply #26 - Posted 2004-01-15 20:16:55 »

Quote
I'm not in favour of further relaxing the spec for the Math class, but perhaps another class could be added with a deliberately insulting name like BadMath which offered no accuracy guarantees but did offer the prospect of speed. The mischievous name might discourage inappropriate use of the class.


This is a particular issue on some hardware platforms.  I can't name the one I'm thinking of because Im under NDA but it has a TON of floating point processors NONE of which meet Java spec Sad
The end result is that a Java app on that platform would be unable to use most of the platform's power unless they down coded all their math to C Sad Sad Sad

Its possible that silicon is getting so dense and so cheap that this wont be a problem in the next generation, but I'm not willing to bet on gate-counting in consumer devices stopping any time soon.

So yes i think a "BadMath" or perhaps more appropriately named "PlatformMath" would be a good addition for some environments.
it needs to be more then just a class though, it has to recognized and supported in the code generator.  You really don't want math primitives treated as method calls, even inlined ones, unless your inliner is so good that it literally makes no difference.


Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Offline Luke

Senior Newbie




I love YaBB 1G - SP1!


« Reply #27 - Posted 2004-01-15 22:03:36 »

Wouldn't a BadMath VM Option make more sense than a BadMath class since you're not going to know whether you need it or not until after you've written the code?
Offline Jeff

JGO Coder




Got any cats?


« Reply #28 - Posted 2004-01-15 22:31:53 »

No, I'd argue against that.  That would violate a primary promise of the VM, that the same code produce the same output for portability purposes.

We're already bending that by saying "this code doesnt HAVE to produce the same output on all platforms" but that should be a documented limit of the code, ergo it should be in the code itself.


Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Offline Jeff

JGO Coder




Got any cats?


« Reply #29 - Posted 2004-01-15 22:35:37 »

Someone asked for "Macro benchmarks".  That sort of depends on your goals.

For your code, the macro benchmark IS your code, or major units thereof.

For libraries, the macro benchmark is typical consumers and varies from library to library.

For instance, to get Java2D tuned to run GUI apps well we took a  bunch of then current Java applications and metered them.

We also knew that we had to support a whole lot of swing ontop of us so we wrote a little harness that brought up a swing gui and "bashed" on it with events.  It wasn't really like a user, but it was more like a user then a microbenchmark that draws 1000 blue lines in a row, which is what the first suite had looked like.


Got a question about Java and game programming?  Just new to the Java Game Development Community?  Try my FAQ.  Its likely you'll learn something!

http://wiki.java.net/bin/view/Games/JeffFAQ
Pages: [1] 2
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

Pippogeek (37 views)
2014-09-24 16:13:29

Pippogeek (29 views)
2014-09-24 16:12:22

Pippogeek (18 views)
2014-09-24 16:12:06

Grunnt (42 views)
2014-09-23 14:38:19

radar3301 (24 views)
2014-09-21 23:33:17

BurntPizza (61 views)
2014-09-21 02:42:18

BurntPizza (30 views)
2014-09-21 01:30:30

moogie (36 views)
2014-09-21 00:26:15

UprightPath (49 views)
2014-09-20 20:14:06

BurntPizza (52 views)
2014-09-19 03:14:18
List of Learning Resources
by Longor1996
2014-08-16 10:40:00

List of Learning Resources
by SilverTiger
2014-08-05 19:33:27

Resources for WIP games
by CogWheelz
2014-08-01 16:20:17

Resources for WIP games
by CogWheelz
2014-08-01 16:19:50

List of Learning Resources
by SilverTiger
2014-07-31 16:29:50

List of Learning Resources
by SilverTiger
2014-07-31 16:26:06

List of Learning Resources
by SilverTiger
2014-07-31 11:54:12

HotSpot Options
by dleskov
2014-07-08 01:59:08
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!