Java-Gaming.org    
Featured games (81)
games approved by the League of Dukes
Games in Showcase (494)
Games in Android Showcase (114)
games submitted by our members
Games in WIP (563)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1] 2
  ignore  |  Print  
  Java Emulator Framework  (Read 8132 times)
0 Members and 1 Guest are viewing this topic.
Offline xbrain

Senior Newbie




Java games are the Future


« Posted 2002-11-04 10:15:08 »

As Java is a great standard, as it offers a lot of nice patterns, as it is a sooo fast runtime language, I wonder why there is no Emulator Design Framework attempt?

I know there is a lot of Emulators project in Java. But there is no project to try to federate them such as has been done with MAME. So, (just to start) I propose:
  - A standard CPU pattern: all CPU emulations attempt have some basic standard functionalities such as cycles management, IRQ management, etc...
  - A set of assembly instructions
  - some CPU input/output: CPU ports management, etc...
  - some very specific hardware management (i.e. custom manufacturer chips)

There could be too a game driver pattern such as in MAME.

hope will interest someone ... Smiley

Note: Sorry for my poor english. Frenchie little guy Wink

XBrain - Don't let others control your Mind...
Offline leknor

Junior Member




ROCK!!!


« Reply #1 - Posted 2002-11-04 13:08:29 »

CottAGE is what you are describing. I haven't looked at it in a while but when I did I thought it was crunchy at best as a framework.
Offline pepe

Junior Member




Nothing unreal exists


« Reply #2 - Posted 2002-11-04 13:41:54 »

Fun, Cottage's emulation layer is called 'jef', for 'Java Emulation Framework', the topic of this forum.
Might be what you look for.  Smiley

( en plus, ils cherchent des programmeurs pour le projet... alors... )

Home page: http://frederic.barachant.com
------------------------------------------------------
GoSub: java2D gamechmark http://frederic.barachant.com/GoSub/GoSub.jnlp
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Reply #3 - Posted 2002-11-08 10:16:06 »

True, JEF needs work to be a real good framework (and a lot of cleaning) but the basic building blocks are there to build your own emulator.
Although I didn't have much time recently to work on it, I'm still in the process of trying to refactor and redesign things.
CottAGE uses JEF to make an arcade emulator based on MAME's driver architecture.
We are looking for contributers for the project to improve it and if you have any questions about how it works, don't hesitate to email me.

Erik Smiley

Offline pepe

Junior Member




Nothing unreal exists


« Reply #4 - Posted 2002-11-27 07:01:24 »

erikd:
what if you were using BCEL  http://jakarta.apache.org/bcel/index.html (or other similar tool) to regenerate a replicate of the rom using only list of calls for each instructions. That way, you'll have a bytecode equivalent of the rom, and wil not need anymore runtime parsing. Moreover, it will be the jvm's job to inline and optimise what can be.
Would be curious to know how fast that can be after such a treatment.

Home page: http://frederic.barachant.com
------------------------------------------------------
GoSub: java2D gamechmark http://frederic.barachant.com/GoSub/GoSub.jnlp
Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Reply #5 - Posted 2002-11-27 07:49:17 »

Quote
what if you were using BCEL  http://jakarta.apache.org/bcel/index.html (or other similar tool) to regenerate a replicate of the rom using only list of calls for each instructions.


Yes, someone else suggested something like that too. It's a fascinating idea, however I think it's close to impossible to do a static recompilation of a rom to bytecodes. How would you distinguish code and data, or handle generated or self-modifying code?

Maybe it is possible to have a dynamic approach, sort of like a "JIT-CPU-native-opcode-to-Java-byte-code-translator"  Cool
I don't have a plan on how I could implement it, though. It would require some sort of mixing between the current interpreting and translating to bytecodes so there would always be some parsing left.

Erik

Offline xbrain

Senior Newbie




Java games are the Future


« Reply #6 - Posted 2002-11-27 07:53:22 »

Seems interesting. But I'm not sure the "fastest way" is the "better way". I think what is important in this project is not to get the fastest emulation code, but a fast development code.

To resume, what is important is not to be the fastest, but to be fast enough... Smiley

XBrain - Don't let others control your Mind...
Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Reply #7 - Posted 2002-11-27 08:03:21 »

Quote
I think what is important in this project is not to get the fastest emulation code, but a fast development code.


You're right, although you can happily write an emulator using JEF (well, the new upcoming version of it anyway Wink) without looking at the CPU code at all so the fastest code doesn't have to stand in the way of fast development.

One a side note, in CottAGE's Z80 based emulators, the main bottleneck is not the CPU emulation but the video emulation.
OTOH, someone is developing an 68000 emulator in for JEF and that will most probably be slow if it's an interpreting core so that might benefit.

Erik

Offline pepe

Junior Member




Nothing unreal exists


« Reply #8 - Posted 2002-11-27 09:34:54 »

Quote
How would you distinguish code and data,

by analysing the instruction path. follow all the paths that the code can use, and whas is left is data. maybe simplistic, but i don't see why it should not work. Maybe i miss something else...
Quote
or handle generated or self-modifying code?

Errrr. did no think that arcade games used such advanced methods. Yes, that would complicate the whole thing.

Home page: http://frederic.barachant.com
------------------------------------------------------
GoSub: java2D gamechmark http://frederic.barachant.com/GoSub/GoSub.jnlp
Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Reply #9 - Posted 2002-11-27 10:06:44 »

Quote
Quote:How would you distinguish code and data,  

by analysing the instruction path. follow all the paths that the code can use, and whas is left is data. maybe simplistic, but i don't see why it should not work. Maybe i miss something else...


You can do that to a certain degree, but still it would be very difficult.
For example jumps are not always static.
Sometimes you have an opcode like JP (HL), which jumps to the address stored in register HL. Which values can HL get? Hard to tell (and be sure too). Sure it may be possible to write some analyzer that might come up with the correct answer (and the complete instruction path), but that would be some piece of code!
Many programs use this opcode, pacman being a popular example Smiley

But maybe it's possible to analyze and translate as much as possible up front, regard the rest as 'data' and 'untranslated code' that will be interpreted or even JIT-ed.

Quote

Quote: or handle generated or self-modifying code?  

Errrr. did no think that arcade games used such advanced methods. Yes, that would complicate the whole thing.


I haven't seen self-modifying code in arcades too, but JEF is not meant to be just for arcade emulation. What I have seen is little pieces of code inserted at an interrupt hook at run-time.

Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline swpalmer

JGO Coder




Where's the Kaboom?


« Reply #10 - Posted 2002-11-30 00:48:32 »

Self modifying code was a popular technique in the 80's  You could often use it to optimize loops so that you didn't have to check conditions inside the loop.   Arcade machines running code from ROMs obviously woul have a harder time.. but there could still be jump tables ect that are copied to RAM.  Games for PC's like the Atari 400, 800, ST, or the Commodore 64, 128, Amiga would be more likely to use self-modifying code I think.

I think it became less popular when processors became mroe advanced and things like seperate data and instruction caches made it very expensive to use self-modifying algorithms.

If it weren't for that, I would figure the best emulation technique would be as suggested... not be to emulate the processor core, but to treat the machine instructions as JIT compilers treat byte code... as instructions for a virtual machine that you will compile to "native" code.

You probably can detect what is code or data at runtime if you can exercise all paths of the program..  if ever the path of execution leads the instruction pointer to address X then address X must have an instruction or the code would crash.   It would be no easy task though to track 'data' that is written to address X in a way that you could  then manage to compile it as well.

Maybe the  thing to do is to assume that there is no self-modifying code and see how far that gets you... then work on patching the 'special cases' where self-modifying code is used.

Offline Seb

Senior Newbie




import com.acme.Bomb;


« Reply #11 - Posted 2002-11-30 19:35:33 »

I'm just curious, remember jump tables? Used to be useful when doing assembly or even C.

In the case of an emulator, if the CPU being emulated had one byte opcodes, then it's possible to build a jump table for them, having a very simple main loop.

Maybe this is the way things usually work, say, in MAME. I've never looked at emulation code.

I wonder how well a jump table would work in Java? Taking into account the way the VM optimizes code.

Of course, you would have to have a class table in this case of classes implementing something like CpuOpcode

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
class CPU {
  CPUOpcode[] opcodes;
  byte[] memory;
  int instructionPointer;

  public void run() {
    do {
      ( opcodes[ memory[ instructionPointer ] ] ).execute( this );
    } while (true);
  }


Of course, it's a tad more complicated than this since there usually are some two byte opcodes and stuff. But the basic idea is that.

It would probably be hard for the VM to optimize some of this I'm sure when compiling to native, perhaps it isn't even a good idea for native emulators. Any comments?

Seb

Offline Seb

Senior Newbie




import com.acme.Bomb;


« Reply #12 - Posted 2002-11-30 19:53:28 »

Just to clarify what I said above a little more.

The idea is eliminate the switch statements in the main loop, which I think are much slower for interpreting, and are way too popular in main message loops for my taste.

Seb

Offline swpalmer

JGO Coder




Where's the Kaboom?


« Reply #13 - Posted 2002-12-01 21:33:10 »

A good compiler will use jump tables for switch statements when appropriate.   A switch does not necessarily compile to the equivalent of
if..else-if...else-if...

Offline Seb

Senior Newbie




import com.acme.Bomb;


« Reply #14 - Posted 2002-12-02 00:45:21 »

Quote
A good compiler will use jump tables for switch statements when appropriate.   A switch does not necessarily compile to the equivalent of  
if..else-if...else-if...


True, but you have no control over it.

Seb


Offline swpalmer

JGO Coder




Where's the Kaboom?


« Reply #15 - Posted 2002-12-02 01:31:57 »

My point is - let the compiler do what it can, if your program runs fast enough, you are done.  If it doesn't, profile the code.  If you find that something like the switch statement is in a critical spot then you can experiment with alternatives.
But it's best to know for sure if you are smarter than your compiler for certain things.  Take for example the poster on these forums that found that manually inlining methods slowed down his code, rather than sped it up.    So try the jump table method and benchmark it.  And keep in mind that particular characteristics of the switch() statment might lead the compiler to optimize differently...  for instance if you have any cases that 'fall though' (no break or return at the end) or possibly if your case values are not sorted the translation to a table will not be easy for the compiler.  And if your table accesses have array bounds checking that may or may not be optimized away by the JIT compiler...  there are so many variables...
You never know... I'm not even sure if javac does that sort of optimization.

Offline xbrain

Senior Newbie




Java games are the Future


« Reply #16 - Posted 2002-12-02 06:46:46 »

I agree with Seb. Your design matches what I think could be a good design (more object oriented). I would like to add that on a Hostpot VM, the result would be better because the VM would easily identify the most frequent opcodes array access. We all know that in a game 90% of the CPU is monopolized by the same 20% of the code.
We progress Smiley

XBrain - Don't let others control your Mind...
Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Reply #17 - Posted 2002-12-02 07:28:52 »

Quote
Quote:A good compiler will use jump tables for switch statements when appropriate.   A switch does not necessarily compile to the equivalent of  
if..else-if...else-if...  

True, but you have no control over it.


You do have some control over it. If you put your cases in ascending order, it will be compiled to a jump table.

We have tried your suggestion to mimic a jump table in java. The MC6809 emulator for example at one point was built exactly how you described (except that we use int arrays for memory because they are faster). However, it was rewritten to a switch/case because that turned out to be a good deal faster.
Might be 'less OO' but as it turns out, emulators are not really java's cup of tea performance wise, so to me 'less OO' is better in this case.

Erik

Offline Seb

Senior Newbie




import com.acme.Bomb;


« Reply #18 - Posted 2002-12-02 17:18:54 »

Well, that was my question, if anybody had tried the method.

I'm also not sure if javac uses internal jump tables at any point for switch statements.

As for doing it by hand. I figured that although bound's cheking is NOT turned off here, (that usually works only for array access inside some 'for' & 'while' loops where the array index is also the loop counter) it could still be faster than even short switch statements.

Also, if javac does implement jump tables when it can, these are rare occasions, one would probably have to be careful to sort the cases, make sure none fall through, make sure there are no gaps, etc..

Seems complicated and any careless modifications could alter performance wildly.

An intermediate solution is probably to research how the original processor did the decode stage and code something analogous to that circuitry.

I'm calling this intermediate because I figure in most cases CPUs start decoding by the most significant bit or bits, perhaps ending up with something similar to a search tree, and some parts in this tree could implement jump tables and some other switch statements or directly execute some opcode.

Seb

Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Reply #19 - Posted 2002-12-02 19:06:44 »

Quote
I'm also not sure if javac uses internal jump tables at any point for switch statements.


Well, AFAIK there are 2 different opcodes in the java bytecodes for switch/case, one of which uses a jump table and is fastest and is used when the cases are in ascending order. I don't think that small gaps make any difference (my z80 emulator does have gaps and I checked that it still uses the lookup switch).
You're right it's not pretty and can be vulnerable code, but it helps when you only put function calls in the switch.

Anyway it's not that bad. I mean, how complicated is it to put all cases in ascending order and not forget the break; statements?
Not really.

BTW,  Seb, could you explain how your intermediate and final solution would help in making things faster?
(Or for which problem was it the solution?  Roll Eyes)
I figure that the closer to the hardware you emulate, the slower the emulation will be because you'll be emulating unneeded details.
(Ok, those details are not always unneeded. For example the Sega sys32 console and the SNES need accuracy to the cycle to keep everything in sync.)

What I think is that the greatest slowdown in emulation using java is that java datatypes don't map very well to native datatypes a lot of the time.
In java, there's a lot of casting and/or a lot of masking needed to convert its datatypes to the emulated datatypes.
In C you have more flavours and low-level control over datatypes for example. One thing that would speed up my z80 emulation a good deal for example is when java had something like UNION.
I think this impacts speed more than the opcode interpretation itself.

Greetings,
Erik

Offline Seb

Senior Newbie




import com.acme.Bomb;


« Reply #20 - Posted 2002-12-02 19:56:12 »

Well, I don't know if it would make things faster, nor do I propose it as a final solution. I'm just discussing the different methods since I find it worth while. With the switch statement working through a lookup, I think it's hard to find something that works faster.

I agree it's not optimal to emulate close to the hardware in many cases. However in the particular case of part of the decode circuitry it appears to me it could be good. But I'm just imagining a particular type of decode circuitry. One that would look like an opcode search tree.

Using a search tree, one could be garanteed that it would find the code for an opcode within, say, 8 or so tree lookups max, for 8 bit opcodes. It also seems flexible enough to account for different bit sizes in opcodes naturally.

This tree wouldn't have to exist as a complete tree either. It's only the search that's structured hierarchically, starting by verifying the most significant bits first, not necessarily one bit at a time.

Different points in this hierarchical decode I suggested one could implement either with switch statements or with jump tables. But, as you said, it can all be done with switch if being careful. You're right, it's not that hard to maintain that code, just gotta' know how to get the compiler to compile right.

Seb

Offline Seb

Senior Newbie




import com.acme.Bomb;


« Reply #21 - Posted 2002-12-02 20:30:27 »

Xbrain,
Although the VM could possibly identify which opcodes are used more frequently, I'm afraid there's not much it can do to optimize those particular code paths.

It can't inline anything a that point based on frequency of access, it has to handle all possible situations. So there's not much optimization possible other than a jump table a this point.

Even the method I proposed in the previous post, the hierarchical decode, is probably slower, I only like it because it seems like an elegant solution, but that would have to be seen in an actual implementation.

Seb

Offline swpalmer

JGO Coder




Where's the Kaboom?


« Reply #22 - Posted 2002-12-03 00:23:49 »

Quote
It can't inline anything a that point based on frequency of access

I think it would probably inline the functions into the switch cases rather than make the function call.   But other than that you are right.. it can't really make the lookup for a particular case go faster.

Offline swpalmer

JGO Coder




Where's the Kaboom?


« Reply #23 - Posted 2002-12-03 00:34:04 »

Quote
Using a search tree, one could be garanteed that it would find the code for an opcode within, say, 8 or so tree lookups max, for 8 bit opcodes. It also seems flexible enough to account for different bit sizes in opcodes naturally.


Search trees (TreeMap) or has maps would work.. but the lookup operation would still be much more expensive than a bounds check on an array.
For 8 bit instructions it is easy enough to fill any potential gaps in the array with a call to trigger an invalid opcode exception.
In this particular case thoughas erikd has determined by experimentation, the switch case is more optimal still.. ultimately compiling down to a lookup table in the bytecode.. so you won't likey get any faster than that for instruction dispatching.
Perhaps more optimizations are waiting to be found in the ALU emulation?
I just had an interesting thought...
Coud you do something really advanced like emulate a very modern CPU core, so for instance on a dual processor machine you could fetch two instructions at a time, and execute them in parallel?  Maybe at that point you would have to get fancy and do instruction reordering etc.. That would be something!
Entertaining perhaps.. but probably too complex and not worth the effort in the end.

Offline Seb

Senior Newbie




import com.acme.Bomb;


« Reply #24 - Posted 2002-12-03 05:59:20 »

Well, a TreeMap wasn't in my mind at all.

Lemme' try to say it clearly in one posting.

CPU opcode formats usually have some sort of semantic logic to them. Take some z80 instructions for example (What I'm gonna say is oversimplified but it makes the point) :


----------- 76 543 210  76543210
LD r,r'     01 ddd sss
LD r, (HL)  01 ddd 110
LD (HL), r  01 110 sss
LD r,n           00 ddd 110  nnnnnnnn
LD (HL), n  00 110 110  nnnnnnnn


Well, having the most significant bit (bit 7) equal to 0 means we're doing a load/store.

  • The next bit (bit 6) equal to 1 means there are no immediate operands and we proceed to figure out destination and source from the next two pairs of 3 bits. If one of these is 6 then it means indirect addressing for source or destination, using the address at HL, otherwise we're referring to one of 7 registers.
  • If instead, bit 6 were 0, then we have an operand on the next byte. If the source is 6, were once more using indirect addressing.


The thing is, this structure lends itself to hierarchical decode (hence my commenting this being similar to a tree search, although it wasn't the most appropriate way to describe it I admit  Tongue )

This, of course, is not as efficient as the switch compiled to a table lookup, I just mentioned it because it seemed elegant and because I wasn't sure at the time if javac did lookup-table optimization on switch/case so I figured it could provide a performance boost. But I wouldn't implement this with a TreeMap, I would hardcode the decoding process.

Seb

Offline Seb

Senior Newbie




import com.acme.Bomb;


« Reply #25 - Posted 2002-12-03 06:07:30 »

Here's another emulation framework in Java. It's not as functional as Cottage.

http://jfrace.sourceforge.net/index.html

Seb

Offline CDU

Senior Newbie




Java games rock!


« Reply #26 - Posted 2002-12-05 05:25:29 »

I'm newbie here (and in Java) and emulator's fan...

Aren't u think Java's speed a problem to develop emulator as SNES or genesis emulator Huh PSX emulator Huh  (I can dream, can't I Huh  Wink )

When I see C or C++ emulators in my AMD 500Mhz + TNT2 so slow (ie less than 60 fps)...

@++

PS : scuse me for my very poor  english... French are very bad for foreign langage  Embarrassed


Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Reply #27 - Posted 2002-12-05 20:59:24 »

If you think emulators written in C or C++ are slow, then yes, there's also a speed problem with java.
But remember that emulators are extremely demanding in terms of CPU speed, no matter in what language you make them.
I believe that if the emulated CPU's datatypes map fairly well to java's datatypes that it doesn't make that much difference in speed if you use java or C++.
For example, when you run CottAGE using the IBM JVM, many games run just about as fast as MAME32 in a window. (it's a shaky comparison, but hey Smiley)

I think a good SNES, Genesis or PSX emulator written in java is possible, if you have a beefy CPU in your machine, but probably it wont be faster than their C or C++ based brothers.

Offline CDU

Senior Newbie




Java games rock!


« Reply #28 - Posted 2002-12-06 07:12:00 »

Well... if it's true, sweet to me to imagine a JBle*m ;-)

When the beginning of this dream ?

Do u know if "advanced console or computer" emulators are written with Java ?








Offline erikd

JGO Ninja


Medals: 16
Projects: 4
Exp: 14 years


Maximumisness


« Reply #29 - Posted 2002-12-06 21:25:06 »

I know an Atari ST emulator (which hardware is comparable to a Genesis). A co-author of CottAGE (Gollum) has made a Gameboy Advance emulator in java and even a PC/MS DOS emulator in java.
There's no more advanced emulators yet, but I know this will change soon.
JEF (Java Emulation Framework) has gone through some serious changes and rewrites and is getting some cool additions like 68000 emulation, Q-Sound, ADPCM, 2 different FM chips and more.

Greetings,
Erik

Pages: [1] 2
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

Dwinin (19 views)
2014-09-12 09:08:26

Norakomi (54 views)
2014-09-10 13:57:51

TehJavaDev (63 views)
2014-09-10 06:39:09

Tekkerue (31 views)
2014-09-09 02:24:56

mitcheeb (53 views)
2014-09-08 06:06:29

BurntPizza (37 views)
2014-09-07 01:13:42

Longarmx (23 views)
2014-09-07 01:12:14

Longarmx (27 views)
2014-09-07 01:11:22

Longarmx (27 views)
2014-09-07 01:10:19

mitcheeb (35 views)
2014-09-04 23:08:59
List of Learning Resources
by Longor1996
2014-08-16 10:40:00

List of Learning Resources
by SilverTiger
2014-08-05 19:33:27

Resources for WIP games
by CogWheelz
2014-08-01 16:20:17

Resources for WIP games
by CogWheelz
2014-08-01 16:19:50

List of Learning Resources
by SilverTiger
2014-07-31 16:29:50

List of Learning Resources
by SilverTiger
2014-07-31 16:26:06

List of Learning Resources
by SilverTiger
2014-07-31 11:54:12

HotSpot Options
by dleskov
2014-07-08 01:59:08
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!