Hi !
Featured games (90)
games approved by the League of Dukes
Games in Showcase (715)
Games in Android Showcase (214)
games submitted by our members
Games in WIP (788)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  ISAM Database - IO and NIO  (Read 3608 times)
0 Members and 1 Guest are viewing this topic.
Offline zparticle

Senior Devvie

Thick As A Brick

« Posted 2004-05-13 14:49:15 »

I often have a need for a small easy to use database so I decided to whip up a quick library for ISAM like database access. I have the basics working now and in a couple of days will be adding indexing using single and concatenated keys. Might also write a little database explorer in swing.  A friend wants me to add network access to it as well.

I'm also using this project to learn a bit about NIO. The first version I'm working on now is all regular io package stuff. I will be converting it to NIO after it has all the features I want.

If anyone else has need for this sort of thing let me know and I'll post the library up when it is finished.

If you haven't used ISAM before setting up a database is very easy, mine is a little bit different from ISAM but has the same outcome:

// create the record format and add the coulmns you need
FixedRecordFormat format = new FixedRecordFormat();
format.addStringField("First Name",30);
format.addStringField("Last Name",30);

// create the empty database files
String location = "\temp\mydatabase";
FixedRecordFormatFile database = new FixedRecordFormatFile(location);

// add a few records
//---------------------------------; // notice that once a database has been created you don't have to set up the record format

Record rec = new Record(database.getFormat());
rec.setStringFieldValue("First Name","Scott");
rec.setStringFieldValue("Last Name","Shaver");

rec.setStringFieldValue("First Name","Bert");
rec.setStringFieldValue("Last Name","Shaver");

// other commands
database.rewind(); // set the current record position to the first record
database.setCurrentRecord(recIndex); // set a specific record position as the current record
database.nextRecord(); // move to the next record position
database.previousRecord(); // move to the previous record position
database.deleteRecord(); // delete the record at the current record position
database.writeRecord(recordToWrite); // write a recrod at the current record position
database.compact(); // remove any empty records from the database file
database.getRecordCount(); // find the total current capacity of the database file

Offline zparticle

Senior Devvie

Thick As A Brick

« Reply #1 - Posted 2004-05-20 13:06:10 »

Okay so I have fiished the regular IO version of the database and converted it to NIO. However I'm having difficulty with a couple of items.

First, there seems to be no way to increase the size of a file that is memory mapped. This means that in the nio version every time I create a new record at the end of the file I have to:

a> close the file
b> open it with regular io and extend it
c> open the file with nio again
d> write the new record into the empty spot at the end.

This is obviously extreemly slow for lots of inserts at the end of the file. Any clue how to get around this?

Second, there is an option to compact the database file removing all empty/deleted records. So I move all the non-empty records to the top of the file and then attempt to truncate the file. This causes a problem just like the first one above. However this time when I try to do the close, truncate, open flow I get an IOException with a windows system error message:

<b>The requested operation cannot be performed on a file with a user-mapped section open.<b>

This happens on the call to truncate the file even though I have previously closed the RandomAccessFile and  FileChannel objects and set the MappedByteBuffer to null. Help?Huh

Offline blahblahblahh

JGO Coder

Medals: 1

« Reply #2 - Posted 2004-05-20 13:43:35 »

This is not a helpful response, I'm afraid, but may be useful anyway Grin. (EDIT: sorry but I have v.little time at the moment, so not enough time to go look up my own refs and past experiences; hopefully what I just spouted is at least slightly helpful)

Memory-mapping in 1.4.x has sufficiently many serious bugs as to be considered unusable by many people Sad. It's a bit like Socket NIO in 1.4.0 and 1.4.1 - there were so many bugs that it COULD NOT be used in real applications, unless you were willing to make huge concessions (one of which I recall was "never run on windows" in several not uncommon scenarios).

So, first port of call is the JDC Bug Parade and an hour of searching for outstanding mem-mapping bugs I'm afraid Sad. Pay particular attention to closed and wontfix bugs - IIRC there's a couple of those that smell more of engineers trying to avoid being the Sun employee who opens the can of worms ... than of things that are genuinely not bugs.

Finally, mem-mapping is often NOT better than using vanilla much so that IIRC the API docs currently say "we suggest you only use mem-mapping where you've experimentally determined that it definitely makes a large positive difference to your particular application".

i.e. mem-mapping is really still a "beta" technology in the JDK. From micro benchmarks I've seen and done, it can be hard to find situations where mem-mapping brings significant performance boosts. At a guess I'd say this is because it's one of those areas of the JDK which so far has had very few staff and so Sun haven't had time to do much more than "bare bones" implementations that are slow inefficient and buggy. (based upon extensive experience with the general rubbishness of the nio.* and socket nio.channels.*, and it's rapid evolution through successive 1.4.x point releases...)

malloc will be first against the wall when the revolution comes...
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline Mark Thornton

Senior Devvie

« Reply #3 - Posted 2004-05-20 13:56:34 »

As the submitter of several memory mapping bugs, I have to agree with blah3.

This one is number 11 on the top 25 RFE list
Offline zparticle

Senior Devvie

Thick As A Brick

« Reply #4 - Posted 2004-05-20 14:29:50 »

Thanks guys. I guess I'm screwed for now.

On the up side I do see huge gains from NIO on the reading side. So for now I suppose I could create static databases using the IO version and read them with NIO. For more dynamic databases I'll simply only use the IO version of my lib.

Pages: [1]
  ignore  |  Print  
You cannot reply to this message, because it is very, very old.

CopyableCougar4 (200 views)
2017-03-24 15:39:42

theagentd (182 views)
2017-03-24 15:32:08

Rule (240 views)
2017-03-19 12:43:22

Rule (224 views)
2017-03-19 12:42:17

Rule (229 views)
2017-03-19 12:36:21

theagentd (248 views)
2017-03-16 05:07:07

theagentd (244 views)
2017-03-15 22:37:06

theagentd (178 views)
2017-03-15 22:32:18

theagentd (172 views)
2017-03-15 22:31:11

ral0r2 (165 views)
2017-03-03 11:52:41
List of Learning Resources
by elect
2017-03-13 14:05:44

List of Learning Resources
by elect
2017-03-13 14:04:45

SF/X Libraries
by philfrei
2017-03-02 08:45:19

SF/X Libraries
by philfrei
2017-03-02 08:44:05

SF/X Libraries
by SkyAphid
2017-03-02 06:38:56

SF/X Libraries
by SkyAphid
2017-03-02 06:38:32

SF/X Libraries
by SkyAphid
2017-03-02 06:38:05

SF/X Libraries
by SkyAphid
2017-03-02 06:37:51 is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!