Java-Gaming.org    
Featured games (78)
games approved by the League of Dukes
Games in Showcase (429)
Games in Android Showcase (89)
games submitted by our members
Games in WIP (466)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: 1 2 [3]
  ignore  |  Print  
  Review my thread pattern?  (Read 8330 times)
0 Members and 1 Guest are viewing this topic.
Offline theagentd
« Reply #60 - Posted 2011-09-16 11:11:13 »

Hehe, this is getting kind of ridiculous. He's making a Minecraft server for god's sake. Single threaded IO should be better as it's easier to use.

Oh, was there an OP?  Smiley

Why hello  Tongue

Quite a conversation I started...haha

Your thread pattern looks good. However you should only multithread things that actually need it. If something executes really fast, the synchronization between the threads are going to make it slower. The best is probably to keep the number of threads dynamic. That way you can adapt the number of threads to the workload.

Myomyomyo.
Offline nsigma
« Reply #61 - Posted 2011-09-16 11:14:11 »

What you're lacking is a benchmark relative to the argument we're having; that is, one comparing multithreaded vs singlethreaded writing

Yes, I've said this a couple of time.  And there's also a logical reason an OS or filesystem might treat these conditions differently (even if none do in actuality).

(either ncq or queue depth are the slowing factor, java blocks until the OS can queue IO requests. I'm going to take a wild guess and say nsigma doesn't have ncq (enabled))

Huh?  Well, the system reports ncq is enabled, and the queue depth as 31.

I'm intrigued from what I understand of ncq (given that my knowledge in this area could be counted on the fingers of no hands  Smiley ) why the results I reported above would suggest this - I would have thought the opposite.

Praxis LIVE - open-source intermedia toolkit and live interactive visual editor
Digital Prisoners - interactive spaces and projections
Offline counterp

Senior Member


Medals: 11



« Reply #62 - Posted 2011-09-16 15:36:56 »

unless your disk is fragmented, writing files sequentially like that should be faster without ncq (although probably the rest of your system would benefit from ncq). I guess this wasn't the main factor though

then my 2nd point applies, which is that the queue can't be processed in time when you continuously add requests that can't be completed in time making the queue fill up, causing writing/reading to slow at a constant rate as more things which need to be queued are slowly more delayed at being queued (as long as you are writing/reading at a constant, fast rate)

realistically, unless your running some popular website (i.e. not minecraft), you will not be writing/reading at this rate so this will never become a problem.
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline Nate

JGO Kernel


Medals: 129
Projects: 3
Exp: 14 years


Esoteric Software


« Reply #63 - Posted 2011-09-16 20:51:18 »

My fancy pants 256gb SSD (probably shaved some life off it :p):
Quote
wrote 1 files of 512MB in 0.819 sec, total throughput: 625.15265MB/sec
wrote 2 files of 512MB in 1.833 sec, total throughput: 558.64703MB/sec
wrote 3 files of 512MB in 2.488 sec, total throughput: 617.36334MB/sec
wrote 4 files of 512MB in 7.421 sec, total throughput: 275.9736MB/sec
wrote 5 files of 512MB in 10.72 sec, total throughput: 238.80597MB/sec
wrote 6 files of 512MB in 13.155 sec, total throughput: 233.52338MB/sec
wrote 7 files of 512MB in 15.02 sec, total throughput: 238.61517MB/sec
wrote 8 files of 512MB in 17.925 sec, total throughput: 228.50768MB/sec
The 3 files was consistently faster like the above.

Here's my 750gb HDD:
Quote
wrote 1 files of 512MB in 0.457 sec, total throughput: 1120.3501MB/sec
wrote 2 files of 512MB in 1.911 sec, total throughput: 535.8451MB/sec
wrote 3 files of 512MB in 2.506 sec, total throughput: 612.92896MB/sec
wrote 4 files of 512MB in 3.859 sec, total throughput: 530.70746MB/sec
wrote 5 files of 512MB in 41.242 sec, total throughput: 62.072643MB/sec
I got bored and stopped it. Then I ran it again:
Quote
wrote 1 files of 512MB in 13.607 sec, total throughput: 37.62769MB/sec
wrote 2 files of 512MB in 40.309 sec, total throughput: 25.403757MB/sec
Again I got bored and stopped it. Did you ruin my HDD controller?!
Yes. Obviously harddrives aren't meant to have data written to them. ... Isn't that just the OS caching the data in RAM and returning instantly?
Seems you have a bit of an attitude. SSDs have a limited number of program/erase cycles. The 750gb HDD should be faster than 37MB/s.

Offline theagentd
« Reply #64 - Posted 2011-09-16 20:59:26 »

My fancy pants 256gb SSD (probably shaved some life off it :p):
Quote
wrote 1 files of 512MB in 0.819 sec, total throughput: 625.15265MB/sec
wrote 2 files of 512MB in 1.833 sec, total throughput: 558.64703MB/sec
wrote 3 files of 512MB in 2.488 sec, total throughput: 617.36334MB/sec
wrote 4 files of 512MB in 7.421 sec, total throughput: 275.9736MB/sec
wrote 5 files of 512MB in 10.72 sec, total throughput: 238.80597MB/sec
wrote 6 files of 512MB in 13.155 sec, total throughput: 233.52338MB/sec
wrote 7 files of 512MB in 15.02 sec, total throughput: 238.61517MB/sec
wrote 8 files of 512MB in 17.925 sec, total throughput: 228.50768MB/sec
The 3 files was consistently faster like the above.

Here's my 750gb HDD:
Quote
wrote 1 files of 512MB in 0.457 sec, total throughput: 1120.3501MB/sec
wrote 2 files of 512MB in 1.911 sec, total throughput: 535.8451MB/sec
wrote 3 files of 512MB in 2.506 sec, total throughput: 612.92896MB/sec
wrote 4 files of 512MB in 3.859 sec, total throughput: 530.70746MB/sec
wrote 5 files of 512MB in 41.242 sec, total throughput: 62.072643MB/sec
I got bored and stopped it. Then I ran it again:
Quote
wrote 1 files of 512MB in 13.607 sec, total throughput: 37.62769MB/sec
wrote 2 files of 512MB in 40.309 sec, total throughput: 25.403757MB/sec
Again I got bored and stopped it. Did you ruin my HDD controller?!
Yes. Obviously harddrives aren't meant to have data written to them. ... Isn't that just the OS caching the data in RAM and returning instantly?
Seems you have a bit of an attitude. SSDs have a limited number of program/erase cycles. The 750gb HDD should be faster than 37MB/s.
I... I'm sorry... I didn't mean to insult you with my ingenious jokes... FYI I do own an SSD, and I'm full aware of the limited number of writes you can do.

Myomyomyo.
Offline Nate

JGO Kernel


Medals: 129
Projects: 3
Exp: 14 years


Esoteric Software


« Reply #65 - Posted 2011-09-17 00:03:58 »

Not insulted, just sarcasm out of the blue seemed odd. Smiley

Offline counterp

Senior Member


Medals: 11



« Reply #66 - Posted 2011-09-18 06:16:13 »

I do believe counterp just got served. Cool

And this is why you should avoid siding with people in situations where you have no knowledge of the topic concerned, especially when you have nothing to contribute.  Wink
Offline theagentd
« Reply #67 - Posted 2011-09-18 10:13:57 »

Jesus, why are people so sensitive? And "related to the topic"? This thread went OT from page one. >_>

Myomyomyo.
Offline nsigma
« Reply #68 - Posted 2011-09-18 13:34:17 »

then my 2nd point applies, which is that the queue can't be processed in time when you continuously add requests that can't be completed in time making the queue fill up, causing writing/reading to slow at a constant rate as more things which need to be queued are slowly more delayed at being queued (as long as you are writing/reading at a constant, fast rate)

Sorry  Huh , still don't get what point you're trying to make here, or how it relates to the results I posted.  Are you saying this is the reason that the throughput for 1 file is slower because of this, or that this explains why I get consistent throughput no matter how many files are being written to?  It sounds more like an argument for throughput decreasing, though that may be how I'm reading what you've written.

I understand how queue saturation through memory use and ultimately drive write speed should be the limiting factor in a filesystem that can handle multiple concurrent writes effectively (which ext4 seems to be getting closer to).  Would I be right to suspect that all / most of those who have posted results to Riven's code have NCQ enabled, and therefore the relevant differences are queuing in the OS / filesystem and not the hardware?

I do believe counterp just got served. Cool

And this is why you should avoid siding with people in situations where you have no knowledge of the topic concerned, especially when you have nothing to contribute.  Wink

Quite! This was the post that made me want to join in this discussion, and read a bit more about the issues - as someone with no knowledge of the topic, I felt perfectly placed to jump in and side with no-one!  Grin

Praxis LIVE - open-source intermedia toolkit and live interactive visual editor
Digital Prisoners - interactive spaces and projections
Offline counterp

Senior Member


Medals: 11



« Reply #69 - Posted 2011-09-18 20:12:00 »

then my 2nd point applies, which is that the queue can't be processed in time when you continuously add requests that can't be completed in time making the queue fill up, causing writing/reading to slow at a constant rate as more things which need to be queued are slowly more delayed at being queued (as long as you are writing/reading at a constant, fast rate)

Sorry  Huh , still don't get what point you're trying to make here, or how it relates to the results I posted.  Are you saying this is the reason that the throughput for 1 file is slower because of this, or that this explains why I get consistent throughput no matter how many files are being written to?  It sounds more like an argument for throughput decreasing, though that may be how I'm reading what you've written.

I understand how queue saturation through memory use and ultimately drive write speed should be the limiting factor in a filesystem that can handle multiple concurrent writes effectively (which ext4 seems to be getting closer to).  Would I be right to suspect that all / most of those who have posted results to Riven's code have NCQ enabled, and therefore the relevant differences are queuing in the OS / filesystem and not the hardware?

Yes to your last point.
Did you try running my test by the way?
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline ra4king

JGO Kernel


Medals: 322
Projects: 2
Exp: 4 years


I'm the King!


« Reply #70 - Posted 2011-09-19 04:06:09 »

It's fun taking sides,especially when I learn about this stuff after you guys explain it Grin

Pages: 1 2 [3]
  ignore  |  Print  
 
 
You cannot reply to this message, because it is very, very old.

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

xsi3rr4x (81 views)
2014-04-15 18:08:23

BurntPizza (73 views)
2014-04-15 03:46:01

UprightPath (84 views)
2014-04-14 17:39:50

UprightPath (67 views)
2014-04-14 17:35:47

Porlus (84 views)
2014-04-14 15:48:38

tom_mai78101 (107 views)
2014-04-10 04:04:31

BurntPizza (167 views)
2014-04-08 23:06:04

tom_mai78101 (263 views)
2014-04-05 13:34:39

trollwarrior1 (214 views)
2014-04-04 12:06:45

CJLetsGame (223 views)
2014-04-01 02:16:10
List of Learning Resources
by SHC
2014-04-18 03:17:39

List of Learning Resources
by Longarmx
2014-04-08 03:14:44

Good Examples
by matheus23
2014-04-05 13:51:37

Good Examples
by Grunnt
2014-04-03 15:48:46

Good Examples
by Grunnt
2014-04-03 15:48:37

Good Examples
by matheus23
2014-04-01 18:40:51

Good Examples
by matheus23
2014-04-01 18:40:34

Anonymous/Local/Inner class gotchas
by Roquen
2014-03-11 15:22:30
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!