efficient Random Writes

[moogie]

I have decided to try my hand at making a simple peered file transfer system based on UDP.

This restricts the size of the blocks of the file being transferred  to 63k so that it will fit in a UDP packet with some control variables.

Since the reading of blocks from one client to send to another client will be random (to attempt to evenly spread the blocks and thus work load) there is little i can do improve the random IO block reads. However this does not seem to be the major bottleneck as i achieve over 5meg a second of random blocks when using one source. With more more sources this throughput increases.

The problem is with the random writes of the blocks received... how can i speed up the random writes?

currently i can only think of two ways, both with disadvantages:

1. queued write

2. post transfer sorting

With the queued write there is to be a thread who polls a queue and takes the first block from the queue and writes it to the local disk. At the same time, blocks from one or multiple sources are adding to the end of the queue.

This as the advantage of not slowing down the throughput of the incoming blocks for the most part, however depending on the maximum size of the queue it will reach a point where it will have to wait until a block is written before accepting more blocks to the queue.

The post transfer sorting will allow the maximum throughput and will never have the situation of artificially pausing recieving the blocks as it will be sequentially writing the blocks as they are recieved most likely in a buffered fashion. In addition to the writing of the blocks, the actual positions of the blocks are also recorded. This is so that after the transfer is finished the file can be reconstructed using the positions stored.

The disadvantage for this method is two fold: it requires more disc space as it needs to store the blocks and the positions and it has to spend time reconstructing the file.


Is there a better way? of if not, which is most commonly used?