Does anyone know when AIO will be integrated for sockets into the kernel. I
was just recently testing out some code that multiplexed socket data and found
that multiple reads will get dealt with out of order.
Due to this I bet multiple sequential BeginWrites will also result in
non-deterministic ordering (though I'd assume all the data from each will be
pushed out in the same order because its not like BeginRead where you expect
not all teh data to be ready/read at one time).
This is very different from Microsoft's IOCPs in which anything posted will get
completed in order... the callbacks' order is not deterministic but that isn't
much of a fixable issue since thread starting is not deterministic.
Does anyone know of a good way to deal with maximizing socket throughput for
linux, perhaps also for windows (but I'm pretty sure queueing up reads is best).
I'm writing up a library that will simplify demultiplexing data from sockets
such that events will get fired when a clump of data is received.
Essentially this is the process:
Writer:
Collect all the data needed, write the length, write the data.
[The length and data could be in different arrays so that copying is
unneeded or the data could be set such that there are 4 bytes to store the
length].
- the writer could be set up so that an app could Post the data
asynchronously and the writer would deal with setting up the data arrays async
or sync (perhaps depending on data length or estimated data length) and then
usee BeginWrite to post the data. With linux perhaps it'd be best to just to
Write if a new worker thread was spawned or the threadpool was used (unless the
expected client/write count would be much larger than expected. Just using
Async mode might also be good for the writer at least... not quite sure.
Reader:
Post a bunch of async reads that are numbered so that data can be
organized in order to patch them together (since callbacks will not necessarily
be in order). [with linux I guess I might just have to do 1 at a time]
Upon receipt of data, post another begin read, put the data into a
sorted list based upon the number, optionally do data processing on the data to
see if we can patch together a message.
Upon receipt of a full message, post an event/callback to the user.
(perhaps justs dumping the message into an array which the user would run
through occasionally to deal with messages).
For the reader I did a small optimization to deal with memory:
- Set up a BlockManager that keeps a set of free blocks.
The BlockManager is empty at the start and has a set size at
construction.
Upon a request for a Block, the manager checks if the free list has a
block to use, if not, a new block of the set size is created and returned.
When the user is done with the block, it can post it back to the
manager for reuse.
No assumptions are made about the contents of the block when it comes
from the block manager (though in my case I make it responsible for atomically
incrementing a blockNumber value which is assigned upon request for the block.
That's used for block ordering on receive.
As soon as I clean up the hack code that I was trying to do to see where the
problems that were occuring, I'll set up a sourceforge project for this C# code
(which I may add more C# network stuff)... I might use Savannah though so that
I can use SVN which I prefer.
For the code I had set up, it worked perfectly on Windows 2003 through many
many many iterations (I had a random length chosen for the data and when a data
block with the length of 3 was received, the callback that read in the message
stuff stopped the server and everything closed (as I set it up)). On linux,
this test runs perfect as long as no multiplexing is done (basically I just set
the number of queued beginReads to 1 [I had Windows 2003 work with 2, 20, and
even 50 queued reads]).
[through a remote desktop connection to a school server, gotta love GRdesktop,
lol]
If any of you have worked out how to queue up multiple beginreads with Mono in
Linux please let me know!
Also, it'd be useful to know if multiple beginwrites are done in order or
not... I don't have the time right now to test this out.
Sorry if this message is all over the place as to topics... drank a big
rum&coke to help quell the frustration I was feeling over the issues with my
seemingly perfect code (though it had become swiss cheese with all the
console.writelines i had to put in it, since even if I had used tracing or
visual studio .net [which turns out wouldnt have cought it] would be quite an
issue with multithreading).
Oh, as a little aside... as I was looking for stuff regarding async IO and
serialization and such (since basically I want to serialize asynchronously
objects or chunks of data) I found something to do with Guids and serializing
them. Apparently Microsoft went against their naming things to mean something
in DotNet when designing/implementing the Guid class.... The data variables are
_a,_b,_c,_d ... also not of the same type so looking at them isnt very
useful... whereas Mono's Guids have meaningful variable names.. so
serialization wouldn't work. Though I found pleasantly that they kept the
actual variable names as comments so at least looking at the code would provide
some idea what they really are. With Mono though, the names don't mean so much
as they would with Microsoft because instead of using timestamps and such, mono
just uses random values. Somewhat Ironic because apparently Microsoft uses
that, but shoves it into the alphabet, and Mono doesnt, but shoved it into
meaningful names for Microsoft's setup, lol.
--
Thomas Harning Jr.
_______________________________________________
Mono-devel-list mailing list
Mono-devel-list@lists.ximian.com
http://lists.ximian.com/mailman/listinfo/mono-devel-list
