Alexander, Thanks for the info on the disk I/O. It certainly explains what I was seeing. I didn't expect the write to block (unless I had the flush option set), but I did expect to see the read blocked, which didh't happen.
Peace, Chaz Alexander Pevzner wrote: > Chaz, > > in UNIX the disk I/O considered non blocking. It means that disk file > descriptor will be always ready for reading/writing if you will pass it > to select() or poll(). And although the actual I/O operation may block, > you will not be able to control this kind of blocking using the > event-driven I/O (multithreaded architecture may help, but may not, > depending on a actual system kernel implementation). > > This is not so stupid decision, because 1) disk operations are usually > backed by the memory cache 2) overlapped disk file I/O looks quite > overcomplicated for a practical usage. > > I've heard that Windows allows the reasonable use of the disk file > handles for overlapped I/O, but I'm not 100% sure. > > I believe that Python etc follows the UNIX model of life. For > portability across platforms, in particular. > > Chaz. wrote: >> Sam, >> >> Thanks for the information, things like this always prove the point. I >> was wondering if anyone has information on the use of event programming >> for file I/O? I know in the Python/Twisted world there is a belief that >> event level programming on file i/o is a waste - that you might just as >> well do normal blocking opens/read/write/close since the kernel will do >> most of the heavy lifting. Does any one know if this is true? >> >> Peace, >> Chanz >> >> Sam Berlin wrote: >>> Yes, true polling is very bad, as is any kind of sleeping or delaying >>> of receiving events. For a few versions of LimeWire, we had a >>> Thread.sleep(50) in the selector thread, in order to work around a few >>> JDK bugs. This had the side-effect of limiting data transfer in >>> proportion to the size of the buffer allocated to read the data, which >>> ended up being ~120KB/s. Once we removed the sleep and woke up the >>> selector whenever something became interested in reading, LAN >>> transfers shot back up to ~5MB/s (or whatever was the saturation point >>> of the LAN). Moral of the story: don't artificially limit your >>> selector. >>> >>> Sam >>> >>> On 10/30/06, Daniel Stutzbach <[EMAIL PROTECTED]> wrote: >>>> I think you are confusing polling with calling poll(). Despite the >>>> name, poll() does not do polling. It blocks until one of a set of >>>> file descriptors becomes ready, then tells you which ones. select() >>>> does the same thing with slightly different semantics. poll() is >>>> good. Polling, on the other hand, is almost always bad. >>>> >>>> On 10/30/06, Lemon Obrien <[EMAIL PROTECTED]> wrote: >>>>> "polling" -- polling is a good thing with p2p; LimeWire is trying to do >>>>> polling; it reduces the number of threads...if you do one for udp, >>>> another >>>>> for tcp, and another for network maintance, you already have three >>>> threads >>>>> running; not even counting inteface...or any thread you'll have to >>>> spawn to >>>>> do udp transfers, or searches... >>>>> >>>>> yes i know "polling" is not the attractive way to create a >>>> server...but...in >>>>> this case; i believe it's preffered. >>>>> >>>>> just my opinion. >>>>> >>>>> "Chaz." <[EMAIL PROTECTED]> wrote: >>>>> I think your are confused about ACE. It might be single threaded but it >>>>> doesn't using polling. Instead it uses event based scheduling and state >>>>> machines. It is a very sophisticated, low overhead system. Check it out >>>>> before "dissing" it. >>>>> >>>>> In the python arena Twisted is a single thread, event passed framework >>>>> and the BitTorrent client is written using it. I have also written an >>>>> application using it that have 5 services running at once and is a very >>>>> sophisticated p2p application (supporting multiple requests at once). >>>>> >>>>> The reason I mentioned ACE at the start of all this was to point out >>>> you >>>>> can really make a sophisticated, high performance application without >>>>> much effort. >>>>> >>>>> Chaz >>>>> >>>>> Lemon Obrien wrote: >>>>>> you're speaking to people who have 15 years plus experience...i >>>>>> persoannly started coding professionally in 92, two years before the >>>>>> internet. >>>>>> >>>>>> abstraction layer. abstraction layer...if it took a whole team to >>>> create >>>>>> ACE, Twisted, Whatever,...with years of testing...well; that's >>>> probably >>>>>> a case where too many cooks are in the kitchen...and after looking at >>>>>> ACE architecture...not to be harsh; but it looked disorganized; and >>>>>> seemed to suck...plus someone mention "single threaded" p2p and >>>> "single" >>>>>> threaded systems don't mix. >>>>>> >>>>>> anyway...i don't believe in following rules laid down by IT people >>>> who >>>>>> do nothing but sit in cubicles. >>>>>> >>>>>> */Antoine Pitrou /* wrote: >>>>>> >>>>>> >>>>>> Le jeudi 26 octobre 2006 à 12:28 -0700, Alex Pankratov a écrit : >>>>>>> But average Linux/Windows coder with a good understanding of >>>> language >>>>>>> standards (assuming C/C++) should be able to produce non-UI >>>>>> abstraction >>>>>>> layer *much* faster than it would take him to learn something like >>>>>> ACE. >>>>>> >>>>>> Wow, it's a joke right? >>>>>> >>>>>> I can't help thinking how tedious it must be to workaround all the >>>>>> various subtleties of each platform's network stack, API, threading >>>>>> semantics etc. There is a reason why people decided to write ACE, >>>>>> Twisted, apr... in the first place. >>>>>> >>>>>> Not to mention that software like ACE or Twisted has been in use and >>>>>> actively maintained for years, and chances are they make the right >>>>>> decisions in a lot of places. Not because their designers are >>>> genious, >>>>>> but because they have actually been tested and fixed to work >>>> correctly >>>>>> in a *lot* of situations. >>>>>> >>>>>> What are the odds that an "average Linux/Windows coder" would be >>>> able to >>>>>> come up with the right decisions at the first attempt to code an >>>> network >>>>>> abstraction library? At what cost? >>>>>> Perhaps "average coder" has a special meaning in your mouth, >>>> because I >>>>>> can't imagine how your claim can be realistic. >>>>>> >>>>>> Oh and by saying "Linux/Windows", you already leave out the BSDs, >>>> MacOS >>>>>> X, and various other OS flavours (embedded stuff, etc.). >>>>>> >>>>>> >>>>>>> Project leader who pushed for using ACE had no better option >>>>>>> than to suggest purchasing paid support from ACE people. >>>>>> And how is that a problem exactly? Does your in-house "average coder" >>>>>> work for free? >>>>>> >>>>>> If the same bug had occurred with an in-house developed library, who >>>>>> could you have paid to solve the problem? >>>>>> Answer: nobody, because nobody outside knows your library, and the >>>> guy >>>>>> who coded is an "average coder" by your own words, so he would have a >>>>>> very hard time debugging bizarre, erratic threading issues. >>>>>> >>>>>> The very fact that you could find someone to fix that - probably >>>>>> specific or exotic - problem is highly positive. >>>>>> >>>>>> regards >>>>>> >>>>>> Antoine. >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> p2p-hackers mailing list >>>>>> [email protected] >>>>>> http://lists.zooko.com/mailman/listinfo/p2p-hackers >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> You don't get no juice unless you squeeze >>>>>> Lemon Obrien, the Third. >>>>>> >>>>>> http://www.tamago.us >>>>>> >>>>>> >>>>>> >>>> ------------------------------------------------------------------------ >>>>>> _______________________________________________ >>>>>> p2p-hackers mailing list >>>>>> [email protected] >>>>>> http://lists.zooko.com/mailman/listinfo/p2p-hackers >>>>> _______________________________________________ >>>>> p2p-hackers mailing list >>>>> [email protected] >>>>> http://lists.zooko.com/mailman/listinfo/p2p-hackers >>>>> >>>>> >>>>> >>>>> You don't get no juice unless you squeeze >>>>> Lemon Obrien, the Third. >>>>> >>>>> http://www.tamago.us >>>>> _______________________________________________ >>>>> p2p-hackers mailing list >>>>> [email protected] >>>>> http://lists.zooko.com/mailman/listinfo/p2p-hackers >>>>> >>>>> >>>>> >>>> -- >>>> Daniel Stutzbach Computer Science Ph.D Student >>>> http://www.barsoom.org/~agthorr University of Oregon >>>> _______________________________________________ >>>> p2p-hackers mailing list >>>> [email protected] >>>> http://lists.zooko.com/mailman/listinfo/p2p-hackers >>>> >>> _______________________________________________ >>> p2p-hackers mailing list >>> [email protected] >>> http://lists.zooko.com/mailman/listinfo/p2p-hackers > > _______________________________________________ > p2p-hackers mailing list > [email protected] > http://lists.zooko.com/mailman/listinfo/p2p-hackers > _______________________________________________ p2p-hackers mailing list [email protected] http://lists.zooko.com/mailman/listinfo/p2p-hackers
