Chaz, in UNIX the disk I/O considered non blocking. It means that disk file descriptor will be always ready for reading/writing if you will pass it to select() or poll(). And although the actual I/O operation may block, you will not be able to control this kind of blocking using the event-driven I/O (multithreaded architecture may help, but may not, depending on a actual system kernel implementation).
This is not so stupid decision, because 1) disk operations are usually backed by the memory cache 2) overlapped disk file I/O looks quite overcomplicated for a practical usage. I've heard that Windows allows the reasonable use of the disk file handles for overlapped I/O, but I'm not 100% sure. I believe that Python etc follows the UNIX model of life. For portability across platforms, in particular. Chaz. wrote: > Sam, > > Thanks for the information, things like this always prove the point. I > was wondering if anyone has information on the use of event programming > for file I/O? I know in the Python/Twisted world there is a belief that > event level programming on file i/o is a waste - that you might just as > well do normal blocking opens/read/write/close since the kernel will do > most of the heavy lifting. Does any one know if this is true? > > Peace, > Chanz > > Sam Berlin wrote: >> Yes, true polling is very bad, as is any kind of sleeping or delaying >> of receiving events. For a few versions of LimeWire, we had a >> Thread.sleep(50) in the selector thread, in order to work around a few >> JDK bugs. This had the side-effect of limiting data transfer in >> proportion to the size of the buffer allocated to read the data, which >> ended up being ~120KB/s. Once we removed the sleep and woke up the >> selector whenever something became interested in reading, LAN >> transfers shot back up to ~5MB/s (or whatever was the saturation point >> of the LAN). Moral of the story: don't artificially limit your >> selector. >> >> Sam >> >> On 10/30/06, Daniel Stutzbach <[EMAIL PROTECTED]> wrote: >>> I think you are confusing polling with calling poll(). Despite the >>> name, poll() does not do polling. It blocks until one of a set of >>> file descriptors becomes ready, then tells you which ones. select() >>> does the same thing with slightly different semantics. poll() is >>> good. Polling, on the other hand, is almost always bad. >>> >>> On 10/30/06, Lemon Obrien <[EMAIL PROTECTED]> wrote: >>>> "polling" -- polling is a good thing with p2p; LimeWire is trying to do >>>> polling; it reduces the number of threads...if you do one for udp, >>> another >>>> for tcp, and another for network maintance, you already have three >>> threads >>>> running; not even counting inteface...or any thread you'll have to >>> spawn to >>>> do udp transfers, or searches... >>>> >>>> yes i know "polling" is not the attractive way to create a >>> server...but...in >>>> this case; i believe it's preffered. >>>> >>>> just my opinion. >>>> >>>> "Chaz." <[EMAIL PROTECTED]> wrote: >>>> I think your are confused about ACE. It might be single threaded but it >>>> doesn't using polling. Instead it uses event based scheduling and state >>>> machines. It is a very sophisticated, low overhead system. Check it out >>>> before "dissing" it. >>>> >>>> In the python arena Twisted is a single thread, event passed framework >>>> and the BitTorrent client is written using it. I have also written an >>>> application using it that have 5 services running at once and is a very >>>> sophisticated p2p application (supporting multiple requests at once). >>>> >>>> The reason I mentioned ACE at the start of all this was to point out >>> you >>>> can really make a sophisticated, high performance application without >>>> much effort. >>>> >>>> Chaz >>>> >>>> Lemon Obrien wrote: >>>>> you're speaking to people who have 15 years plus experience...i >>>>> persoannly started coding professionally in 92, two years before the >>>>> internet. >>>>> >>>>> abstraction layer. abstraction layer...if it took a whole team to >>> create >>>>> ACE, Twisted, Whatever,...with years of testing...well; that's >>> probably >>>>> a case where too many cooks are in the kitchen...and after looking at >>>>> ACE architecture...not to be harsh; but it looked disorganized; and >>>>> seemed to suck...plus someone mention "single threaded" p2p and >>> "single" >>>>> threaded systems don't mix. >>>>> >>>>> anyway...i don't believe in following rules laid down by IT people >>> who >>>>> do nothing but sit in cubicles. >>>>> >>>>> */Antoine Pitrou /* wrote: >>>>> >>>>> >>>>> Le jeudi 26 octobre 2006 à 12:28 -0700, Alex Pankratov a écrit : >>>>>> But average Linux/Windows coder with a good understanding of >>> language >>>>>> standards (assuming C/C++) should be able to produce non-UI >>>>> abstraction >>>>>> layer *much* faster than it would take him to learn something like >>>>> ACE. >>>>> >>>>> Wow, it's a joke right? >>>>> >>>>> I can't help thinking how tedious it must be to workaround all the >>>>> various subtleties of each platform's network stack, API, threading >>>>> semantics etc. There is a reason why people decided to write ACE, >>>>> Twisted, apr... in the first place. >>>>> >>>>> Not to mention that software like ACE or Twisted has been in use and >>>>> actively maintained for years, and chances are they make the right >>>>> decisions in a lot of places. Not because their designers are >>> genious, >>>>> but because they have actually been tested and fixed to work >>> correctly >>>>> in a *lot* of situations. >>>>> >>>>> What are the odds that an "average Linux/Windows coder" would be >>> able to >>>>> come up with the right decisions at the first attempt to code an >>> network >>>>> abstraction library? At what cost? >>>>> Perhaps "average coder" has a special meaning in your mouth, >>> because I >>>>> can't imagine how your claim can be realistic. >>>>> >>>>> Oh and by saying "Linux/Windows", you already leave out the BSDs, >>> MacOS >>>>> X, and various other OS flavours (embedded stuff, etc.). >>>>> >>>>> >>>>>> Project leader who pushed for using ACE had no better option >>>>>> than to suggest purchasing paid support from ACE people. >>>>> And how is that a problem exactly? Does your in-house "average coder" >>>>> work for free? >>>>> >>>>> If the same bug had occurred with an in-house developed library, who >>>>> could you have paid to solve the problem? >>>>> Answer: nobody, because nobody outside knows your library, and the >>> guy >>>>> who coded is an "average coder" by your own words, so he would have a >>>>> very hard time debugging bizarre, erratic threading issues. >>>>> >>>>> The very fact that you could find someone to fix that - probably >>>>> specific or exotic - problem is highly positive. >>>>> >>>>> regards >>>>> >>>>> Antoine. >>>>> >>>>> >>>>> _______________________________________________ >>>>> p2p-hackers mailing list >>>>> [email protected] >>>>> http://lists.zooko.com/mailman/listinfo/p2p-hackers >>>>> >>>>> >>>>> >>>>> >>>>> You don't get no juice unless you squeeze >>>>> Lemon Obrien, the Third. >>>>> >>>>> http://www.tamago.us >>>>> >>>>> >>>>> >>> ------------------------------------------------------------------------ >>>>> _______________________________________________ >>>>> p2p-hackers mailing list >>>>> [email protected] >>>>> http://lists.zooko.com/mailman/listinfo/p2p-hackers >>>> >>>> _______________________________________________ >>>> p2p-hackers mailing list >>>> [email protected] >>>> http://lists.zooko.com/mailman/listinfo/p2p-hackers >>>> >>>> >>>> >>>> You don't get no juice unless you squeeze >>>> Lemon Obrien, the Third. >>>> >>>> http://www.tamago.us >>>> _______________________________________________ >>>> p2p-hackers mailing list >>>> [email protected] >>>> http://lists.zooko.com/mailman/listinfo/p2p-hackers >>>> >>>> >>>> >>> >>> -- >>> Daniel Stutzbach Computer Science Ph.D Student >>> http://www.barsoom.org/~agthorr University of Oregon >>> _______________________________________________ >>> p2p-hackers mailing list >>> [email protected] >>> http://lists.zooko.com/mailman/listinfo/p2p-hackers >>> >> _______________________________________________ >> p2p-hackers mailing list >> [email protected] >> http://lists.zooko.com/mailman/listinfo/p2p-hackers > _______________________________________________ p2p-hackers mailing list [email protected] http://lists.zooko.com/mailman/listinfo/p2p-hackers
