Hi, Just coming back to the server end of things. Kent, could I please ask you to confirm that I'm not doing anything abnormal with the ThreadingMixIn? The code is here at rafb: http://www.rafb.net/paste/results/915JVm90.html
Basically, when I test it using a script to generate the datagrams via localhost, the server misses anywhere between 200 - 400 datagrams out of 1041. As my appn will be receiving packets over a LAN, I've stuck time.sleep() in the packet generator with a delay of 5ms to simulate LAN latency, and the server copes fine, so no doubt I'm worrying needlessly, but I just want to check that I am using the Thread mix in correctly. Just refined my tests down, at a delay of 2.5ms, all packets are received, at 2ms, 40-50 are missed. How I'm testing will likely never occur in production use, I'm generating a continuous stream of packets via loopback, production will be sporadic packets via network or loopback, but I'm just being paranoid. Much thanks, Liam Clarke On 2/8/06, Kent Johnson <[EMAIL PROTECTED]> wrote: > Liam Clarke wrote: > > On 2/7/06, Kent Johnson <[EMAIL PROTECTED]> wrote: > > > >>Liam Clarke wrote: > >> > >>>Hi all, > >>> > >>>About to embark on my first foray into threading, and rather unsure of > >>>initial approach. I have a basic UDPServer from SocketServer running > >>>using serve_forever(). I'd like to stick this in a thread where it can > >>>endlessly loop waiting for incoming packets, and pushing the received > >>>data into a list where it can be retrieved and operated on by another > >>>thread, until such time as the controlling app terminates it. > >> > >>Threaded servers are supported by the standard library. Instead of using > >>UDPServer use this class: > >> class ThreadingUDPServer(ThreadingMixIn, UDPServer): pass > >> > >>Then your request handler will be called in a separate thread. You don't > >>need to queue the requests yourself, the server will create the thread > >>and dispatch to it when a request is received. > >> > >>Kent > > > > Ah... so then calling serve_forever() won't block? > > It will still block, but your request handler will be called in a new > thread for each request. > > > > My current code, which illustrates what I'm attempting can be found > > here - http://www.rafb.net/paste/results/oIqI3Q11.html > > This is probably fine. You will only be able to handle one request at a > time but the request handling is fast - just read the data and drop it > in the Queue - so that may well be OK. You could also change it to use > ThreadingUDPServer leaving everything else the same - this would allow > multiple simultaneous connections. > > With your design, you will process the requests one at a time in the > data_cruncher and you need only one database connection. > > An alternate design is to use ThreadingUDPServer and have the request > handler do the data crunching and database access. Each handler would > have its own database connection. The advantage of this would be to have > multiple threads of crunching and database access running at one time > which *may* allow faster throughput. > > Yet another alternative would be to have a thread pool pulling requests > from the Queue. This gives you better control over the number of worker > threads and would let you persist the database connection for each > worker. You can find several recipes for thread pools in the online > Python cookbook. > > HTH > Kent > > > > > Basically, the server isn't the main focus, what I'm aiming for looks > > similar to this > > > > server-----Queue--data_cruncher--MySQLconnection > > server---/ / > > server-/ / > > server -/ > > > > So basically having data_cruncher to crunch whatever data there is in > > the queue and send it off to the database is the priority. I'm having > > trouble with SocketServer, need to do some more experimenting, so I'll > > get back to it. > > > > Regards, > > > > Liam Clarke > > > > > > > _______________________________________________ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor > _______________________________________________ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
