Re: [pygame] pygame networking redux
So... To summarise, it seems that people think Twisted is better than a re-implementation, and that running the network loop can should be an optional possibility. It seems people are using PB with some success. Is an RPC-like system such as PB preferable to a simple message/event passing system?
Re: [pygame] pygame networking redux
Simon Wittber kirjoitti: I think that is a matter of one's perspective on what is 'nice'. sure seems like that. Also, I don't really think it is such a big project. I've built this sort of low level system before (using TCP instead of UDP) with higher level stuff to pass dictionaries around in network messages... and ok.. but, they end up being 10k code, and much less intrusive than the twisted reactor. (http://fibranet.googlecode.com/svn/trunk/nanotubes.py) 'intrusive'? as in .. filesize? did not look at that code yet, will certainly check out, thanks for mentioning :) I also want to use UDP, so I have more opportunities to tweak my protocol to fit the way my games / app want to work. I was also curious if there are ways to use Twisted with UDP, I actually thought/assumed there are. Then again, we have not tested yet for the relatively-low-latency networking need we have for a game now, will be interesting to see if TCP cuts it -- if not, there are existing libs that use UDP for these kinds of things too, which we can test, if TCP, and hence some of the existing simple tools, is too slow. I planned to test TCP with Tofu first (the twisted, but not pb?, using thing from the folks who make soya3d (oomadness)), and then perhaps look at Verse and/or VOB from interreality.org .. then again I should first check what game libs there are, do remember that there were some folks use in some SDL games *googles a bit so you dont have to teach me* uh well game networking library gives a lot now. Well, we'll proceed along that plan soon enough - am also interested to here and see how your things work. ~Toni
Re: [pygame] pygame networking redux
Another library worth considering is: http://pyraknet.slowchop.com/ - I haven't used it myself, but I've heard many good things about raknet. It's a networking library written primarily for games. Phil Toni Alatalo [EMAIL PROTECTED] wrote: Simon Wittber kirjoitti: I think that is a matter of one's perspective on what is 'nice'. sure seems like that. Also, I don't really think it is such a big project. I've built this sort of low level system before (using TCP instead of UDP) with higher level stuff to pass dictionaries around in network messages... and ok.. but, they end up being 10k code, and much less intrusive than the twisted reactor. (http://fibranet.googlecode.com/svn/trunk/nanotubes.py) 'intrusive'? as in .. filesize? did not look at that code yet, will certainly check out, thanks for mentioning :) I also want to use UDP, so I have more opportunities to tweak my protocol to fit the way my games / app want to work. I was also curious if there are ways to use Twisted with UDP, I actually thought/assumed there are. Then again, we have not tested yet for the relatively-low-latency networking need we have for a game now, will be interesting to see if TCP cuts it -- if not, there are existing libs that use UDP for these kinds of things too, which we can test, if TCP, and hence some of the existing simple tools, is too slow. I planned to test TCP with Tofu first (the twisted, but not pb?, using thing from the folks who make soya3d (oomadness)), and then perhaps look at Verse and/or VOB from interreality.org .. then again I should first check what game libs there are, do remember that there were some folks use in some SDL games *googles a bit so you dont have to teach me* uh well game networking library gives a lot now. Well, we'll proceed along that plan soon enough - am also interested to here and see how your things work. ~Toni - Looking for a deal? Find great prices on flights and hotels with Yahoo! FareChase.
Re: [pygame] pygame networking redux
hi not sure if directly related to this but I find OSC protocol very interesting to send info over the net, it was developed as a replacement of MIDI, but it is just a network protocol as far as i understand. There are python bindings and I did a layer around the original bindings to simplify the api as much as possible. http://www.ixi-software.net/content/download/simpleosc0.2.3.zip I cannot find a link to the original OSC.py but it is included in the zip above in case you want to work directly with the sockets, etc... enrike Phil Hassey(e)k dio: Another library worth considering is: http://pyraknet.slowchop.com/ - I haven't used it myself, but I've heard many good things about raknet. It's a networking library written primarily for games. ** Phil */Toni Alatalo [EMAIL PROTECTED]/* wrote: Simon Wittber kirjoitti: I think that is a matter of one's perspective on what is 'nice'. sure seems like that. Also, I don't really think it is such a big project. I've built this sort of low level system before (using TCP instead of UDP) with higher level stuff to pass dictionaries around in network messages... and ok.. but, they end up being 10k code, and much less intrusive than the twisted reactor. (http://fibranet.googlecode.com/svn/trunk/nanotubes.py) 'intrusive'? as in .. filesize? did not look at that code yet, will certainly check out, thanks for mentioning :) I also want to use UDP, so I have more opportunities to tweak my protocol to fit the way my games / app want to work. I was also curious if there are ways to use Twisted with UDP, I actually thought/assumed there are. Then again, we have not tested yet for the relatively-low-latency networking need we have for a game now, will be interesting to see if TCP cuts it -- if not, there are existing libs that use UDP for these kinds of things too, which we can test, if TCP, and hence some of the existing simple tools, is too slow. I planned to test TCP with Tofu first (the twisted, but not pb?, using thing from the folks who make soya3d (oomadness)), and then perhaps look at Verse and/or VOB from interreality.org .. then again I should first check what game libs there are, do remember that there were some folks use in some SDL games *googles a bit so you dont have to teach me* uh well game networking library gives a lot now. Well, we'll proceed along that plan soon enough - am also interested to here and see how your things work. ~Toni Looking for a deal? Find great prices on flights and hotels http://us.rd.yahoo.com/evt=47094/*http://farechase.yahoo.com/;_ylc=X3oDMTFicDJoNDllBF9TAzk3NDA3NTg5BHBvcwMxMwRzZWMDZ3JvdXBzBHNsawNlbWFpbC1uY20- with Yahoo! FareChase.
Re: [pygame] pygame networking redux
I was also curious if there are ways to use Twisted with UDP, I actually thought/assumed there are. You can do UDP with twisted, but it's extremely underdeveloped, if you have your heart set on UDP i'd probably suggest pretty much anything else. Nathan
Re: [pygame] pygame networking redux
There's also that P2P protocol in twisted I think. It has a Q in the name, but I can't find it or remember it. It is supposed to be able to handle things like NAT traversal too. On 6/6/07, 12 Dogs [EMAIL PROTECTED] wrote: I was also curious if there are ways to use Twisted with UDP, I actually thought/assumed there are. You can do UDP with twisted, but it's extremely underdeveloped, if you have your heart set on UDP i'd probably suggest pretty much anything else. Nathan
Re: [pygame] pygame networking redux
I'm not sure this is true or not, but it's something I think is the case (untested)... It seems that running twisted in a separate thread would be better for keeping more consistent latency in networking. If you wanted to run your networking at different frequencies to the rest of your system. It seems that some of the time, this would be the case. Much like sound requires updating at different frequencies. Iterating it at eg. 72fps means that the quickest data can go out is at 1/72 messages per second? If you don't have a consistent frame rate, say a frame too 1/14th of a second because of some heavy graphics - then messages would only come out every 1/14th of a second. Whereas if you ran twisted in another thread it would be separate of those concerns - and could run at a different frequency. Maybe a lower frequency even. As a bonus all your networking stuff can be handled by a separate CPU/Core if available. Or am I way off here? Maybe it doesn't matter. On 6/6/07, Simon Wittber [EMAIL PROTECTED] wrote: Richard Jones just mentioned to me off list how he is using twisted. (He is having issues with the mailing list). Using twisted like this is not intrusive, as it lets me keep control of the main loop. Thanks Richard! We've just run into this ourselves, but in our case integrating Twisted into a pyglet application (though not into its event system yet - just into the main loop). The docs are extremely sparse in this area, but the following is mostly working for us: reactor.startRunning(1) (or 0 if you don't want it to take over signals) while main_loop: reactor.iterate(0.01) then when you want to exit your program, use: reactor.stop(); reactor.iterate(0.01)
Re: [pygame] pygame networking redux
Perhaps useful. Seems like premature optimization to me though, and unlikely to gain enough to be worth the pain of dealing with threads. -Jasper René Dudfield wrote: I'm not sure this is true or not, but it's something I think is the case (untested)... It seems that running twisted in a separate thread would be better for keeping more consistent latency in networking. If you wanted to run your networking at different frequencies to the rest of your system. It seems that some of the time, this would be the case. Much like sound requires updating at different frequencies. Iterating it at eg. 72fps means that the quickest data can go out is at 1/72 messages per second? If you don't have a consistent frame rate, say a frame too 1/14th of a second because of some heavy graphics - then messages would only come out every 1/14th of a second. Whereas if you ran twisted in another thread it would be separate of those concerns - and could run at a different frequency. Maybe a lower frequency even. As a bonus all your networking stuff can be handled by a separate CPU/Core if available. Or am I way off here? Maybe it doesn't matter. On 6/6/07, Simon Wittber [EMAIL PROTECTED] wrote: Richard Jones just mentioned to me off list how he is using twisted. (He is having issues with the mailing list). Using twisted like this is not intrusive, as it lets me keep control of the main loop. Thanks Richard! We've just run into this ourselves, but in our case integrating Twisted into a pyglet application (though not into its event system yet - just into the main loop). The docs are extremely sparse in this area, but the following is mostly working for us: reactor.startRunning(1) (or 0 if you don't want it to take over signals) while main_loop: reactor.iterate(0.01) then when you want to exit your program, use: reactor.stop(); reactor.iterate(0.01)
Re: [pygame] pygame networking redux
René Dudfield [EMAIL PROTECTED] said: Iterating it at eg. 72fps means that the quickest data can go out is at 1/72 messages per second? You probably don't mean this - 1/72 messages per second is awfully slow. What I've done in situations like this before is to create an outgoing message queue - as my game loop generates information that needs to go out on the wire, it gets added to the queue. Then, after the game update, I do my network housekeeping, which sends each message from the queue out over the network. I also spend that time to pull all waiting traffic in off the net - could be zero, one, or many messages waiting to be processed inbound and outbound. If you don't have a consistent frame rate, say a frame too 1/14th of a second because of some heavy graphics - then messages would only come out every 1/14th of a second. Yeah, providing a queue permits spiky circumstances to be handled as soon as possible - though you may decide to limit the amount of traffic to move in or out. For example, if you get a LOT of network traffic all at once (a new player entering the game, sending out a lot of initial state information in a short period of time), it's possible that processing all of the waiting network messages in one frame could take too long, causing a visible frame rate drop. So, instead, you might wish to choose a maximum number of messages to process per frame. Whereas if you ran twisted in another thread it would be separate of those concerns - and could run at a different frequency. Maybe a lower frequency even. Yeah, the idea is tempting, but I'm nervous about multithreading, particularly making sure the threads communicate correctly. There are a number of approaches to getting that right (mutexes, lock free algorithms, channels), but that sort of thinking hurts my brain, and I write games to have fun. So I avoid threads unless they're far and away the best choice for the job. As a bonus all your networking stuff can be handled by a separate CPU/Core if available. That would be nice, but with current versions of Python (anything with a Global Interpreter Lock - see, for example http://docs.python.org/api/threads.html ), you're not going to get much concurrency on multicore machines. I'm not sure, maybe there's some benefit to be had with low-level networking I/O being processed independently, but the python code will all be serialized (i.e. not concurrent) because of that GIL. If multithreaded Python networking really excites you, I suggest you take a look at http://www.stackless.com - a modification of the Python interpreter which permits a lot of stuff that I find fascinating, but again, makes my brain hurt, so I've almost never tried to integrate the ideas with game development. I'm not saying threads can't be used to good effect, they just require my brain to stretch in ways it's not comfortable with. If they work for you, that's great. -Dave LeCompte
Re: [pygame] pygame networking redux
I've had a look at the latest Twisted (2.5), and it seems that there is no longer any support for integrating the twisted event loop with pygame. Bah. () Once all this low-level stuff is done, we could add lobby systems, score systems etc which are fairly generic and easily used across different games/apps. I'm not so sure I understand what you want your self-made lowlevel classes to do, but I have been using twisteds perspective broker (sort of RPC) for all my networking needs. I made a NetworkController class that works with PB calls and feeds the network-events into the local event queue not truely into the real pygame-event-queue, but I guess that wouldn't be too tough to extend. I apologize if I totally misunderstood what you are trying to do but working with sockets is bäähh, I'd rather build on twisted for any networking needs. greetings
Re: [pygame] pygame networking redux
ba:a:hh ??? I play on SimonW's bah in german... the : should be on top of the a if you email client supports it :-)
Re: [pygame] pygame networking redux
On 6/5/07, Simon Oberhammer [EMAIL PROTECTED] wrote: I apologize if I totally misunderstood what you are trying to do but working with sockets is bäähh, I'd rather build on twisted for any networking needs. I'm still undecided on Twisted. I haven't done any serious work with it since 1.3. Looking at the current docs however, I can't see how to iterate the pygame event queue at the same time as run the reactor.run() call. How do you process your pygame event queue? -- :: Simon Wittber :: http://www.linkedin.com/in/simonwittber :: phone: +61.4.0135.0685 :: jabber/msn: [EMAIL PROTECTED]
Re: [pygame] pygame networking redux
Don't know if this will be helpful, but a while ago i started writing a lemmings clone using twisted for networking. the file in question is available here: http://thescarydoor.googlecode.com/svn/trunk/default.py i didn't have much problems with the twisted loop and the event code, i just didn't use the event loop. The only other option is to use twisted in a second thread which can get a little messy but not too bad as it's just 1 thread. Nathan On 6/5/07, Simon Wittber [EMAIL PROTECTED] wrote: On 6/5/07, Simon Oberhammer [EMAIL PROTECTED] wrote: I apologize if I totally misunderstood what you are trying to do but working with sockets is bäähh, I'd rather build on twisted for any networking needs. I'm still undecided on Twisted. I haven't done any serious work with it since 1.3. Looking at the current docs however, I can't see how to iterate the pygame event queue at the same time as run the reactor.run() call. How do you process your pygame event queue? -- :: Simon Wittber :: http://www.linkedin.com/in/simonwittber :: phone: +61.4.0135.0685 :: jabber/msn: [EMAIL PROTECTED]
Re: [pygame] pygame networking redux
Simon Wittber wrote: Ah I can see how you solved the problem: def main(self): self.checkEvents(pygame.event.get()) ... snip lots of code ... #come back here at a rate of approx 30fps reactor.callLater(0.03, self.main) Basically you asked twisted to call your function, and when your function finishes, you ask twisted to call it again. I'd prefer not to use twisted this way. It turns what should be a simple, tight inner loop into a function call which has to work its way through twisted's scheduling mechanics for every frame. -Sw. This is essentially what I do, yielding 60 frames/second of 3D and animation without any trouble. I don't think the twisted overhead is as big as you fear, and it's definitely several orders of magnitude nicer than dealing with low level networking protocols. It has been easily fast enough, Perspective Broker saved me from (re)writing tons of code, and I've undoubtedly avoided plenty of nasty networking bugs as well. What would you hope to gain by rolling your own networking code? -Jasper
