Re: Learning python networking

2014-01-15 Thread William Ray Wing
On Jan 15, 2014, at 7:52 AM, Chris Angelico  wrote:

[megabyte]

> One of the fundamentals of the internet is that connections *will*
> break. A friend of mine introduced me to Magic: The Gathering via a
> program that couldn't handle drop-outs, and it got extremely
> frustrating - we couldn't get a game going. Build your server such
> that your clients can disconnect and reconnect, and you protect
> yourself against half the problem; allow them to connect and kick the
> other connection off, and you solve the other half. (Sometimes, the
> server won't know that the client has gone, so it helps to be able to
> kick like that.) It might not be an issue when you're playing around
> with localhost, and you could even get away with it on a LAN, but on
> the internet, it's so much more friendly to your users to let them
> connect multiple times like that.

But note VERY carefully that this can open HUGE security holes if not done with 
extreme care.

Leaving a dangling connection (not session, TCP closes sessions) open is an 
invitation so bad things happening.

-Bill
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Learning python networking

2014-01-15 Thread Chris Angelico
On Thu, Jan 16, 2014 at 3:43 AM, William Ray Wing  wrote:
> I was assuming another user picking up the connection using sniffed 
> credentials (and yes, despite all the work on ssh, not all man-in-the-middle 
> attacks have been killed).

If that can happen, then I would much prefer that it kick my
connection off - at least that way, I have some chance of knowing it's
happened. But I suspect that this sort of thing is way WAY out of the
league of the OP's stated problem :)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Learning python networking

2014-01-15 Thread William Ray Wing
On Jan 15, 2014, at 11:31 AM, Chris Angelico  wrote:

> On Thu, Jan 16, 2014 at 3:25 AM, William Ray Wing  wrote:
>> On Jan 15, 2014, at 7:52 AM, Chris Angelico  wrote:
>>> One of the fundamentals of the internet is that connections *will*
>>> break. A friend of mine introduced me to Magic: The Gathering via a
>>> program that couldn't handle drop-outs, and it got extremely
>>> frustrating - we couldn't get a game going. Build your server such
>>> that your clients can disconnect and reconnect, and you protect
>>> yourself against half the problem; allow them to connect and kick the
>>> other connection off, and you solve the other half.
>> 
>> But note VERY carefully that this can open HUGE security holes if not done 
>> with extreme care.
>> 
>> Leaving a dangling connection (not session, TCP closes sessions) open is an 
>> invitation so bad things happening.
> 
> Not sure what you mean here. I'm assuming an authentication system
> that stipulates one single active connection per authenticated user
> (if you reauthenticate with the same credentials, it'll disconnect the
> other one on the presumption that the connection's been lost). In
> terms of resource wastage, there's no difference between disconnecting
> now and letting it time out, and waiting the ten minutes (or whatever)
> and then terminating cleanly. Or do you mean another user gaining
> access? It's still governed by the same authentication.
> 

I was assuming another user picking up the connection using sniffed credentials 
(and yes, despite all the work on ssh, not all man-in-the-middle attacks have 
been killed).

-Bill

> ChrisA
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Learning python networking

2014-01-15 Thread Chris Angelico
On Thu, Jan 16, 2014 at 3:31 AM, Chris Angelico  wrote:
> I'm assuming an authentication system
> that stipulates one single active connection per authenticated user

Incidentally, in an environment where everything's trusted (LAN or
localhost), the "authentication system" can be as simple as "type a
user name". I've done systems like that; first line entered becomes
the handle or key, and it does the same kick-off system on duplicate.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Learning python networking

2014-01-15 Thread Chris Angelico
On Thu, Jan 16, 2014 at 3:25 AM, William Ray Wing  wrote:
> On Jan 15, 2014, at 7:52 AM, Chris Angelico  wrote:
>> One of the fundamentals of the internet is that connections *will*
>> break. A friend of mine introduced me to Magic: The Gathering via a
>> program that couldn't handle drop-outs, and it got extremely
>> frustrating - we couldn't get a game going. Build your server such
>> that your clients can disconnect and reconnect, and you protect
>> yourself against half the problem; allow them to connect and kick the
>> other connection off, and you solve the other half.
>
> But note VERY carefully that this can open HUGE security holes if not done 
> with extreme care.
>
> Leaving a dangling connection (not session, TCP closes sessions) open is an 
> invitation so bad things happening.

Not sure what you mean here. I'm assuming an authentication system
that stipulates one single active connection per authenticated user
(if you reauthenticate with the same credentials, it'll disconnect the
other one on the presumption that the connection's been lost). In
terms of resource wastage, there's no difference between disconnecting
now and letting it time out, and waiting the ten minutes (or whatever)
and then terminating cleanly. Or do you mean another user gaining
access? It's still governed by the same authentication.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Learning python networking

2014-01-15 Thread Chris Angelico
On Wed, Jan 15, 2014 at 11:52 PM, Chris Angelico  wrote:
> One of the fundamentals of the internet is that connections *will*
> break. A friend of mine introduced me to Magic: The Gathering via a
> program that couldn't handle drop-outs, and it got extremely
> frustrating - we couldn't get a game going. Build your server such
> that your clients can disconnect and reconnect, and you protect
> yourself against half the problem; allow them to connect and kick the
> other connection off, and you solve the other half.

Case in point, and a very annoying one: Phone queues do NOT handle
drop-outs. There's no way to reconnect to the queue and resume your
place, you have to start over from the back of the queue. I'm
currently on hold to my ISP because of an outage, and the cordless
phone ran out of battery 27 minutes into an estimated 30-minute wait
time. (Though I suspect it'd be a lot longer than 30 minutes. Those
wait times are notoriously inaccurate.) So now I'm waiting, AGAIN, and
those previous 27 minutes of sitting around with their on-hold music
playing through speakerphone were of no value whatsoever. I can't
transfer to a different handset or connection, I have to just hope
that this one will get through.

With TCP-based servers, it's easy to do better than that - all you
have to do is separate the connection state from the actual socket,
and hang onto a "connection" for some period of time after its socket
disconnects (say, 10-15 minutes). Your users will thank you!

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Learning python networking

2014-01-15 Thread Chris Angelico
On Thu, Jan 16, 2014 at 12:31 AM, Frank Millman  wrote:
> I think you may have omitted a line there -
>
> def gets():
> while '\n' not in buffer:
> data = sock.recv(1024)
> if not data:
> # Client is disconnected, handle it gracefully
> return None # or some other sentinel
> #-->
> buffer = buffer + data
> #-->
> line, buffer = buffer.split(b'\n',1)
> return line.decode().replace('\r', '')

Yes, indeed I did, thanks. Apart from using augmented assignment,
that's exactly what I would have put there, if I'd actually taken a
moment to test the code.

> Also, as I am looking at it, I notice that the second line should say -
>
> while b'\n' not in buffer:

Right again. Fortunately, Py3 would catch that one with a TypeError.
See? This is why you should use Py3. :)

> I feel a bit guilty nitpicking, as you have provided a wonderfully
> comprehensive answer, but I wanted to make sure the OP did not get confused.

No no, nitpicking is exactly what ensures that the end result is
correct. If I got offended at you correcting my code, it would imply
that I think myself perfect (or at least, that I consider you to be
utterly incapable of noticing my errors), which is provably false :)
One of the mind-set changes that I had to introduce at work was that
people don't own code, the repository does - if you see an improvement
to something I wrote, or I see an improvement to something you wrote,
they're improvements to be committed, not turf wars to be battled
over.

Especially on something like this, please *do* catch other people's mistakes :)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Learning python networking

2014-01-15 Thread Frank Millman

"Chris Angelico"  wrote in message 
news:CAPTjJmpb6yr-VpWypbJQn0a=pnjvnv2cchvbzak+v_5josq...@mail.gmail.com...
> You just run a loop like this:
>
> buffer = b''
>
> def gets():
>while '\n' not in buffer:
>data = sock.recv(1024)
>if not data:
># Client is disconnected, handle it gracefully
>return None # or some other sentinel
>line, buffer = buffer.split(b'\n',1)
>return line.decode().replace('\r', '')
>

I think you may have omitted a line there -

def gets():
while '\n' not in buffer:
data = sock.recv(1024)
if not data:
# Client is disconnected, handle it gracefully
return None # or some other sentinel
#-->
buffer = buffer + data
#-->
line, buffer = buffer.split(b'\n',1)
return line.decode().replace('\r', '')

Also, as I am looking at it, I notice that the second line should say -

while b'\n' not in buffer:

I feel a bit guilty nitpicking, as you have provided a wonderfully 
comprehensive answer, but I wanted to make sure the OP did not get confused.

Frank Millman



-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Learning python networking

2014-01-15 Thread Chris Angelico
On Wed, Jan 15, 2014 at 9:37 PM, Paul Pittlerson  wrote:
> I'm sorry if this is a bit late of a response, but here goes.
>
> Big thanks to Chris Angelico for his comprehensive reply, and yes, I do have 
> some questions!

Best way to learn! And the thread's not even a week old, this isn't
late. Sometimes there've been responses posted to something from
2002... now THAT is thread necromancy!!

>> On Thursday, January 9, 2014 1:29:03 AM UTC+2, Chris Angelico wrote:
>> Those sorts of frameworks would be helpful if you need to scale to
>> infinity, but threads work fine when it's small.
> That's what I thought, but I was just asking if it would be like trivially 
> easy to set up this stuff in some of those frameworks. I'm sticking to 
> threads for now because the learning curve of twisted seems too steep to be 
> worth it at the moment.

I really don't know, never used the frameworks. But threads are fairly
easy to get your head around. Let's stick with them.

>> Absolutely! The thing to look at is MUDs and chat servers. Ultimately,
>> a multiplayer game is really just a chat room with a really fancy
>> front end.
> If you know of any open source projects or just instructional code of this 
> nature in general I'll be interested to take a look. For example, you 
> mentioned you had some similar projects of your own..?
>

Here's something that I did up as a MUD-writing tutorial for Pike:

http://rosuav.com/piketut.zip

I may need to port that tutorial to Python at some point. In any case,
it walks you through the basics. (Up to section 4, everything's the
same, just different syntax for the different languages. Section 5 is
Pike-specific.)

>> The server shouldn't require interaction at all. It should accept any
>> number of clients (rather than getting the exact number that you
>> enter), and drop them off the list when they're not there. That's a
>> bit of extra effort but it's hugely beneficial.
> I get what you are saying, but I should mention that I'm just making a 2 
> player strategy game at this point, which makes sense of the limited number 
> of connections.

One of the fundamentals of the internet is that connections *will*
break. A friend of mine introduced me to Magic: The Gathering via a
program that couldn't handle drop-outs, and it got extremely
frustrating - we couldn't get a game going. Build your server such
that your clients can disconnect and reconnect, and you protect
yourself against half the problem; allow them to connect and kick the
other connection off, and you solve the other half. (Sometimes, the
server won't know that the client has gone, so it helps to be able to
kick like that.) It might not be an issue when you're playing around
with localhost, and you could even get away with it on a LAN, but on
the internet, it's so much more friendly to your users to let them
connect multiple times like that.

>> One extremely critical point about your protocol. TCP is a stream -
>> you don't have message boundaries. You can't depend on one send()
>> becoming one recv() at the other end. It might happen to work when you
>> do one thing at a time on localhost, but it won't be reliable on the
>> internet or when there's more traffic. So you'll need to delimit
>> messages; I recommend you use one of two classic ways: either prefix
>> it with a length (so you know how many more bytes to receive), or
>> terminate it with a newline (which depends on there not being a
>> newline in the text).
> I don't understand. Can you show some examples of how to do this?

Denis gave a decent explanation of the problem, with a few
suggestions. One of the easiest to work with (and trust me, you will
LOVE the ease of debugging this kind of system) is the line-based
connection. You just run a loop like this:

buffer = b''

def gets():
while '\n' not in buffer:
data = sock.recv(1024)
if not data:
# Client is disconnected, handle it gracefully
return None # or some other sentinel
line, buffer = buffer.split(b'\n',1)
return line.decode().replace('\r', '')

You could put this into a class definition that wraps up all the
details. The key here is that you read as much as you can, buffering
it, and as soon as you have a newline, you return that. This works
beautifully with the basic TELNET client, so it's easy to see what's
going on. Its only requirement is that there be no newlines *inside*
commands. The classic MUD structure guarantees that (if you want a
paragraph of text, you have some marker that says "end of paragraph" -
commonly a dot on a line of its own, which is borrowed from SMTP), and
if you use json.dumps() then it'll use two characters "\\" and "n" to
represent a newline, so that's safe too.

The next easiest structure to work with is length-preceded, which
Denis explained. Again, you read until you have a full packet, but
instead of "while '\n' not in buffer", it would be "while
len(buffer)> Another rather important point, in two halves. You're writing 

Re: Learning python networking

2014-01-15 Thread Denis McMahon
On Wed, 15 Jan 2014 02:37:05 -0800, Paul Pittlerson wrote:

>> One extremely critical point about your protocol. TCP is a stream - you
>> don't have message boundaries. You can't depend on one send() becoming
>> one recv() at the other end. It might happen to work when you do one
>> thing at a time on localhost, but it won't be reliable on the internet
>> or when there's more traffic. So you'll need to delimit messages; I
>> recommend you use one of two classic ways: either prefix it with a
>> length (so you know how many more bytes to receive), or terminate it
>> with a newline (which depends on there not being a newline in the
>> text).

> I don't understand. Can you show some examples of how to do this?

How much do you understand about tcp/ip networking? because when trying 
to build something on top of tcp/ip, it's a good idea to understand the 
basics of tcp/ip first.

A tcp/ip connection is just a pipe that you pour data (octets, more or 
less analagous to bytes or characters) into at one end, and it comes out 
at the other end.

For your stream of octets (bytes / characters) to have any meaning to a 
higher level program, then the applications using the pipe at both ends 
have to understand that a message has some structure.

The message structure might be to send an n character message length 
count (where in a simple protocol n would have to be a fixed number) 
followed by the specified number of characters.

Assuming your maximum message length is  characters:

You could send the characters  followed by  characters of message 
content.

The receiving end would receive 4 characters, convert them to the number 
, and assume the next  characters will be the message. Then it 
expects another 4 character number.

You could send json encoded strings, in which case each message might 
start with the "{" character and end with the "}" character, but you 
would have to allow for the fact that "}" can also occur within a json 
encoded string.

You might decide that each message is simply going to end with a specific 
character or character sequence.

Whatever you choose, you need some way for the receiving application to 
distinguish between individual messages in the stream of octets / bytes / 
characters that is coming out of the pipe.

-- 
Denis McMahon, denismfmcma...@gmail.com
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Learning python networking

2014-01-15 Thread Paul Pittlerson
I'm sorry if this is a bit late of a response, but here goes.

Big thanks to Chris Angelico for his comprehensive reply, and yes, I do have 
some questions!


> On Thursday, January 9, 2014 1:29:03 AM UTC+2, Chris Angelico wrote:
> Those sorts of frameworks would be helpful if you need to scale to
> infinity, but threads work fine when it's small.
That's what I thought, but I was just asking if it would be like trivially easy 
to set up this stuff in some of those frameworks. I'm sticking to threads for 
now because the learning curve of twisted seems too steep to be worth it at the 
moment.

> Absolutely! The thing to look at is MUDs and chat servers. Ultimately,
> a multiplayer game is really just a chat room with a really fancy
> front end.
If you know of any open source projects or just instructional code of this 
nature in general I'll be interested to take a look. For example, you mentioned 
you had some similar projects of your own..?


> The server shouldn't require interaction at all. It should accept any
> number of clients (rather than getting the exact number that you
> enter), and drop them off the list when they're not there. That's a
> bit of extra effort but it's hugely beneficial.
I get what you are saying, but I should mention that I'm just making a 2 player 
strategy game at this point, which makes sense of the limited number of 
connections.

> One extremely critical point about your protocol. TCP is a stream -
> you don't have message boundaries. You can't depend on one send()
> becoming one recv() at the other end. It might happen to work when you
> do one thing at a time on localhost, but it won't be reliable on the 
> internet or when there's more traffic. So you'll need to delimit
> messages; I recommend you use one of two classic ways: either prefix
> it with a length (so you know how many more bytes to receive), or
> terminate it with a newline (which depends on there not being a
> newline in the text).
I don't understand. Can you show some examples of how to do this?

> Another rather important point, in two halves. You're writing this for
> Python 2, and you're writing with no Unicode handling. I strongly
> recommend that you switch to Python 3 and support full Unicode.
Good point, however the framework I'm using for graphics does not currently 
support python3. I could make the server scripts be in python3, but I don't  
think the potential confusion is worth it until the whole thing can be in the 
same version.


> Note, by the way, that it's helpful to distinguish "data" and "text",
> even in pseudo-code. It's impossible to send text across a socket -
> you have to send bytes of data. If you keep this distinction clearly
> in your head, you'll have no problem knowing when to encode and when
> to decode. For what you're doing here, for instance, I would packetize
> the bytes and then decode into text, and on sending, I'd encode text
> (UTF-8 would be hands-down best here) and then packetize. There are
> other options but that's how I'd do it.
I'm not sure what you are talking about here. Would you care to elaborate on 
this please (it interests and confuses) ?


I'm posting this on google groups, so I hope the formatting turns out ok :P 
thanks.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Learning python networking

2014-01-09 Thread Alister
On Wed, 08 Jan 2014 19:49:40 -0800, Dan Stromberg wrote:

> 
> The third quote, from Brian Kernighan, seems to underestimate the
> complexity of asynchronous programming in the large - it's probably not
> just twice as hard.

Perhaps it should be rephrased as "at least twice as hard"

It really does pay to follow the KISS (Keep It Simple Stupid) philosophy.






-- 
Save energy:  Drive a smaller shell.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Learning python networking

2014-01-08 Thread Chris Angelico
On Thu, Jan 9, 2014 at 2:49 PM, Dan Stromberg  wrote:
> The third quote, from Brian Kernighan, seems to underestimate the
> complexity of asynchronous programming in the large - it's probably
> not just twice as hard.

Yeah, which is why I recommended a threaded approach to the OP. It
won't scale to millions of simultaneous connections... but he's
unlikely to have more than half a dozen in testing, and even in
production, most games aren't going to run more than a couple hundred
players. (A huge server might have more players than that
simultaneously logged in, but they'll be running different games. If
thread count becomes a problem, fork() to create a game, job done.)
Threads are easier to get your head around: this and that happen at
the same time. Sometimes it means creating lots of threads to do
similar things (one for the GUI, one for this socket, one for that
socket, one for the cron handler, etc, etc), but is that really a
problem? Probably not.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Learning python networking

2014-01-08 Thread Dan Stromberg
On Wed, Jan 8, 2014 at 4:07 PM, Chris Angelico  wrote:
> Maybe it's not the best way to do things, but it can be extremely
> simple in the code.

For small projects, the added complexity doesn't bite you.  At least, not much.

For large projects, with thousands or millions of callbacks, it can be
very difficult to track down bugs in who-knows-which callback, given
that they're all being called asynchronously.  It's vastly simpler to
fire up a debugger against a process, or insert print statements that
are related to each other in time in some way.

> It's
> really easy once you get your head around it.

IMO, it's not a matter of wrapping your head around it, but respecting
complexity.

Three quotes come to mind:
* Make things as simple as possible, but not simpler.
* Fools ignore complexity. Pragmatists suffer it. Some can avoid it.
Geniuses remove it.
* Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are, by
definition, not smart enough to debug it.

I'm not trying to call you or anyone else a fool, but the Alan Perlis
quote (the second) does give a flavor of what I'm trying to say.

The third quote, from Brian Kernighan, seems to underestimate the
complexity of asynchronous programming in the large - it's probably
not just twice as hard.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Learning python networking

2014-01-08 Thread Chris Angelico
On Thu, Jan 9, 2014 at 10:53 AM, Dan Stromberg  wrote:
>> Using Python 3.4 (which isn't yet
>> stable, but you can download betas) also gives you an asyncio module,
>> but I'd leave that aside for the moment; first figure out threading,
>> it's likely to be easier.
>
> Personally, I don't like asynchronous I/O, EG twisted.  It tends to
> give very complex, uniprocessor solutions to problems, even when those
> problems have quite simple, alternative solutions.  I'd rather just
> accept and fork in most cases, with or without multiprocessing.

I haven't used the Python asyncio module yet (haven't gotten around to
it, no other reason), but in my Pike work, I've gotten quite friendly
with a callback system. It plays nicely with the structure I have for
code reloading, by separating code and data, and then simply running
everything off a single thread, one callback at a time. Works really
well when everything's waiting anyway. It does force you to think
about things differently, though. Case in point: My MUD client allows
an arbitrary number of code files to inspect a line of text that the
user's entered, before it goes to the server. Any one of them can
suppress the line, in which case subsequent hooks won't see it and it
won't go to the server. What about changing the line as it goes
through? That's easy enough, I guess. But what if that change involves
a popup message and user response... or a network request to a remote
server? (I do, in fact, have exactly that. Long URLs get shortened via
tinyurl.com.) A threaded model wouldn't even help much, here, unless
I'm prepared to spawn a separate thread for every command the user
enters, which is stupid overhead. Instead, I have a function that
reinjects the line and carries on as if it hadn't been suppressed in
the first place.

Maybe it's not the best way to do things, but it can be extremely
simple in the code. Most operations don't even need the
continuation-call mechanism that I used there; mostly, everything's
just (conceptually) "when this happens, call this function". The fact
that they're serialized doesn't matter. Events from the GUI, incoming
socket data (on any socket - there might be multiple), time-delay
events (eg a ticking clock), etc - everything works the same way. It's
really easy once you get your head around it.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Learning python networking

2014-01-08 Thread Dan Stromberg
Nice response Chris.  Seriously.

On Wed, Jan 8, 2014 at 3:29 PM, Chris Angelico  wrote:
> One extremely critical point about your protocol. TCP is a stream -
> you don't have message boundaries. You can't depend on one send()
> becoming one recv() at the other end. It might happen to work when you
> do one thing at a time on localhost, but it won't be reliable on the
> internet or when there's more traffic. So you'll need to delimit
> messages; I recommend you use one of two classic ways: either prefix
> it with a length (so you know how many more bytes to receive), or
> terminate it with a newline (which depends on there not being a
> newline in the text).

Completely agree, and I'll point out
http://stromberg.dnsalias.org/~dstromberg/bufsock.html , which makes
it a little easier to deal with delimiters like newlines or null
termination.

> Another rather important point, in two halves. You're writing this for
> Python 2, and you're writing with no Unicode handling. I strongly
> recommend that you switch to Python 3 and support full Unicode.

Agreed.  It's time modernize.  Don't leave out people on the other
side of the world for no reason other than a modicum of convenience
for the developer.

> Using Python 3.4 (which isn't yet
> stable, but you can download betas) also gives you an asyncio module,
> but I'd leave that aside for the moment; first figure out threading,
> it's likely to be easier.

Personally, I don't like asynchronous I/O, EG twisted.  It tends to
give very complex, uniprocessor solutions to problems, even when those
problems have quite simple, alternative solutions.  I'd rather just
accept and fork in most cases, with or without multiprocessing.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Learning python networking

2014-01-08 Thread Chris Angelico
On Thu, Jan 9, 2014 at 9:27 AM, Paul Pittlerson  wrote:
> I'm trying to learn about socket, how to create and handle connections in 
> python.

Awesome! I *love* socket networking. (Really. It's awesome. I've
written a couple of MUD servers and quite a few MUD clients.)

> This is the best I could come up with on my own, please take a look and give 
> me critique:
> Server script:
> http://pastebin.com/KtapYfM0
>
> Client script:
> http://pastebin.com/t4dYygmX

On this list we prefer in-line code as part of the post, but these are
a little long. Posting like this creates a dependency on that web
site, so a lot of people either can't or won't see your code.

> How to run it:
> I open 3 terminals, in one I start the server script, and enter into it 
> something like 'accept 2'
>
> Then in the two other terminals I start client scripts which will "connect to 
> the server" so to speak.
>
> Now I can communicate between them by sending messages, and exit the whole 
> operation by typing 'exit' into the server.

>From what I'm seeing in that code, all communication is one-way,
right? The server sends to the clients, nothing comes back?

> Is the code overly complicated? More precisely: is there a more elegant and 
> simple way to achieve the same thing? Some people have mentioned things like 
> twisted and asyncore, but I don't know anything about them. If it turns out 
> this kind of concept is very straight forward to set up in either of those I 
> would be interested in sample code.

Those sorts of frameworks would be helpful if you need to scale to
infinity, but threads work fine when it's small.

> I'm specifically looking into this kind of communication because I want to 
> make a small multiplayer game.

Absolutely! The thing to look at is MUDs and chat servers. Ultimately,
a multiplayer game is really just a chat room with a really fancy
front end.

So, some comments on your code.

The server shouldn't require interaction at all. It should accept any
number of clients (rather than getting the exact number that you
enter), and drop them off the list when they're not there. That's a
bit of extra effort but it's hugely beneficial.

One extremely critical point about your protocol. TCP is a stream -
you don't have message boundaries. You can't depend on one send()
becoming one recv() at the other end. It might happen to work when you
do one thing at a time on localhost, but it won't be reliable on the
internet or when there's more traffic. So you'll need to delimit
messages; I recommend you use one of two classic ways: either prefix
it with a length (so you know how many more bytes to receive), or
terminate it with a newline (which depends on there not being a
newline in the text).

Another rather important point, in two halves. You're writing this for
Python 2, and you're writing with no Unicode handling. I strongly
recommend that you switch to Python 3 and support full Unicode. Your
current code might work fine so long as everyone uses the same
codepage, but then you'll meet someone from Greece or Russia and
they'll be typing in gibberish because you're decoding it wrongly.
(You can get that sort of thing even without crossing country borders,
but then it's tempting to just say "Don't use funny characters"; but
the problem isn't funny characters, of which there aren't any (I
swear, Spike Milligan is *so* not funny... okay, that's not true), but
of encodings and character sets.) Using Python 3.4 (which isn't yet
stable, but you can download betas) also gives you an asyncio module,
but I'd leave that aside for the moment; first figure out threading,
it's likely to be easier.

So here's how I'd structure a program like this.

# Server
bind socket to port and listen on it
while True:
accept socket
spawn thread

thread:
register self with list of connected clients
while socket connected:
receive data
try to parse a message out of data - if not, buffer it
handle message, eg by sending to all connected clients
unregister self from connected client list


# Client
connect to server
start thread
while True:
accept input from user
act on input, which might 'break'
shut down socket cleanly

thread:
receive data from socket
try to parse a message out of data, as with the server
# you might even be able to use the same code for both
handle message, eg by displaying to screen



That's a threaded system. It emphasizes that the server is doing one
"thing" for each connected client (listening for incoming socket
data), plus one more "thing" (listening for new clients), and the
client is doing two "things" (listening for commands from the human,
and listening for messages from the server). Moving to an async I/O
system doesn't change that, it just runs it all on one thread. So do
whichever's easier to get your head around :)

Note, by the way, that it's helpful to distinguish "data" and "text",
even in pseudo-code. It's impossible to send text across a so

Learning python networking

2014-01-08 Thread Paul Pittlerson
I'm trying to learn about socket, how to create and handle connections in 
python.

This is the best I could come up with on my own, please take a look and give me 
critique:
Server script:
http://pastebin.com/KtapYfM0

Client script:
http://pastebin.com/t4dYygmX

How to run it:
I open 3 terminals, in one I start the server script, and enter into it 
something like 'accept 2'

Then in the two other terminals I start client scripts which will "connect to 
the server" so to speak.

Now I can communicate between them by sending messages, and exit the whole 
operation by typing 'exit' into the server.

Is the code overly complicated? More precisely: is there a more elegant and 
simple way to achieve the same thing? Some people have mentioned things like 
twisted and asyncore, but I don't know anything about them. If it turns out 
this kind of concept is very straight forward to set up in either of those I 
would be interested in sample code.

I'm specifically looking into this kind of communication because I want to make 
a small multiplayer game.
-- 
https://mail.python.org/mailman/listinfo/python-list