Re: How should I handle socket receiving?
On Mar 14, 1:33 pm, MRAB wrote: > On 14/03/2011 19:47, Hans wrote: > > > > > On Mar 12, 10:13 pm, Tim Roberts wrote: > >> Hans wrote: > > >>> I'm thinking to write a code which to: > >>> 1. establish tons of udp/tcp connections to a server > > >> What does "tons" mean? Tens? Hundreds? > > >>> my question is how should I handle receiving traffic from each > >>> connection respectively? > > >> You're really going to want to use "select". You can store the objects in > >> a dictionary where the key is the socket number. That way, you can use the > >> result of the select and get your network object directly. > >> -- > >> Tim Roberts, t...@probo.com > >> Providenza& Boekelheide, Inc. > > > I wrote code like this: > > main proc: > > import socket_thread > > #start 1000 connection > > while i<1000: > > my_socket=socket_thread.socket_thread(i,host,port) > > my_socket.send(some_data) > > my_socket.recv() > > This won't run as-is because you never assign to "i". > > > > > > > socket_thread.py > > class socket_thread: > > def __init__: > > self.soc_handle=socket.socket(socket.IF_INET,socket.DGRAM) > > def send(data): > > self.soc_handle.send(data) > > def recv(): > > while 1: > > > input_list,output_list,exec_list=select.select([self.soc_handle],[],[], > > 2) > > data=input_list[0].recv(2048) > > print data > > > But it does not work as I hope. main proc can only initiate one thread > > and then trapped by it, cannot get out. > > I'm sure I missed something but I don't know. Thanks for any help. > > Your "socket_thread" class is just a normal class. You create an > instance, use it to send data, and then call its "recv" method, which > loops forever.- Hide quoted text - > > - Show quoted text -- Hide quoted text - > > - Show quoted text - Thanks. "i" problem is OK, I can find and fix it easily. I also understand the second problem, but I don't know how to fix it. What's your meaning of "normal class"? Can I specify it as some special class and then it can work as I hope? thanks again. -- http://mail.python.org/mailman/listinfo/python-list
Re: How should I handle socket receiving?
On 14/03/2011 19:47, Hans wrote: On Mar 12, 10:13 pm, Tim Roberts wrote: Hans wrote: I'm thinking to write a code which to: 1. establish tons of udp/tcp connections to a server What does "tons" mean? Tens? Hundreds? my question is how should I handle receiving traffic from each connection respectively? You're really going to want to use "select". You can store the objects in a dictionary where the key is the socket number. That way, you can use the result of the select and get your network object directly. -- Tim Roberts, t...@probo.com Providenza& Boekelheide, Inc. I wrote code like this: main proc: import socket_thread #start 1000 connection while i<1000: my_socket=socket_thread.socket_thread(i,host,port) my_socket.send(some_data) my_socket.recv() This won't run as-is because you never assign to "i". socket_thread.py class socket_thread: def __init__: self.soc_handle=socket.socket(socket.IF_INET,socket.DGRAM) def send(data): self.soc_handle.send(data) def recv(): while 1: input_list,output_list,exec_list=select.select([self.soc_handle],[],[], 2) data=input_list[0].recv(2048) print data But it does not work as I hope. main proc can only initiate one thread and then trapped by it, cannot get out. I'm sure I missed something but I don't know. Thanks for any help. Your "socket_thread" class is just a normal class. You create an instance, use it to send data, and then call its "recv" method, which loops forever. -- http://mail.python.org/mailman/listinfo/python-list
Re: How should I handle socket receiving?
On Mar 12, 10:13 pm, Tim Roberts wrote: > Hans wrote: > > >I'm thinking to write a code which to: > >1. establish tons of udp/tcp connections to a server > > What does "tons" mean? Tens? Hundreds? > > >my question is how should I handle receiving traffic from each > >connection respectively? > > You're really going to want to use "select". You can store the objects in > a dictionary where the key is the socket number. That way, you can use the > result of the select and get your network object directly. > -- > Tim Roberts, t...@probo.com > Providenza & Boekelheide, Inc. I wrote code like this: main proc: import socket_thread #start 1000 connection while i<1000: my_socket=socket_thread.socket_thread(i,host,port) my_socket.send(some_data) my_socket.recv() socket_thread.py class socket_thread: def __init__: self.soc_handle=socket.socket(socket.IF_INET,socket.DGRAM) def send(data): self.soc_handle.send(data) def recv(): while 1: input_list,output_list,exec_list=select.select([self.soc_handle],[],[], 2) data=input_list[0].recv(2048) print data But it does not work as I hope. main proc can only initiate one thread and then trapped by it, cannot get out. I'm sure I missed something but I don't know. Thanks for any help. -- http://mail.python.org/mailman/listinfo/python-list
Re: How should I handle socket receiving?
Hans wrote: > >I'm thinking to write a code which to: >1. establish tons of udp/tcp connections to a server What does "tons" mean? Tens? Hundreds? >my question is how should I handle receiving traffic from each >connection respectively? You're really going to want to use "select". You can store the objects in a dictionary where the key is the socket number. That way, you can use the result of the select and get your network object directly. -- Tim Roberts, t...@probo.com Providenza & Boekelheide, Inc. -- http://mail.python.org/mailman/listinfo/python-list
Re: How should I handle socket receiving?
I'm abs not sure but maybe you'll need to put each client into separate thread; like this def Client_func(s2, cn): while 1: data = cn.recv(4096) if not data: s2.shutdown(1) return s2.sendall(data) cn, addr = s1.accept() s2 = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s2.connect((the_server_host, the_server_port)) thread.start_new_thread(Client_func, (s2, cn,)) -- http://mail.python.org/mailman/listinfo/python-list
Re: How should I handle socket receiving?
On Fri, Mar 11, 2011 at 3:30 PM, Hans wrote: > I'm thinking to write a code which to: > 1. establish tons of udp/tcp connections to a server > 2. send packets from each connections > 3. receive packets from each connections and then do something based > on received content and connection statues. > 4. repeat step 2 and step 3. > > my question is how should I handle receiving traffic from each > connection respectively? two ways I'm thinking are: > 1. setup connections one by one, put socket handler into an > array(dictionary?), array also record each handler's status. then in a > big loop, keep using "select" to check sockets, if any socket get > something received, then find the socket in the array, based on the > socket status to do things should be done. this one is like single- > thread? > This should work, except you perhaps don't need to keep the sockets in an array or find them in that array, because select is going to return a list of the ones that are ready for I/O. > 2. (I don't know if this one works, but I prefer this one if it do > works. ) Setup connection object, write codes in the object to handle > sending/receiving and status. let each object to check/receive its > socket by itslef(I don't know how yet). Then In the main proc, I just > initiate those objects > Each instance of your object(s) could select on just one socket. Start each object doing its thing in parallel. How you do that depends on what kind of concurrency you choose, but there tend to be lots of nice example programs that you can readily locate using google, showing how. CPython's threading is not strong (not sure how much it's improved in 3.2), so for concurrency, you'd probably best use the multiprocessing module or CPython 3.2's "futures" - or an alternative python implementation that threads well, like Jython. Oh, and you could try greenlets. In a way this is simpler than #1, in a way it's not. Supposedly tornado gets lots of great performance -because- it's single threaded. And it makes sense that this would be fast, up to a point, because you eliminate lots of context switching. However, on a system with lots of cores, and perhaps to future-proof your code (most likely individual cores aren't getting much faster anymore, but the number of cores is most likely going up), you're probably better off with something concurrent like your #2. HTH -- http://mail.python.org/mailman/listinfo/python-list
How should I handle socket receiving?
I'm thinking to write a code which to: 1. establish tons of udp/tcp connections to a server 2. send packets from each connections 3. receive packets from each connections and then do something based on received content and connection statues. 4. repeat step 2 and step 3. my question is how should I handle receiving traffic from each connection respectively? two ways I'm thinking are: 1. setup connections one by one, put socket handler into an array(dictionary?), array also record each handler's status. then in a big loop, keep using "select" to check sockets, if any socket get something received, then find the socket in the array, based on the socket status to do things should be done. this one is like single- thread? 2. (I don't know if this one works, but I prefer this one if it do works. ) Setup connection object, write codes in the object to handle sending/receiving and status. let each object to check/receive its socket by itslef(I don't know how yet). Then In the main proc, I just initiate those objects. Any responses would be very welcome, thanks. -- http://mail.python.org/mailman/listinfo/python-list
