Re: Plz help..SocketServer UDP server losing lots of packets

[James Mills]

On Fri, Nov 7, 2008 at 1:43 AM, Ben Sizer <[EMAIL PROTECTED]> wrote:
> On Nov 6, 12:46 am, "James Mills" <[EMAIL PROTECTED]>
> wrote:
>>
>> Try these instead:
>>  * UDPServer 
>> ->http://trac.softcircuit.com.au/circuits/browser/examples/udpserver.py
>>  * UDPClient  
>> ->http://trac.softcircuit.com.au/circuits/browser/examples/udpclient.py
>
> Since there's no contact details on the Circuits site, and the guest
> Trac account doesn't work (it asks the admin to verify the email),
> then I'll post here: it appears that the Hello World example is wrong,
> trying to add an instance of a 'hello' object that doesn't exist. I've
> not tried to run the example, but it certainly confused me when trying
> to work out how Circuits works.

Ben, thanks for your feedback! I've fixed this
now! I've also added come contact details for
myself on the website and fixed the guest
account! Thanks for taking an interest.

--JamesMills

-- 
--
-- "Problems are solved by method"
--
http://mail.python.org/mailman/listinfo/python-list

Re: Plz help..SocketServer UDP server losing lots of packets

[Steve Holden]

I D wrote:
> Hello James,
> Thanks for your response.
> But I cannot use a third party software, I need to use the exisiting
> API's within python.
> As I am new to python, I suspected that I should go by a simpler
> approach and so
> scrapped off the below code and wrote a very simple UDP server code as
> follows:
>  
> logFileName = '/home/msat/gsc/logs/' + compNum + '/logparsertracedump.log'
> logfile = open(logFileName, "w")
> from socket import *
> host = "121.3.0.1 "
> port = PORT
> buf = 4096
> addr = (host,port)
> # Create socket and bind to address
> UDPSock = socket(AF_INET,SOCK_DGRAM)
> UDPSock.setsockopt(SOL_SOCKET, SO_RCVBUF, 8388608)
> UDPSock.bind(addr)
> while 1:
> data,addr = UDPSock.recvfrom(buf)
> if not fileExists(logFileName):
> logfile = open(logFileName, "a")
> logfile.writelines(data)

It would make more sense to leave the file open outside the loop. If
it's important to have each packet individually logged you can call the
file's .flush() method after each .writelines() call. No need to
continually re-open it. I doubt this is the source of your packet loss,
however. Maybe there's a bug in fileExists (for which you could have
used os.path.exists, by the way).

> # Close socket
> UDPSock.close()
>  
How do you anticipate this line will ever be executed?

Your network code looks basically OK, though a little horrible with
things like 8388608 for the socket options. See if you get any ideas from

  http://holdenweb.com/docs/NetProg.pdf

though it doesn't tell you much you don't appear to know already.

regards
 Steve

> Even this seems to lose packets, I would really appreciate if any
> pointers can be provided to improve my code.
> 
> Thanks,
> Sam
>  
> On Wed, Nov 5, 2008 at 7:46 PM, James Mills
> <[EMAIL PROTECTED] > wrote:
> 
> On Thu, Nov 6, 2008 at 9:53 AM,  <[EMAIL PROTECTED]
> > wrote:
> > logFileName = 'log.txt'
> >logfile = open(logFileName, "a")
> >class MyUDPServer(SocketServer.UDPServer):
> >def server_bind(self):
> >self.socket.setsockopt(socket.SOL_SOCKET,
> > socket.SO_RCVBUF, 8388608)
> >self.socket.bind(self.server_address)
> >
> >class LogsDumpHandler(SocketServer.DatagramRequestHandler):
> >def handle(self):
> >global logfile
> > # fileExists is a function which checks the existence of a file
> >if not fileExists(logFileName):
> >logfile = open(logFileName, "a")
> >logfile.writelines(self.rfile.readlines())
> >
> >server = MyUDPServer(("",PORT), LogsDumpHandler)
> >server.serve_forever()
> >logfile.close()
> 
> This is horrible code :/
> 
> Try these instead:
>  * UDPServer ->
> http://trac.softcircuit.com.au/circuits/browser/examples/udpserver.py
>  * UDPClient  ->
> http://trac.softcircuit.com.au/circuits/browser/examples/udpclient.py
> 
> Example usage
> --
> 
> ~/circuits/examples
> $ ./udpserver.py
> 127.0.0.1 , 9000:
> test
> 127.0.0.1 , 9000:
> hi
> 
> ~/circuits/examples
> $ ./udpclient.py -b 127.0.0.1:9000 
> 127.0.0.1:8000 
> test
> 127.0.0.1 , 8000:
> test
> hi
> 127.0.0.1 , 8000:
> hi
> 
> You can download circuits from http://trac.softcircuit.com.au/circuits/
> or get the latest development version by using Mercurial (1):
> hg clone http://hg.softcircuit.com.au/projects/circuits/
> 
> cheers
> James
> 
> [1] http://www.selenic.com/mercurial/wiki/
> 
> --
> --
> -- "Problems are solved by method"
> 
> 
> 
> 
> 
> --
> http://mail.python.org/mailman/listinfo/python-list


-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

--
http://mail.python.org/mailman/listinfo/python-list

Re: Plz help..SocketServer UDP server losing lots of packets

[Ben Sizer]

On Nov 6, 12:46 am, "James Mills" <[EMAIL PROTECTED]>
wrote:
>
> Try these instead:
>  * UDPServer 
> ->http://trac.softcircuit.com.au/circuits/browser/examples/udpserver.py
>  * UDPClient  
> ->http://trac.softcircuit.com.au/circuits/browser/examples/udpclient.py

Since there's no contact details on the Circuits site, and the guest
Trac account doesn't work (it asks the admin to verify the email),
then I'll post here: it appears that the Hello World example is wrong,
trying to add an instance of a 'hello' object that doesn't exist. I've
not tried to run the example, but it certainly confused me when trying
to work out how Circuits works.

--
Ben Sizer
--
http://mail.python.org/mailman/listinfo/python-list

Re: Plz help..SocketServer UDP server losing lots of packets

[I D]

On Thu, Nov 6, 2008 at 10:27 AM, James Mills
<[EMAIL PROTECTED]>wrote:

> On Fri, Nov 7, 2008 at 12:57 AM, I D <[EMAIL PROTECTED]> wrote:
> > Thanks for your response.
> > But I cannot use a third party software, I need to use the exisiting
> API's
> > within python.
>
> Why ?


@Sam@ That is the requirement of the assignment.


>
>
> > Even this seems to lose packets, I would really appreciate if any
> pointers
> > can be provided to improve my code.
>
> You do realize that UDP is the User Datagram
> Protocol and has no Control mechanisms.
> You _cannot_ guarantee that packets will
> arrive in the right order, if at all.


@Sam@ I understand that, but a similar UDP server written in C works
perfect, not even a single packet loss, that leads me to believe that I am
not doing something right in my python code.

>
>
> In all my tests however on local machines
> and across machines in a LAN, I don't
> loose packets. I do recommend you use
> circuits (or twisted).
>
> --JamesMills
>
> --
>  --
> -- "Problems are solved by method"
>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Plz help..SocketServer UDP server losing lots of packets

[James Mills]

On Fri, Nov 7, 2008 at 12:57 AM, I D <[EMAIL PROTECTED]> wrote:
> Thanks for your response.
> But I cannot use a third party software, I need to use the exisiting API's
> within python.

Why ?

> Even this seems to lose packets, I would really appreciate if any pointers
> can be provided to improve my code.

You do realize that UDP is the User Datagram
Protocol and has no Control mechanisms.
You _cannot_ guarantee that packets will
arrive in the right order, if at all.

In all my tests however on local machines
and across machines in a LAN, I don't
loose packets. I do recommend you use
circuits (or twisted).

--JamesMills

-- 
--
-- "Problems are solved by method"
--
http://mail.python.org/mailman/listinfo/python-list

Re: Plz help..SocketServer UDP server losing lots of packets

[I D]

Hello James,
Thanks for your response.
But I cannot use a third party software, I need to use the exisiting API's
within python.
As I am new to python, I suspected that I should go by a simpler approach
and so
scrapped off the below code and wrote a very simple UDP server code as
follows:

logFileName = '/home/msat/gsc/logs/' + compNum + '/logparsertracedump.log'
logfile = open(logFileName, "w")
from socket import *
host = "121.3.0.1"
port = PORT
buf = 4096
addr = (host,port)
# Create socket and bind to address
UDPSock = socket(AF_INET,SOCK_DGRAM)
UDPSock.setsockopt(SOL_SOCKET, SO_RCVBUF, 8388608)
UDPSock.bind(addr)
while 1:
data,addr = UDPSock.recvfrom(buf)
if not fileExists(logFileName):
logfile = open(logFileName, "a")
logfile.writelines(data)
# Close socket
UDPSock.close()

Even this seems to lose packets, I would really appreciate if any pointers
can be provided to improve my code.

Thanks,
Sam

On Wed, Nov 5, 2008 at 7:46 PM, James Mills <[EMAIL PROTECTED]>wrote:

> On Thu, Nov 6, 2008 at 9:53 AM,  <[EMAIL PROTECTED]> wrote:
> > logFileName = 'log.txt'
> >logfile = open(logFileName, "a")
> >class MyUDPServer(SocketServer.UDPServer):
> >def server_bind(self):
> >self.socket.setsockopt(socket.SOL_SOCKET,
> > socket.SO_RCVBUF, 8388608)
> >self.socket.bind(self.server_address)
> >
> >class LogsDumpHandler(SocketServer.DatagramRequestHandler):
> >def handle(self):
> >global logfile
> > # fileExists is a function which checks the existence of a file
> >if not fileExists(logFileName):
> >logfile = open(logFileName, "a")
> >logfile.writelines(self.rfile.readlines())
> >
> >server = MyUDPServer(("",PORT), LogsDumpHandler)
> >server.serve_forever()
> >logfile.close()
>
> This is horrible code :/
>
> Try these instead:
>  * UDPServer ->
> http://trac.softcircuit.com.au/circuits/browser/examples/udpserver.py
>  * UDPClient  ->
> http://trac.softcircuit.com.au/circuits/browser/examples/udpclient.py
>
> Example usage
> --
>
> ~/circuits/examples
> $ ./udpserver.py
> 127.0.0.1, 9000:
> test
> 127.0.0.1, 9000:
> hi
>
> ~/circuits/examples
> $ ./udpclient.py -b 127.0.0.1:9000 127.0.0.1:8000
> test
> 127.0.0.1, 8000:
> test
> hi
> 127.0.0.1, 8000:
> hi
>
> You can download circuits from http://trac.softcircuit.com.au/circuits/
> or get the latest development version by using Mercurial (1):
> hg clone http://hg.softcircuit.com.au/projects/circuits/
>
> cheers
> James
>
> [1] http://www.selenic.com/mercurial/wiki/
>
> --
> --
> -- "Problems are solved by method"
>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Plz help..SocketServer UDP server losing lots of packets

[James Mills]

On Thu, Nov 6, 2008 at 9:53 AM,  <[EMAIL PROTECTED]> wrote:
> logFileName = 'log.txt'
>logfile = open(logFileName, "a")
>class MyUDPServer(SocketServer.UDPServer):
>def server_bind(self):
>self.socket.setsockopt(socket.SOL_SOCKET,
> socket.SO_RCVBUF, 8388608)
>self.socket.bind(self.server_address)
>
>class LogsDumpHandler(SocketServer.DatagramRequestHandler):
>def handle(self):
>global logfile
> # fileExists is a function which checks the existence of a file
>if not fileExists(logFileName):
>logfile = open(logFileName, "a")
>logfile.writelines(self.rfile.readlines())
>
>server = MyUDPServer(("",PORT), LogsDumpHandler)
>server.serve_forever()
>logfile.close()

This is horrible code :/

Try these instead:
 * UDPServer ->
http://trac.softcircuit.com.au/circuits/browser/examples/udpserver.py
 * UDPClient  ->
http://trac.softcircuit.com.au/circuits/browser/examples/udpclient.py

Example usage
--

~/circuits/examples
$ ./udpserver.py
127.0.0.1, 9000:
test
127.0.0.1, 9000:
hi

~/circuits/examples
$ ./udpclient.py -b 127.0.0.1:9000 127.0.0.1:8000
test
127.0.0.1, 8000:
test
hi
127.0.0.1, 8000:
hi

You can download circuits from http://trac.softcircuit.com.au/circuits/
or get the latest development version by using Mercurial (1):
hg clone http://hg.softcircuit.com.au/projects/circuits/

cheers
James

[1] http://www.selenic.com/mercurial/wiki/

-- 
--
-- "Problems are solved by method"
--
http://mail.python.org/mailman/listinfo/python-list

Plz help..SocketServer UDP server losing lots of packets

[id . engg]

logFileName = 'log.txt'
logfile = open(logFileName, "a")
class MyUDPServer(SocketServer.UDPServer):
def server_bind(self):
self.socket.setsockopt(socket.SOL_SOCKET,
socket.SO_RCVBUF, 8388608)
self.socket.bind(self.server_address)

class LogsDumpHandler(SocketServer.DatagramRequestHandler):
def handle(self):
global logfile
# fileExists is a function which checks the existence of a file
if not fileExists(logFileName):
logfile = open(logFileName, "a")
logfile.writelines(self.rfile.readlines())

server = MyUDPServer(("",PORT), LogsDumpHandler)
server.serve_forever()
logfile.close()

The above python code is a UDP server based on SocketServer framework
and I am losing a lot of messages, I have tried almost everything,
increasing the size of the receive buffer, changing the way i read the
messages coming into server.
The above code is supposed to work as a tracing server and it does but
is losing messages.
Is there a way I can improve this code?
Any help will be highly appreciated.

Thanks,
Sam
--
http://mail.python.org/mailman/listinfo/python-list