Re: [Tutor] Socket Timeout Handling
Ok, just found your message in the archives. Thanks very much for that! By way of response-- >>That may be because your question ventures into fairly deep areas of >> networking >> that most folk who are just learning Python(ie readers of this list) >> have probably >> not encountered. True enough:-) I am learning this as I go. >>If you do want to get into depth on Python networking you may find the >> A-Press book Python Network Programming useful - I do regularly :-). I came across it a few times, was not sure if it would be as useful as more general books, but I think I will get it soon. >> However the only information I could see about timeouts there was >>related to socket.settimeout() which ISTR you didn't want to/couldn't >>use... The docs show there is settimeout() and setdefaulttimeout(). I will try to explain below why I did not think I could use these. I might have been wrong... > What is the best practice for checking for network connectivity > errors when making network calls? Is it better to wrap the functions > that make said calls in threads and time them? > Or to use timeout variables for modules like socket? >> Personally if i was doingt that I'd almost certainy put it in a thread >> and apply a timeout within the thread. but not having tried that I >> don't >> know how easy it would be! A friend shared an implementation of this with me that works well. I have attached it. Maybe you will find it useful! > I am trying to figure out the optimal way to make socket connections > (INET) and check for timeouts. The socket module has > settimeout(timeout) > and setdefaulttimeout(timeout). However, so far as I can tell, these > apply > to socket objects. The type of socket connection I want to make is > getfqdn(address). >> I don't understand, getfqdn() returns a domain name not an a socket? Yes. Example: >>> import socket >>> socket.getfqdn("64.233.169.99") 'yo-in-f99.google.com' And therein lies the rub! The main function of the script I have this in is to go through a list of IP addresses, and find their FQDN, and other information. It is at this step, when I get the FQDN, that I wanted to do some find of timeout setting. But since it is *not* a socket object, I cannot use settimeout()... > So I can set the default timeout for socket, but not a socket object > (makes sense so far). I cannot use the getfqdn(address) method on > a socket object, I have to use it on socket. >> Sorry it's not making sense to me, getfqdn takes a host name not a >> socket. Exactly my problem:-) >> setdefaulttimout is a function in the socket module not a method of >> socket. >> Thus you woulfd call that before reating a new socket: Thanks for the clarification, I will try this. >> What are you trying to do? Establish a socket connection to something >> or just do a name check that times out more quickly?(or slowly) Just trying to get the FQDNs from a list of IPs, and want to have some sane error handling in place in case my connection dies during the queries. Thanks! class ThreadTimeoutError(Exception): pass from threading import Thread class _ThreadedMethod(Thread): def __init__(self, target, args, kwargs): Thread.__init__(self) self.setDaemon(True) self.target, self.args, self.kwargs = target, args, kwargs self.start() def run(self): try: self.result = self.target(*self.args, **self.kwargs) except Exception, e: self.exception = e except: self.exception = Exception() else: self.exception = None def TimeoutThread(timeout=None): def timeoutthread_proxy(method): if hasattr(method, "__name__"): method_name = method.__name__ else: method_name = 'unknown' def timeoutthread_invocation_proxy(*args, **kwargs): worker = _ThreadedMethod(method, args, kwargs) if timeout is None: return worker worker.join(timeout) if worker.isAlive(): raise ThreadTimeoutError( "A call to %s() has timed out" % method_name) elif worker.exception is not None: raise worker.exception else: return worker.result timeoutthread_invocation_proxy.__name__= method_name return timeoutthread_invocation_proxy return timeoutthread_proxy ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Socket Timeout Handling
>I did send you a response and it is listed on the gmane archive >so if you didn't see it something has gone adrift somewhere. Just searched all my mail, for some reason I did not get this. I will check the archive. Thanks! >The solution you posted seems to bear no resemblence >to the problem you posted? How does this relate to setting >socket timeouts or using getfqdn()? Initially I was asking about how to set socket timeouts. But my general query was just suggestions on how best to detect internet connection upness. Since I "did not" get any responses to the question in the form of socket timeouts in particular, I resent the question in a more general form. It was that form that my last reply was to address. You are right on what it does. I thought at first it would be best to use something native to the classes/functions I was using in the standard library if possible. The solution I ended up with was more general, but serves the purpose. -Sam ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Socket Timeout Handling
"wormwood_3" <[EMAIL PROTECTED]> wrote > Have not gotten any responses on this, I did send you a response and it is listed on the gmane archive so if you didn't see it something has gone adrift somewhere. > But, I did find a decent recipe on ASPN that serves the purpose, The solution you posted seems to bear no resemblence to the problem you posted? How does this relate to setting socket timeouts or using getfqdn()? This solution simply does a ping on a site to see if the network is available. If you had asked how to check if you were connected to the network you likely would have gotten several responses! As ever, if you ask the wrong question you will get the wrong answer! :-) > def checkURL(url): >"""For checking internet connection. Taken from recipe: >http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/101276"""; >try: >p = urlparse(url) >h = HTTP(p[1]) >h.putrequest('HEAD', p[2]) >h.endheaders() >if h.getreply()[0] == 200: return 1 >else: return 0 >except: >return 0 > > The nice thing about this check is that is just looks at the head of > the site, and so is rather fast. > I am trying to figure out the optimal way to make socket connections > (INET) and check for timeouts. The socket module has > settimeout(timeout) > and setdefaulttimeout(timeout). However, so far as I can tell, these > apply to socket objects. The type of socket connection I want to > make is getfqdn(address). So I can set the default timeout for > socket, > but not a socket object (makes sense so far). I cannot use the > getfqdn(address) method on a socket object, I have to use it on > socket. This means (as I understand it thus far), that while I can > set a timeout value for socket objects, this will not apply to when > I use the getfqdn() method, which is where I need a timeout check! -- Alan Gauld Author of the Learn to Program web site http://www.freenetpages.co.uk/hp/alan.gauld ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Socket Timeout Handling
Have not gotten any responses on this, nor very much by way of searching, which is strange and a little disappointing for such a seemingly basic thing. (May just be too obvious of a thing, so no one wanted to post the solution:-). ) But, I did find a decent recipe on ASPN that serves the purpose, so I will share in case others needed to do the same check as I did. Before I needed to make the network call in my program, I have the following: if checkURL('http://www.google.com/'): networkup = True else: networkup = False if networkup: print "Internet connection seems to be up." else: print "Internet connection seems to be down. Please check it", print "and retry." sys.exit() Then I continue with my network calls. The function I use is: def checkURL(url): """For checking internet connection. Taken from recipe: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/101276"""; try: p = urlparse(url) h = HTTP(p[1]) h.putrequest('HEAD', p[2]) h.endheaders() if h.getreply()[0] == 200: return 1 else: return 0 except: return 0 The nice thing about this check is that is just looks at the head of the site, and so is rather fast. One other consideration: While it is a rare day indeed that Google.com is ever down, to be even safer, would could check against several reliable sites, such as Amazon, Yahoo, w3c.org, etc. The status of each check could be put in a list, and if any list item was networkup, then the internet connection may be considered up. Hope someone finds this useful. -Sam ___ - Original Message From: wormwood_3 <[EMAIL PROTECTED]> To: Python Tutorlist Sent: Thursday, September 6, 2007 4:46:08 PM Subject: Re: [Tutor] Socket Timeout Handling Since no one bit on this yet, let me simplify to the core issue I am having: What is the best practice for checking for network connectivity errors when making network calls? Is it better to wrap the functions that make said calls in threads and time them? Or to use timeout variables for modules like socket? Something else? I found some good general info here: http://www.onlamp.com/pub/a/python/2003/11/06/python_nio.html But I have had a hard time finding info on network error handling specifically. Thoughts? __ - Original Message From: wormwood_3 <[EMAIL PROTECTED]> To: Python Tutorlist Sent: Thursday, September 6, 2007 9:40:21 AM Subject: [Tutor] Socket Timeout Handling I am trying to figure out the optimal way to make socket connections (INET) and check for timeouts. The socket module has settimeout(timeout) and setdefaulttimeout(timeout). However, so far as I can tell, these apply to socket objects. The type of socket connection I want to make is getfqdn(address). So I can set the default timeout for socket, but not a socket object (makes sense so far). I cannot use the getfqdn(address) method on a socket object, I have to use it on socket. This means (as I understand it thus far), that while I can set a timeout value for socket objects, this will not apply to when I use the getfqdn() method, which is where I need a timeout check! Some example code for the steps so far: >>> import socket >>> conn = socket.socket() >>> conn.setdefaulttimeout(2.0) Traceback (most recent call last): File "", line 1, in AttributeError: '_socketobject' object has no attribute 'setdefaulttimeout' >>> socket.setdefaulttimeout(2.0) >>> conn.getfqdn("64.33.212.2") Traceback (most recent call last): File "", line 1, in AttributeError: '_socketobject' object has no attribute 'getfqdn' >>> socket.getfqdn("64.33.212.2") '64-33-212-2.customers.pingtone.net' >>> # Disconnected network connection here ... >>> socket.getfqdn("64.33.212.2") '64.33.212.2' >>> # Reconnected network connection here >>> socket.getfqdn("64.33.212.2") '64-33-212-2.customers.pingtone.net' After I disconnected my network connection and called getfqdn(), it returned the IP address I called it with after about 25 seconds. So the default timeout was ignored? Is there some other way to call this function so that I can check for timeouts? Should I instead just put my network calls in a thread and see how long they take, stopping them after a certain period? Thanks for any help. -Sam ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Socket Timeout Handling
"wormwood_3" <[EMAIL PROTECTED]> wrote > Since no one bit on this yet, let me simplify to the core issue I am > having: That may be because your question ventures into fairly deep areas of networking that most folk who are just learning Python(ie readers of this list) have probably not encountered. I've used python for network programming but not extensively and certainly not for hard core production use so can't help you. Similarly I've used sockets extesively from C/C++ but I'm not sure how well that transfers to Python without doing more digging than I hsave time for right now. If you do want to get into depth on Python networking you may find the A-Press book Python Network Programming useful - I do regularly :-). However the only information I could see about timeouts there was related to socket.settimeout() which ISTR you didn't want to/couldn't use... > What is the best practice for checking for network connectivity > errors when making network calls? Is it better to wrap the functions > that make said calls in threads and time them? > Or to use timeout variables for modules like socket? Personally if i was doingt that I'd almost certainy put it in a thread and apply a timeout within the thread. but not having tried that I don't know how easy it would be! > But I have had a hard time finding info on network error handling > specifically. The A-Press book does devote several pages in total to socket error handling including connection errors, timeout errors and transmission errors. > > I am trying to figure out the optimal way to make socket connections > (INET) and check for timeouts. The socket module has > settimeout(timeout) > and setdefaulttimeout(timeout). However, so far as I can tell, these > apply > to socket objects. The type of socket connection I want to make is > getfqdn(address). I don't understand, getfqdn() returns a domain name not an a socket? > So I can set the default timeout for socket, but not a socket object > (makes sense so far). I cannot use the getfqdn(address) method on > a socket object, I have to use it on socket. Sorry it's not making sense to me, getfqdn takes a host name not a socket. import socket conn = socket.socket() conn.setdefaulttimeout(2.0) setdefaulttimout is a function in the socket module not a method of socket. Thus you woulfd call that before reating a new socket: >>> socket.setdefaulttimeout(5) # set default t/o of 5 secs >>> conn = socket.socket()# with timeout of 5 socket.setdefaulttimeout(2.0) conn.getfqdn("64.33.212.2") Again getfqdn is a function in the socket module not a method. But if you give it an IP saddress it will just return that IP address! socket.getfqdn("64.33.212.2") > '64-33-212-2.customers.pingtone.net' OK, Apparently it will give you more... :-) # Disconnected network connection here > ... socket.getfqdn("64.33.212.2") > '64.33.212.2' > After I disconnected my network connection and called getfqdn(), > it returned the IP address I called it with after about 25 seconds. > So the default timeout was ignored? Yes because there was no socket being created it was doing a name lookup using standard DNS etc, and with the network disconnected timed out at the OS level I suspect. If you want to control the DNS lookup you will need to cofde that manually I suspect. (As I say, this is way deeper than I've needed to peer into these type of operations, the defaults have worked for me!) > Is there some other way to call this function so that I can check > for timeouts? Should I instead just put my network calls in a > thread and see how long they take, stopping them after a certain > period? I don't think that would necessarily help here. What are you trying to do? Establish a socket connection to something or just do a name check that times out more quickly?(or slowly) > Thanks for any help. Not sure how much help, not even sure I understand what you are up to, but it might spark an idea... -- Alan Gauld Author of the Learn to Program web site http://www.freenetpages.co.uk/hp/alan.gauld ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Socket Timeout Handling
Since no one bit on this yet, let me simplify to the core issue I am having: What is the best practice for checking for network connectivity errors when making network calls? Is it better to wrap the functions that make said calls in threads and time them? Or to use timeout variables for modules like socket? Something else? I found some good general info here: http://www.onlamp.com/pub/a/python/2003/11/06/python_nio.html But I have had a hard time finding info on network error handling specifically. Thoughts? __ - Original Message From: wormwood_3 <[EMAIL PROTECTED]> To: Python Tutorlist Sent: Thursday, September 6, 2007 9:40:21 AM Subject: [Tutor] Socket Timeout Handling I am trying to figure out the optimal way to make socket connections (INET) and check for timeouts. The socket module has settimeout(timeout) and setdefaulttimeout(timeout). However, so far as I can tell, these apply to socket objects. The type of socket connection I want to make is getfqdn(address). So I can set the default timeout for socket, but not a socket object (makes sense so far). I cannot use the getfqdn(address) method on a socket object, I have to use it on socket. This means (as I understand it thus far), that while I can set a timeout value for socket objects, this will not apply to when I use the getfqdn() method, which is where I need a timeout check! Some example code for the steps so far: >>> import socket >>> conn = socket.socket() >>> conn.setdefaulttimeout(2.0) Traceback (most recent call last): File "", line 1, in AttributeError: '_socketobject' object has no attribute 'setdefaulttimeout' >>> socket.setdefaulttimeout(2.0) >>> conn.getfqdn("64.33.212.2") Traceback (most recent call last): File "", line 1, in AttributeError: '_socketobject' object has no attribute 'getfqdn' >>> socket.getfqdn("64.33.212.2") '64-33-212-2.customers.pingtone.net' >>> # Disconnected network connection here ... >>> socket.getfqdn("64.33.212.2") '64.33.212.2' >>> # Reconnected network connection here >>> socket.getfqdn("64.33.212.2") '64-33-212-2.customers.pingtone.net' After I disconnected my network connection and called getfqdn(), it returned the IP address I called it with after about 25 seconds. So the default timeout was ignored? Is there some other way to call this function so that I can check for timeouts? Should I instead just put my network calls in a thread and see how long they take, stopping them after a certain period? Thanks for any help. -Sam ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor