Re: urllib2.urlopen timeout
Zdenek Maxa wrote:
> Hi,
>
> I would like to ask how I should set timeout for a call:
>
> f = urllib2.urlopen(url)
>
>
> I know that Python 2.6 offers
> urllib2.urlopen(url[, data][, timeout])
> which would elegantly solved my problem, but I have to stick to Python 2.5.
>
There are three solutions that I know about:
1) Make your own little HTTP library and set timeouts for each of your sockets.
2) The same as the above, except use asynchronous sockets and the the select
module.
3) If you're on a Unix style system, use the signal module (the example is
helpful http://docs.python.org/library/signal.html#example)
Here's some code:
import urllib2
import signal
class TimeoutException(Exception):
"""SIGALARM was sent to the process"""
pass
def raise_timeout(signum, frame):
raise TimeoutException("Timeout!")
signal.signal(signal.SIGALRM, raise_timeout)
try:
signal.alarm(5) # raise alarm in 5 seconds
data = urllib2.urlopen("http://www.google.com";).readlines()
except TimeoutException, ex:
data = None
finally:
signal.alarm(0) # disable alarm
HTH,
Ben
--
http://mail.python.org/mailman/listinfo/python-list
Re: Download the "head" of a large file?
Dennis Lee Bieber wrote: On Mon, 27 Jul 2009 13:38:25 -0700 (PDT), erikcw declaimed the following in gmane.comp.python.general: Something like the Python equivalent of curl http://url.com/file.xml | head -c 2048 Presuming that | is a shell pipe operation, then doesn't that command line use "curl" to download the entire file, and "head" to display just the first 2k? No, the entire file is not downloaded. My understanding of why this is (which could be wrong) is that the output of curl is piped to head, and once head gets the first 2k it closes the pipe. Then, when curl tries to write to the pipe again, it gets sent the SIGPIPE signal at which point it exits. Cheers, Ben -- http://mail.python.org/mailman/listinfo/python-list
Re: Download the "head" of a large file?
erikcw wrote: > ...download just the first few lines of a large (50mb) text file form a > server to save bandwidth. Something like the Python equivalent of curl > http://url.com/file.xml | head -c 2048 If you're OK calling curl and head from within python: from subprocess import Popen, PIPE url = "http://docs.python.org/"; p1 = Popen(["curl", url], stdout = PIPE, stderr = PIPE) p2 = Popen(["head", "-c", "1024"], stdin = p1.stdout, stdout = PIPE) p2.communicate()[0] If you want a pure python approach: import urllib2 url = "http://docs.python.org/"; req = urllib2.Request(url) f = urllib2.urlopen(req) f.read(1024) HTH, Ben -- http://mail.python.org/mailman/listinfo/python-list
Re: Catching control-C
Michael Mossey wrote: > On Jul 6, 2:47 pm, Philip Semanchuk wrote: >> On Jul 6, 2009, at 5:37 PM, Michael Mossey wrote: >> >>> What is required in a python program to make sure it catches a control- >>> c on the command-line? Do some i/o? The OS here is Linux. >> You can use a try/except to catch a KeyboardInterrupt exception, or you >> can trap it using the signal >> module:http://docs.python.org/library/signal.html >> >> You want to trap SIGINT. >> >> HTH Philip > > Thanks to both of you. However, my question is also about whether I need to > be doing i/o or some similar operation for my program to notice in any shape > or form that Control-C has been pressed. You don't need to be doing I/O in order to raise a KeyboardIneterrupt. For example, the following program should use up a lot of your CPU until you hit Ctrl-C. >>> while True: ... pass ... ^CTraceback (most recent call last): File "", line 1, in KeyboardInterrupt > In the past, I've written Python programs that go about their business > ignoring Ctrl-C. Can you be more specific? Can you share the relevant sections of these programs? Were these programs multi-threaded? > But I want to understand better what the "secret" is to responding to a > ctrl-C in any shape or form. Strictly speaking, I don't think you can always respond to a Ctrl-C in any shape or form. Quoting from the signal module: Although Python signal handlers are called asynchronously as far as the Python user is concerned, they can only occur between the "atomic" instructions of the Python interpreter. This means that signals arriving during long calculations implemented purely in C (such as regular expression matches on large bodies of text) may be delayed for an arbitrary amount of time. HTH, Ben -- http://mail.python.org/mailman/listinfo/python-list
Idioms and Anti-Idioms Question
I have a question about the "Using Backslash to Continue Statements" in the howto "Idioms and Anti-Idioms in Python" (http://docs.python.org/howto/doanddont.html#using-backslash-to-continue-statements) It says: "...if the code was: value = foo.bar()['first'][0]*baz.quux(1, 2)[5:9] \ + calculate_number(10, 20)*forbulate(500, 360) then it would just be subtly wrong." What is subtly wrong about this piece of code? I can't see any bugs and can't think of subtle gotchas (e.g. the '\' is removed or the lines become separated, because in both cases an IndentationError would be raised). Cheers, Ben -- http://mail.python.org/mailman/listinfo/python-list
Re: Status of Python threading support (GIL removal)?
Jure Erznožnik wrote: > See here for introduction: > http://groups.google.si/group/comp.lang.python/browse_thread/thread/370f8a1747f0fb91 > > Digging through my problem, I discovered Python isn't exactly thread > safe and to solve the issue, there's this Global Interpreter Lock > (GIL) in place. > Effectively, this causes the interpreter to utilize one core when > threading is not used and .95 of a core when threading is utilized. > > Is there any work in progess on core Python modules that will > permanently resolve this issue? > Is there any other way to work around the issue aside from forking new > processes or using something else? There is a group of people working on an alternative implementation to Python that, among other things, will not have a GIL: http://code.google.com/p/unladen-swallow/ There was even a successful attempt to remove the GIL from CPython, but it caused single threaded python code to be much slower. See more here: http://www.python.org/doc/faq/library/#can-t-we-get-rid-of-the-global-interpreter-lock Cheers, Ben -- http://mail.python.org/mailman/listinfo/python-list
Re: Pythonic way to overwrite a file
Cameron Pulsford wrote: > Hey all, hopefully a simple question. > > I'm writing a simple python tool that opens a file, and does something like > > for line in file.readlines(): > temp.write(line.doStuff()) > > However, I want to provide the option do this "in place", as in have the > destination file be the same as the source file. > Is loading the whole file into memory an option? If so, this should work: filename = "spam.txt" lines = open(filename).readlines()# Read new = [somefunction(line) for line in lines] # Change open(filename, 'w').writelines(new) # Write If you want to stick with your original approach, but want it to work cross platform, then I think this is what you want: http://docs.python.org/library/shutil#shutil.move HTH, Ben -- http://mail.python.org/mailman/listinfo/python-list
Re: Printing dictionary values rather than references
Amit Dor-Shifer wrote:
> Hi all.
>
> I'd like to print-out a dictionary of objects. The printed values are
> references. How Do I print the actual objects.
>
> Thanks,
> Amit
How about this:
class MyClass:
def __str__(self):
return str(self.__dict__)
def __repr__(self):
return str(self.__dict__)
if __name__ == '__main__':
my_dict = dict()
classA = MyClass()
setattr(classA, "attr-1", "val-1")
my_dict['a']= classA
print my_dict
''' Actual output: {'attr-1': 'val-1'}'''
Though I'm guessing someone is going to say that this is not how repr is
supposed be used. See this for more information:
http://docs.python.org/reference/datamodel.html#object.__repr__
Cheers,
Ben
--
http://mail.python.org/mailman/listinfo/python-list
Re: Start the interactive shell within an application
If you're looking to debug your program, try "import pdb" and then wherever you want to debug put: pdb.set_trace() Your program will then enter the debugger when it executes that line. It's quite nice really. If you get confused on what to do, just type "help" http://docs.python.org/library/pdb.html Cheers, Ben Javier Collado wrote: > Take a look either at code.interact or at > IPython.ipapi.launch_new_instance. Basically, the only thing that you > have to provide is a dictionary object that contains the namespace > that you would like to have in your shell once it's launched. > > Best regards, > Javier > > 2009/6/9 eGlyph : >> On Jun 9, 11:49 am, Jean-Michel Pichavant >> wrote: >>> I'm sometimes tired of adding prints to scan the current namespace so >>> I'd like to pause the execution and give the user the shell prompt. >>> This is obviously for debugging purpose. >> This is definitely doable, have look at rhythmbox or gedit - they >> provide an interactive console. >> Also, have a look at IPython, they have a recipe and an example of >> embedding IPython into a user's application. >> -- >> http://mail.python.org/mailman/listinfo/python-list >> -- http://mail.python.org/mailman/listinfo/python-list
