Re: multiprocessing and dictionaries
> Bjorn Meyer (BM) wrote: >BM> I am trying to convert a piece of code that I am using the thread module >with >BM> to the multiprocessing module. >BM> The way that I have it set up is a chunk of code reads a text file and >assigns >BM> a dictionary key multiple values from the text file. I am using locks to >write >BM> the values to the dictionary. >BM> The way that the values are written is as follows: >BM>mydict.setdefault(key, []).append(value) >BM> The problem that I have run into is that using multiprocessing, the key >gets >BM> set, but the values don't get appended. >BM> I've even tried the Manager().dict() option, but it doesn't seem to work. >BM> Is this not supported at this time or am I missing something? I think you should give more information. Try to make a *minimal* program that shows the problem and include it in your posting or supply a download link. -- Piet van Oostrum URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4] Private email: p...@vanoostrum.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Question about generators
> Cameron Pulsford (CP) wrote: >CP> I read it on the haskell site in their sieves/prime wheel section, >CP> I guess I misunderstood something. (east to do over there...) I did >CP> verify it against established list of primes and other generators >CP> I've written that use more normal methods, but I only hand verified >CP> it. If it is used in a sieve then the non-primes will be sieved out. >CP> It is at least interesting though, I can probably extend it to >CP> check for primality by using a more normal sieve method. It might >CP> be pretty fast too because generally it does only generate primes, >CP> and the few non primes it does generate could be caught quickly >CP> using a scratching out technique. it does only generate primes => it does generate all primes. >CP> When I was initially looking at it there are some interesting >CP> patterns I might be able to extend into a generator that would >CP> yield only correct sets of numbers for the 6x + n pattern. As I wrote in my previous reply, in your use case the non-primes are harmless. But it is useful to reflect that in the naming of your identifiers. *And please, don't top post.* -- Piet van Oostrum URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4] Private email: p...@vanoostrum.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Addind imports to a class namespace
Thank you for your replies. I have factored out the dependency and everything is solved. Cheers, Ryan -- http://mail.python.org/mailman/listinfo/python-list
Re: Infinite loops and synchronization
> Vincent Gulinao (VG) wrote: >VG> lst = list() >VG> (lst populated by async twisted deferred callbacks) >VG> while True: >VG>if len(lst) == SOME_NUMBER: >VG>return lst >VG> Q1: is this a common OK practice? I'm worried infinite loops hogs memory. >VG> Q2: operating on list from threads (mostly appends) must be safe, >VG> right (synchronization)? I am not familiar enough with twisted, but I think the principle is independent from twisted. This loop will not hog memory but it will hog CPU time. You should use a synchronisation construct like threading.Condition or a Semaphore. Here is my suggestion with a Condition: Global somewhere: lst_cond = Condition() In your loop: lst = list() # Why not lst = []? while True: # strange while/if combo if len(lst) == SOME_NUMBER: return lst Make that: with lst_cond: while len(lst) < SOME_NUMBER: lst_cond.wait() return lst In the callback: with lst_cond: lst.append(new_value) lst_cond.notify() In case you don't have a python that supports the with statement (not even `from future') you should use: lst_cond.acquire() try: . finally: lst_cond.release() I think the solution with a semaphore is less elegant. global: sem = Semaphore() loop: for i in range(SOME_NUMBER): sem.acquire() return lst In callback: lst.append(new_value) sem.release() *Be careful: I haven't tested this code (not even syntax checked). So consider it pseudo code.* -- Piet van Oostrum URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4] Private email: p...@vanoostrum.org -- http://mail.python.org/mailman/listinfo/python-list
Re: the ultimate logging facility for debugging code
On Jul 10, 10:39 am, Jean-Michel Pichavant
wrote:
> Greetings,
>
> Sorry for the dubious title :o). I was wandering if there is a standard
> (or reliable) python module that implements the following feature:
>
> http://code.activestate.com/recipes/198078/
> >>>Recipe 198078: Wrapping method calls (meta-class example)
> >>>
> >>>A metaclass is used to wrap all (or just some) methods forlogging
> purposes. The underlying mechanism can be used as well to check pre/post
> conditions, attribute access,... The basic point is, that the actual
> class must not be changed in any way to achive the desired effect.
>
> I had to adapt the code to some of my needs, but now it's almost
> working, by simply defining a metaclass for any of my class, I get the
> class methods calls logged with the instance that called the method, the
> method name, the call line in the file, the parameter **value** passed
> to the method and the returned value.
> It just saved me from hours of ipython interactive debugging with the
> old print way, and to beauty of it: I don't have to write any
> "self._log('Please log me')" line.
>
> As it is so useful, I bet there is a module that is doing this in a more
> reliable way that I did. Anyone has heard of it ?
>
> Jean-Michel
Check out the Python Decorator Library:
http://wiki.python.org/moin/PythonDecoratorLibrary
and you should be able to adapt what you find there to your needs.
Regards,
Vinay Sajip
--
http://mail.python.org/mailman/listinfo/python-list
Re: Infinite loops and synchronization
On Jul 13, 6:06 am, Vincent Gulinao wrote: > lst = list() > > (lst populated by async twisted deferred callbacks) > > while True: > if len(lst) == SOME_NUMBER: > return lst > > Q1: is this a common OK practice? I'm worried infinite loops hogs memory. > Q2: operating on list from threads (mostly appends) must be safe, > right (synchronization)? Q1: I'll answer your question with another. What's so fundamentally different between your infinite loop and this one: while len(lst) != SOME_NUMBER: pass return lst which is not an "infinite loop"[1]. Why would yours be any worse in terms of memory than mine? Are you allocating anything that would hog memory? Of course, like Piet said, it *will* hog your CPU, so you want a time.sleep(.1) in there, at the least. Of course, the question is: why aren't you using a semaphore to let you know you can proceed, and make the provider increment the semaphore? [1] -- well, it can be, if len(lst) == SOME_NUMBER never comes about, and I'd hazard a guess that that's pretty much where your fear of memory hogging comes from: it's easy to allocate stuff and not deallocate it within a cycle, only to find the bounds on that cycle going awry. -- http://mail.python.org/mailman/listinfo/python-list
simple question about Dictionary type containing List objects
Hi group,
I have a dict object like
emails={'mycontacts': [ 'x...@gmail.com, 'y...@gmail.com',
'z...@gmail.com'], 'myname':['gganesh']}
I need to get the lenght of the list mycontacts ,like
mycontacts_numbers=3
help me to solve
Thanks
--
http://mail.python.org/mailman/listinfo/python-list
Re: simple question about Dictionary type containing List objects
On Mon, Jul 13, 2009 at 6:34 PM, gganesh wrote:
> Hi group,
> I have a dict object like
> emails={'mycontacts': [ 'x...@gmail.com, 'y...@gmail.com',
> 'z...@gmail.com'], 'myname':['gganesh']}
> I need to get the lenght of the list mycontacts ,like
> mycontacts_numbers=3
>
>>> len(emails['mycontacts'])
3
HTH,
Ching-Yun "Xavier" Ho, Technical Artist
Contact Information
Mobile: (+61) 04 3335 4748
Skype ID: SpaXe85
Email: cont...@xavierho.com
Website: http://xavierho.com/
--
http://mail.python.org/mailman/listinfo/python-list
Re: The meaning of "=" (Was: tough-to-explain Python)
In message , Aahz wrote:
> In article ,
> Lawrence D'Oliveiro wrote:
>>In message , Aahz wrote:
>>>
>>> It helps to remember that names and namespaces are in many
>>> ways syntactic sugar for dicts or lists.
>>
>>Interesting, though, that Python insists on maintaining a distinction
>>between c["x"] and c.x, whereas JavaScript doesn't bother.
>
> Why do you say "insists"?
>
> class AttrDict:
> def __getitem__(self, key):
> return getattr(self, key)
OK, let's try it:
>>> c = {}
>>> c["x"] = 3
>>> c.x = 4
Traceback (most recent call last):
File "", line 1, in
AttributeError: 'dict' object has no attribute 'x'
>>> class AttrDict:
... def __getitem__(self, key):
... return getattr(self, key)
...
>>> c.x = 4
Traceback (most recent call last):
File "", line 1, in
AttributeError: 'dict' object has no attribute 'x'
Nope, still doesn't work...
--
http://mail.python.org/mailman/listinfo/python-list
Re: Infinite loops and synchronization
In message , Vincent Gulinao wrote: > Q1: is this a common OK practice? I'm worried infinite loops hogs memory. The problem is not that the loop is infinite, but that it busy-waits, hogging CPU. -- http://mail.python.org/mailman/listinfo/python-list
Re: MySQLdb + SSH Tunnel
In message , Emile van Sebille wrote: > ssh with something like... > > ssh -lroot -L3306:C:3306 B > > Watch out for other instances of mysql on A or B... You can use a non-default local port and specify that in your local connection parameters. Similarly you can tell the remote server to use a non-default port in its /etc/my.cnf. -- http://mail.python.org/mailman/listinfo/python-list
Re: Looking for a tool to checkfor python script backward compatibility
In message <4a59951a$0$10853$426a7...@news.free.fr>, Baptiste Lepilleur wrote: > I'm looking for a tool that could be used in a pre-commit step to check > that only features available in a "old" python version are used, say > python 2.3 for example. The only sure way would be to run the script under an appropriately old version of Python. -- http://mail.python.org/mailman/listinfo/python-list
Re: simple question about Dictionary type containing List objects
gganesh wrote:
> I have a dict object like
> emails={'mycontacts': [ 'x...@gmail.com, 'y...@gmail.com',
> 'z...@gmail.com'], 'myname':['gganesh']}
> I need to get the lenght of the list mycontacts ,like
> mycontacts_numbers=3
mycontacts = emails['mycontacts']
mycontacts_number = len(mycontacts)
A list doesn't change in behaviour when it is contained in a dictionary.
Uli
--
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
--
http://mail.python.org/mailman/listinfo/python-list
Re: send keys to focused window
In message , Broken wrote: > I am new to Python, and I'm miserably failing to send specific keys to > (say) notepad. I don't understand why you need to automate a GUI front-end, meant for human use, to a function that can be directly performed without that front-end anyway. -- http://mail.python.org/mailman/listinfo/python-list
Re: Check file is locked?
In message <652cca82-44a3-473f-b640- c2336a9cf...@v15g2000prn.googlegroups.com>, Rajat wrote: > ... my whole idea is to close the wordpad / notepad application so that I > can delete the file and the directory where this file resides. Don't you think the user might have that application open for a reason? -- http://mail.python.org/mailman/listinfo/python-list
Re: A Bug By Any Other Name ...
In message <4a538a71$0$30236$9b4e6...@newsspool1.arcor-online.net>, Stefan Behnel wrote: > Lawrence D'Oliveiro wrote: >> >> I wonder how many people have been tripped up by the fact that >> >> ++n >> >> and >> >> --n >> >> fail silently for numeric-valued n. > > I doubt that there are many. Plus, you misspelled them from the more > obvious > > n++ > > and > > n-- > > which *do* fail with a SyntaxError. Funny, you accuse me of "misspelling", when it's _your_ version that fails with errors! -- http://mail.python.org/mailman/listinfo/python-list
Re: A Bug By Any Other Name ...
In message , MRAB wrote: > I wonder whether the problem with assignment in conditionals in C is due > to the assignment operator being "=". If it was ":=", would the error > still occur? One of the original C designers, Kernighan or Ritchie, admitted that he made the assignment operator "=" instead of ":=" just to save typing one character, because he thought assignments were more common than tests for equality. -- http://mail.python.org/mailman/listinfo/python-list
Re: Blocking XMPP API?
In message , Gabriel Rossetti wrote: > I am looking for blocking XMPP API. I'm wanting to make a webservice > that uses XMPP, but all the XMPP libs I find are non-blocking (i.e. w/ > callbacks). So why not have the caller sleep or something until the callback is invoked? It's easier to turn an async API into a synchronous one than the other way round. -- http://mail.python.org/mailman/listinfo/python-list
Re: The meaning of "="
> Lawrence D'Oliveiro (LD) wrote:
>LD> In message , Aahz wrote:
>Aahz> class AttrDict:
>Aahz> def __getitem__(self, key):
>Aahz> return getattr(self, key)
>LD> OK, let's try it:
>LD> >>> c = {}
>LD> >>> c["x"] = 3
>LD> >>> c.x = 4
>LD> Traceback (most recent call last):
>LD> File "", line 1, in
>LD> AttributeError: 'dict' object has no attribute 'x'
>LD> >>> class AttrDict:
>LD> ... def __getitem__(self, key):
>LD> ... return getattr(self, key)
>LD> ...
>LD> >>> c.x = 4
>LD> Traceback (most recent call last):
>LD> File "", line 1, in
>LD> AttributeError: 'dict' object has no attribute 'x'
>LD> Nope, still doesn't work...
Of course you need c = AttrDict()
And to get c.x = 4 working you also need a __setitem__.
And to get c["x"] working AtrrDict should subclass dict:
>>> class AttrDict(dict):
but these are only minor details :=)
--
Piet van Oostrum
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: p...@vanoostrum.org
--
http://mail.python.org/mailman/listinfo/python-list
how to set timeout while colling a soap method?
Hello!
I wrote some some python code for executing a soap method:
import SOAPpy
from SOAPpy import WSDL
_server = WSDL.Proxy(some wsdl)
r=_server.generuj(some parameters...)
print r.encode('cp1250')
It works fine. However, the execution time of this soap method might be
long. Therefore, I would like to set a timeout like 1 minute, after which I
would post a timeoute message.
Any ideas?
Greats,
--
View this message in context:
http://www.nabble.com/how-to-set-timeout-while-colling-a-soap-method--tp24461403p24461403.html
Sent from the Python - python-list mailing list archive at Nabble.com.
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to check if any item from a list of strings is in a big string?
Matt, how many words are you looking for, in how long a string ? Were you able to time any( substr in long_string ) against re.compile ( "|".join( list_items )) ? (REs are my method of choice, but different inputs of course give different times -- see google regex speed site:groups.google.com / site:stackoverflow.com .) cheers -- denis -- http://mail.python.org/mailman/listinfo/python-list
Best Way to Handle All Exceptions
Hello, I'm fairly new at Python so hopefully this question won't be too awful. I am writing some code that will FTP to a host, and want to catch any exception that may occur, take that and print it out (eventually put it into a log file and perform some alerting action). I've figured out two different ways to do this, and am wondering which is the best (i.e. cleanest, 'right' way to proceed). I'm also trying to understand exactly what occurs for each one. The first example: from ftplib import FTP try: ftp = FTP(ftp_host) ftp.login(ftp_user, ftp_pass) except Exception, err: print err This works fine. I read through the documentation, and my understanding is that there is a built-in exceptions module in python, that is automatically available in a built-in namespace. Within that module is an 'Exception' class which would contain whatever exception is thrown. So, I'm passing that to the except, along with err to hold the value and then print it out. The second example: from ftplib import FTP import sys try: ftp = FTP(ftp_host) ftp.login(ftp_user, ftp_pass) except: print sys.exc_info() Here I, for the most part, get the same thing. I'm not passing anything to except and just printing out the exception using a method defined in the sys module. So, I'm new to Python... I've made it this far and am happy, but want to make sure I'm coding correctly from the start. Which method is the better/cleaner/more standard way to continue? Thanks for any help. -- http://mail.python.org/mailman/listinfo/python-list
Is it possible to use keyword arguments from C?
I am learning about the python C-api in order to contribute a feature to numpy. I see a discussion (http://docs.python.org/extending/ extending.html#keyword-parameters-for-extension-functions) on how to create a function or method in C that accepts kwargs, but is it possible to call such a method from C using kwargs rather than positional arguments? -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use keyword arguments from C?
On Jul 13, 9:38 am, Darren Dale wrote: > I am learning about the python C-api in order to contribute a feature > to numpy. I see a discussion (http://docs.python.org/extending/ > extending.html#keyword-parameters-for-extension-functions) on how to > create a function or method in C that accepts kwargs, but is it > possible to call such a method from C using kwargs rather than > positional arguments? The answer was yes, this can be done. I found a short discussion at the bottom of section 1.6: http://docs.python.org/extending/extending.html#calling-python-functions-from-c -- http://mail.python.org/mailman/listinfo/python-list
python make dies :libtk8.5.so: cannot open shared object file: No such file or directory
Trying to build python-2.6.2 ./configure --prefix=/usr/local --exec-prefix=/usr/local LDFLAGS="-L/ usr/local" (runs through happily, had to make some libs local) make runs most of the way until… building '_tkinter' extension gcc -pthread -fPIC -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall - Wstrict-prototypes -DWITH_APPINIT=1 -I/usr/X11R6/include -I. -I/tmp/ meld/Python-2.6.2/./Include -I. -IInclude -I./Include -I/usr/local/ include -I/tmp/meld/Python-2.6.2/Include -I/tmp/meld/Python-2.6.2 -c / tmp/meld/Python-2.6.2/Modules/_tkinter.c -o build/temp.linux-i686-2.6/ tmp/meld/Python-2.6.2/Modules/_tkinter.o gcc -pthread -fPIC -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall - Wstrict-prototypes -DWITH_APPINIT=1 -I/usr/X11R6/include -I. -I/tmp/ meld/Python-2.6.2/./Include -I. -IInclude -I./Include -I/usr/local/ include -I/tmp/meld/Python-2.6.2/Include -I/tmp/meld/Python-2.6.2 -c / tmp/meld/Python-2.6.2/Modules/tkappinit.c -o build/temp.linux-i686-2.6/ tmp/meld/Python-2.6.2/Modules/tkappinit.o gcc -pthread -shared build/temp.linux-i686-2.6/tmp/meld/Python-2.6.2/ Modules/_tkinter.o build/temp.linux-i686-2.6/tmp/meld/Python-2.6.2/ Modules/tkappinit.o -L/usr/X11R6/lib64 -L/usr/X11R6/lib -L/usr/local/ lib -ltk8.5 -ltcl8.5 -lX11 -o build/lib.linux-i686-2.6/_tkinter.so *** WARNING: renaming "_tkinter" since importing it failed: libtk8.5.so: cannot open shared object file: No such file or directory Failed to find the necessary bits to build these modules: _sqlite3 bsddb185 sunaudiodev To find the necessary bits, look in setup.py in detect_modules() for the module's name. Failed to build these modules: _tkinter running build_scripts # cd /usr/local/lib # ls -la | grep libtk8.5.so -r-xr-xr-x 1 root root 1112606 Jul 10 13:28 libtk8.5.so Am I missing something, it’s there? Regards, -Tony -- http://mail.python.org/mailman/listinfo/python-list
Re: python make dies :libtk8.5.so: cannot open shared object file: No such file or directory
Tony Lay wrote: > # cd /usr/local/lib > > # ls -la | grep libtk8.5.so > > -r-xr-xr-x 1 root root 1112606 Jul 10 13:28 libtk8.5.so > > Am I missing something, it’s there? Is /usr/local/lib in your library search path? It looks like it isn't. Check /etc/ld.so.conf and /etc/ld.so.conf.d/. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: Best Way to Handle All Exceptions
seldan24 wrote: > Hello, > > I'm fairly new at Python so hopefully this question won't be too > awful. I am writing some code that will FTP to a host, and want to > catch any exception that may occur, take that and print it out > (eventually put it into a log file and perform some alerting action). > I've figured out two different ways to do this, and am wondering which > is the best (i.e. cleanest, 'right' way to proceed). I'm also trying > to understand exactly what occurs for each one. > > The first example: > > from ftplib import FTP > try: > ftp = FTP(ftp_host) > ftp.login(ftp_user, ftp_pass) > except Exception, err: > print err > > This works fine. I read through the documentation, and my > understanding is that there is a built-in exceptions module in python, > that is automatically available in a built-in namespace. Within that > module is an 'Exception' class which would contain whatever exception > is thrown. So, I'm passing that to the except, along with err to hold > the value and then print it out. > > The second example: > > from ftplib import FTP > import sys > try: > ftp = FTP(ftp_host) > ftp.login(ftp_user, ftp_pass) > except: > print sys.exc_info() > > Here I, for the most part, get the same thing. I'm not passing > anything to except and just printing out the exception using a method > defined in the sys module. > > So, I'm new to Python... I've made it this far and am happy, but want > to make sure I'm coding correctly from the start. Which method is the > better/cleaner/more standard way to continue? Thanks for any help. The latter is - unfortunately - the better. This comes from python allowing all kinds of objects being thrown as exceptions, not only those extending from a common ancestor like Exception. You seem to have a sensible take on this, but anyway I'd like to mention that using these kinds of catch-all code is rarely justified, as it imposes the danger of not handling errors at all. So make sure the program spits out a sensible error-message, and then dies. Unless it's a server-process of course. Diez -- http://mail.python.org/mailman/listinfo/python-list
Re: Problem with two instances of PySerial
> On Thu, 9 Jul 2009 08:40:18 -0500, Shine Jose wrote: > I achieve this by creating a separate worker thread to poll > the serial port for arrival of data and then updating the > required widget in main thread of program. For polling the > serial port, I create a separate instance of pySerial object. > However, I am unable to read any data in the worker thread. > Can the reason for this be 2 instances of pySerial objects > being connected to serial port. Yes. > The reason I had this doubt was because Serial.Connect() in > the worker thread did not throw any exception & isOpen() > method returns true. Under Unix, you are allowed to open a device or file as many times as you want (there are some common cases where there is a very useful thing to do). However, for something like a serial port, there's only one copy of each received data byte. Depending on exactly how you structure your code, the incoming data may be divided up amongst multiple readers, or one particular reader may get all of it. -- Grant Edwards grante Yow! I joined scientology at at a garage sale!! visi.com -- http://mail.python.org/mailman/listinfo/python-list
Re: need to write a assembly progrm
On 2009-07-12, Rhodri James wrote: >> The poorly punctuated paragraph sounds, to me, as if they mean >> they want (Windows assumed) EXE files that can be dropped onto >> remote/laptop machines without performing a full installation >> of Python... >> >> But yes... "assembly programs" does, to me, also mean things >> containing mnemonics for machine level opcodes. > > Given that to me, "assembly programs" does not mean .EXE files > at all, not even a little bit, I'm prefering to refuse to > guess :-) Ah, but guessing what a poster meant to ask and then answering the "guessed" question is a pretty popular sport here in c.l.p. The discussions resulting from "guessed" questions are often more informative than the answer to the intended question (if the OP ever returns to clarify his intent). -- Grant Edwards grante Yow! Bo Derek ruined at my life! visi.com -- http://mail.python.org/mailman/listinfo/python-list
Re: python make dies :libtk8.5.so: cannot open shared object file: No such file or directory
On Jul 13, 10:09 am, Christian Heimes wrote: > Tony Lay wrote: > > # cd /usr/local/lib > > > # ls -la | grep libtk8.5.so > > > -r-xr-xr-x 1 root root 1112606 Jul 10 13:28 libtk8.5.so > > > Am I missing something, it’s there? > > Is /usr/local/lib in your library search path? It looks like it isn't. > Check /etc/ld.so.conf and /etc/ld.so.conf.d/. > > Christian I added /usr/local/lib to ld.so.conf.d/python26.conf and ran ldconfig (thanks Chrisitan) Ran a make distclean (had to move the files and got a “file not found” from my previous working directory). Everything compiled and installed like a charm. Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: Best Way to Handle All Exceptions
seldan24 wrote: Hello, I'm fairly new at Python so hopefully this question won't be too awful. I am writing some code that will FTP to a host, and want to catch any exception that may occur, take that and print it out (eventually put it into a log file and perform some alerting action). I've figured out two different ways to do this, and am wondering which is the best (i.e. cleanest, 'right' way to proceed). I'm also trying to understand exactly what occurs for each one. The first example: from ftplib import FTP try: ftp = FTP(ftp_host) ftp.login(ftp_user, ftp_pass) except Exception, err: print err This works fine. I read through the documentation, and my understanding is that there is a built-in exceptions module in python, that is automatically available in a built-in namespace. Within that module is an 'Exception' class which would contain whatever exception is thrown. So, I'm passing that to the except, along with err to hold the value and then print it out. There isn't an "exceptions module"; exceptions are part of the language. The second example: from ftplib import FTP import sys try: ftp = FTP(ftp_host) ftp.login(ftp_user, ftp_pass) except: print sys.exc_info() This is called a "bare exception handler". It's virtually never the right way to do it. Here I, for the most part, get the same thing. I'm not passing anything to except and just printing out the exception using a method defined in the sys module. So, I'm new to Python... I've made it this far and am happy, but want to make sure I'm coding correctly from the start. Which method is the better/cleaner/more standard way to continue? Thanks for any help. You should use the most specific exception possible because at lot of things could raise an exception: >>> foo Traceback (most recent call last): File "", line 1, in foo NameError: name 'foo' is not defined >>> try: foo except Exception, e: print "*** Caught an exception ***" print e *** Caught an exception *** name 'foo' is not defined -- http://mail.python.org/mailman/listinfo/python-list
Re: Automate rsync w/ authentication
On Jul 10, 12:03 pm, mgi...@motorola.com (Gary Duzan) wrote: > In article > <3af970b1-b454-4d56-a33f-889ecfaca...@l28g2000vba.googlegroups.com>, > > Bryan wrote: > > >rsyncExec = '/usr/bin/ssh' > >source = 'r...@10.0.45.67:/home/bry/jquery.lookup' > >dest = '/home/bry/tmp' > >rshArg = '-e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"' > >args = [rsyncExec, '-a', '-v', '--dry-run', rshArg, source, dest] > > I think you want -e and the ssh command to be separate args. > Something like: > > rshArg = '/usr/bin/ssh -i /home/bry/keys/brybackup.key' > args = [rsyncExec, '-a', '-v', '--dry-run', '-e', rshArg, source, dest] > > or: > > rshArgs = [ '-e', '/usr/bin/ssh -i /home/bry/keys/brybackup.key' ] > args = [rsyncExec, '-a', '-v', '--dry-run'] + rshArgs + [ source, dest] > > Gary Duzan > Motorola H&NM Separating the argument parts worked. Strangely though, I don't need to do that for arguments such as --files-from='/path/to/file'. Also, in this example code I had the rsync executable path pointing to the ssh program, so no wonder I was getting the output of ssh! -- http://mail.python.org/mailman/listinfo/python-list
Re: Best Way to Handle All Exceptions
On Jul 13, 10:33 am, MRAB wrote: > seldan24 wrote: > > Hello, > > > I'm fairly new at Python so hopefully this question won't be too > > awful. I am writing some code that will FTP to a host, and want to > > catch any exception that may occur, take that and print it out > > (eventually put it into a log file and perform some alerting action). > > I've figured out two different ways to do this, and am wondering which > > is the best (i.e. cleanest, 'right' way to proceed). I'm also trying > > to understand exactly what occurs for each one. > > > The first example: > > > from ftplib import FTP > > try: > > ftp = FTP(ftp_host) > > ftp.login(ftp_user, ftp_pass) > > except Exception, err: > > print err > > > This works fine. I read through the documentation, and my > > understanding is that there is a built-in exceptions module in python, > > that is automatically available in a built-in namespace. Within that > > module is an 'Exception' class which would contain whatever exception > > is thrown. So, I'm passing that to the except, along with err to hold > > the value and then print it out. > > There isn't an "exceptions module"; exceptions are part of the language. > > > The second example: > > > from ftplib import FTP > > import sys > > try: > > ftp = FTP(ftp_host) > > ftp.login(ftp_user, ftp_pass) > > except: > > print sys.exc_info() > > This is called a "bare exception handler". It's virtually never the > right way to do it. > > > Here I, for the most part, get the same thing. I'm not passing > > anything to except and just printing out the exception using a method > > defined in the sys module. > > > So, I'm new to Python... I've made it this far and am happy, but want > > to make sure I'm coding correctly from the start. Which method is the > > better/cleaner/more standard way to continue? Thanks for any help. > > You should use the most specific exception possible because at lot of > things could raise an exception: > > >>> foo > > Traceback (most recent call last): > File "", line 1, in > foo > NameError: name 'foo' is not defined > >>> try: > foo > except Exception, e: > print "*** Caught an exception ***" > print e > > *** Caught an exception *** > name 'foo' is not defined- Hide quoted text - > > - Show quoted text - Thank you both for your input. I want to make sure I get started on the right track. For this particular script, I should have included that I would take the exception contents, and pass those to the logging module. For this particular script, all exceptions are fatal and I would want them to be. I just wanted a way to catch them and log them prior to program termination. I can understand why folks should always specify exact exceptions where possible if they plan on doing something with them, but I just want to log and terminate (and alarm) when any exception, irregardless of what it is, occurs; that's why I'm using the catch-all approach. I most likely should have put in an exit() in my snippet; sorry about that. I did notice that Python allows different objects to be thrown. This threw me off a bit because, at first, I was expecting everything to come through as nice, easy tuples. It turns out that ftplib throws some class from the sockets module (my terminology may be off here). As for terminology, sorry about the 'exceptions' misuse. The Python documentation refers to 'exceptions' as a module, albeit built-in, so I used that language accordingly (link to doc: http://docs.python.org/library/exceptions.html?highlight=exceptions#module-exceptions). Anyway, thanks again for the help and advice, it is greatly appreciated. -- http://mail.python.org/mailman/listinfo/python-list
Re: Best Way to Handle All Exceptions
On Jul 13, 2:26 pm, seldan24 wrote: > The first example: > > from ftplib import FTP > try: > ftp = FTP(ftp_host) > ftp.login(ftp_user, ftp_pass) > except Exception, err: > print err *If* you really do want to catch *all* exceptions (as mentioned already it is usually better to catch specific exceptions) this is the way to do it. To know why you should look at the class hierarchy on http://docs.python.org/library/exceptions.html. The reason is that you almost never want to be catching SystemExit, KeyboardInterrupt etc. catching them will give you trouble at some point (unless you really know what you're doing but then I would suggest you list them explicitly instead of using the bare except statement). While it is true that you could raise an object that is not a subclass from Exception it is very bad practice, you should never do that. And I've haven't seen an external module in the wild that does that in years and the stdlib will always play nice. Regards Floris -- http://mail.python.org/mailman/listinfo/python-list
Passing python list from C to python
I'm very new at wrapping Python/C, and I have run into some problems.
I have one python module that provides me with a list (provideBuffer
in provideBuff.py):
Py_Initialize();
pName = PyString_FromString("provideBuff");
pModule = PyImport_Import(pName);
pFunc = PyObject_GetAttrString(pModule,"provideBuffer");
pValue = PyObject_CallObject(pFunc,NULL);
pValue is now a PyList - i've even verified this with:
int a = PyList_Check(pValue);
printf("%d\n", a);
However, I want to send this PyList to another python module, but I
don't know how to do this. Initially I though I could just do like
above, only swapping NULL with pValue, but that is not working.
pName2 = PyString_FromString("C-embedding");
pModule2 = PyImport_Import(pName2);
pFunc2 = PyObject_GetAttrString(pModule2,"buff");
pValue2 = PyObject_CallObject(pFunc2,pValue);
pValue2 is now False! So i guess i cannot pass pValue as an argument
to PyObject_CallObject when i want to pass an python list as an
argument. But how must a go about to make this work?
--
http://mail.python.org/mailman/listinfo/python-list
list of all possible values
Hi guys. I was thinking about a problem I had: suppose I have a list of possible values. I want to to have a list of all possible lists of length n whose values are in that original list. For example: if my values are ['a', 'b', 'c'], then all possible lists of length 2 would be: aa, ab, ac, ba, bb, bc, ca, cb, cc. I created a recursive program to do it, but I was wondering if there was a better way of doing it (possibly with list comprehensions). Here's my recursive version: vals = ['a', 'b', 'c'] def foo(length): if length <=0: return [] if length == 1: return [[x] for x in vals] else: return [x + [y] for x in foo(length - 1) for y in vals] print foo(3) Thanks, David -- http://mail.python.org/mailman/listinfo/python-list
Re: list of all possible values
For example: if my values are ['a', 'b', 'c'], then all possible lists of length 2 would be: aa, ab, ac, ba, bb, bc, ca, cb, cc. I created a recursive program to do it, but I was wondering if there was a better way of doing it (possibly with list comprehensions). Here's my recursive version: vals = ['a', 'b', 'c'] def foo(length): if length <=0: return [] if length == 1: return [[x] for x in vals] else: return [x + [y] for x in foo(length - 1) for y in vals] Sounds like you want one of the combinitoric generators found in itertools[1] -- in this case, the itertools.product() does what you describe. According to the docs, it was added in 2.6 so if you're running an older version, you'd have to back-port it. -tkc [1] http://docs.python.org/library/itertools.html#itertools.product -- http://mail.python.org/mailman/listinfo/python-list
Re: Passing python list from C to python
On Jul 14, 1:47 am, hartley wrote:
> I'm very new at wrapping Python/C, and I have run into some problems.
>
> I have one python module that provides me with a list (provideBuffer
> in provideBuff.py):
>
> Py_Initialize();
> pName = PyString_FromString("provideBuff");
> pModule = PyImport_Import(pName);
>
> pFunc = PyObject_GetAttrString(pModule,"provideBuffer");
>
> pValue = PyObject_CallObject(pFunc,NULL);
>
> pValue is now a PyList - i've even verified this with:
>
> int a = PyList_Check(pValue);
> printf("%d\n", a);
>
> However, I want to send this PyList to another python module,
Please explain "send" ... do you mean the C equivalent of the Python
statement C_embedding.buff = the_pylist ?
BTW C-embedding would trigger a syntax error in Python source; best to
avoid ...
> but I
> don't know how to do this. Initially I though I could just do like
> above, only swapping NULL with pValue, but that is not working.
>
> pName2 = PyString_FromString("C-embedding");
> pModule2 = PyImport_Import(pName2);
> pFunc2 = PyObject_GetAttrString(pModule2,"buff");
Get?? Do you want Set? Is buff a Python function? Or is it the
destination of the "sending"? Any good reason for not checking the
return value for an error? [Rhetorical question; answer == "No"]
> pValue2 = PyObject_CallObject(pFunc2,pValue);
CallObject?? You used this before because you had a function and
wanted to call it because it returned you a value now you want to
do one of (in Python terms)
value = amodule.anattr
value = getattr(amodule, "anattr")
or
amodule.anattr = value
setattr(amodule, "anattr", value)
> pValue2 is now False!
False?? Do you mean the C NULL?
> So i guess i cannot pass pValue as an argument
> to PyObject_CallObject when i want to pass an python list as an
> argument. But how must a go about to make this work?
It's mainly just a matter of (1) knowing what you want to do (2)
picking the API that does what you want (3) checking the returned
value for error after every call.
HTH,
John
--
http://mail.python.org/mailman/listinfo/python-list
Re: list of all possible values
David Gibb:
> For example: if my values are ['a', 'b', 'c'], then all possible lists
> of length 2 would be: aa, ab, ac, ba, bb, bc, ca, cb, cc.
>>> from itertools import product
>>> list(product("abc", repeat=2))
[('a', 'a'), ('a', 'b'), ('a', 'c'), ('b', 'a'), ('b', 'b'), ('b',
'c'), ('c', 'a'), ('c', 'b'), ('c', 'c')]
Bye,
bearophile
--
http://mail.python.org/mailman/listinfo/python-list
Re: multiprocessing and dictionaries
On Monday 13 July 2009 01:56:08 Piet van Oostrum wrote:
> > Bjorn Meyer (BM) wrote:
> >
> >BM> I am trying to convert a piece of code that I am using the thread
> > module with BM> to the multiprocessing module.
> >
> >BM> The way that I have it set up is a chunk of code reads a text file and
> > assigns BM> a dictionary key multiple values from the text file. I am
> > using locks to write BM> the values to the dictionary.
> >BM> The way that the values are written is as follows:
> >BM> mydict.setdefault(key, []).append(value)
> >
> >BM> The problem that I have run into is that using multiprocessing, the
> > key gets BM> set, but the values don't get appended.
> >BM> I've even tried the Manager().dict() option, but it doesn't seem to
> > work.
> >
> >BM> Is this not supported at this time or am I missing something?
>
> I think you should give more information. Try to make a *minimal* program
> that shows the problem and include it in your posting or supply a
> download link.
> --
> Piet van Oostrum
> URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
> Private email: p...@vanoostrum.org
Here is what I have been using as a test.
This pretty much mimics what I am trying to do.
I put both threading and multiprocessing in the example which shows the output
that I am looking for.
#!/usr/bin/env python
import threading
from multiprocessing import Manager, Process
name = ('test1','test2','test3')
data1 = ('dat1','dat2','dat3')
data2 = ('datA','datB','datC')
def thread_test(name,data1,data2, d):
for nam in name:
for num in range(0,3):
d.setdefault(nam, []).append(data1[num])
d.setdefault(nam, []).append(data2[num])
print 'Thread test dict:',d
def multiprocess_test(name,data1,data2, mydict):
for nam in name:
for num in range(0,3):
mydict.setdefault(nam, []).append(data1[num])
mydict.setdefault(nam, []).append(data2[num])
print 'Multiprocess test dic:',mydict
if __name__ == '__main__':
mgr = Manager()
md = mgr.dict()
d = {}
m = Process(target=multiprocess_test, args=(name,data1,data2,md))
m.start()
t = threading.Thread(target=thread_test, args=(name,data1,data2,d))
t.start()
m.join()
t.join()
print 'Thread test:',d
print 'Multiprocess test:',md
Thanks
Bjorn
--
http://mail.python.org/mailman/listinfo/python-list
Germany Ups Terrorism Alert Before Election
With Germany going to the polls in a general election in three months, authorities are on high alert after detecting an increase in online warnings of terrorist attacks targeting the country. The German government held high-level talks with top security and intelligence chiefs in Berlin on Thursday to discuss the growing threat posed by Islamic extremists and to coordinate counterterrorism measures. Intelligence officials are alarmed by the rising number of videos posted online by militant Islamists who say they are specifically targeting Germany. Up to 13 videos are reported to have appeared on the Web since January, many of them referring to the deployment of German troops in Afghanistan. (See pictures of U.S. Marines opening a new offensive in Afghanistan.) for more http://www.terrorismsearch.blogspot.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Blocking XMPP API?
Lawrence D'Oliveiro wrote: In message , Gabriel Rossetti wrote: I am looking for blocking XMPP API. I'm wanting to make a webservice that uses XMPP, but all the XMPP libs I find are non-blocking (i.e. w/ callbacks). So why not have the caller sleep or something until the callback is invoked? It's easier to turn an async API into a synchronous one than the other way round. Yes, that is what I ended up doing, thanks for the response though :-) Gabriel -- http://mail.python.org/mailman/listinfo/python-list
Memory error due to big input file
Hi All,
I have a similar problem that many new python users might encounter. I would
really appreciate if you could help me fix the error.
I have a big text file with size more than 2GB. It turned out memory error
when reading in this file. Here is my python script, the error occurred at
line -- self.fh.readlines().
import math
import time
class textfile:
def __init__(self,fname):
self.name=fname
self.fh=open(fname)
self.fh.readline()
self.lines=self.fh.readlines()
a=textfile("/home/sservice/nfbc/GenoData/CompareCalls3.diff")
lfile=len(a.lines)
def myfun(snp,start,end):
subdata=a.lines[start:end+1]
NEWmiss=0
OLDmiss=0
DIFF=0
for row in subdata:
k=row.split()
if (k[3]=="0/0") & (k[4]!="0/0"):
NEWmiss=NEWmiss+1
elif (k[3]!="0/0") & (k[4]=="0/0"):
OLDmiss=OLDmiss+1
elif (k[3]!="0/0") & (k[4]!="0/0"):
DIFF=DIFF+1
result.write(snp+" "+str(NEWmiss)+" "+str(OLDmiss)+" "+str(DIFF)+"\n")
result=open("Summary_noLoop_diff3.txt","w")
result.write("SNP NEWmiss OLDmiss DIFF\n")
start=0
snp=0
for i in range(lfile):
if (i==0): continue
after=a.lines[i].split()
before=a.lines[i-1].split()
if (before[0]==after[0]):
if (i!=(lfile-1)): continue
else:
end=lfile-1
myfun(before[0],start,end)
snp=snp+1
else:
end=i-1
myfun(before[0],start,end)
snp=snp+1
start=i
if (i ==(lfile-1)):
myfun(after[0],start,start)
snp=snp+1
result.close()
sincerely, phoebe
--
http://mail.python.org/mailman/listinfo/python-list
Re: Memory error due to big input file
sityee kong wrote: Hi All, I have a similar problem that many new python users might encounter. I would really appreciate if you could help me fix the error. I have a big text file with size more than 2GB. It turned out memory error when reading in this file. Here is my python script, the error occurred at line -- self.fh.readlines(). [snip code] Your 'error' is that you're running it on a computer with insufficient memory. -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory error due to big input file
phoebe> I have a big text file with size more than 2GB. It turned out
phoebe> memory error when reading in this file. Here is my python
phoebe> script, the error occurred at line -- self.fh.readlines().
phoebe> import math
phoebe> import time
phoebe> class textfile:
phoebe> def __init__(self,fname):
phoebe> self.name=fname
phoebe> self.fh=open(fname)
phoebe> self.fh.readline()
phoebe> self.lines=self.fh.readlines()
Don't do that. The problem is that you are trying to read the entire file
into memory. Learn to operate a line (or a few lines) at a time. Try
something like:
a = open("/home/sservice/nfbc/GenoData/CompareCalls3.diff")
for line in a:
do your per-line work here
--
Skip Montanaro - s...@pobox.com - http://www.smontanaro.net/
when i wake up with a heart rate below 40, i head right for the espresso
machine. -- chaos @ forums.usms.org
--
http://mail.python.org/mailman/listinfo/python-list
Re: Threading.Condition problem
Piet van Oostrum wrote:
Gabriel Rossetti (GR) wrote:
GR> Sorry if this appears twice, I sent it once with an attachment and it never
GR> arrived so maybe the attachment is posing problems. I inlined the code this
GR> time (at the bottom), thank you,
GR> Gabriel
GR> ## Original message
GR> Hello everyone,
GR> I wrote a small example that listens for xmpp msgs in a thread. The main
GR> program calls a function that blocks (using Condition.wait) until a msg
GR> has been received and then returns the msg. When a msg arrives, it is
GR> put in a variable in the thread's object, it then calls the notify()
GR> attr on the Condition object. For some reason, this doesn't work, the
GR> thread gets the msg, tries to notify the Condition object, fails because
GR> the lock has not been acquired yet and blocks. I tried ignoring the
GR> failure, thinking that since it has not been acquired yet then when it
GR> is, it will get the msg right away and never call Condition.wait, thus
GR> not causing any problems, but this does not work either. Does someone
GR> know what I am doing wrong? I attached the code to this msg.
The code that puts the message in the variable should also acquire the
lock:
def onMessage(self, conn, msg):
with self._cv:
self.message = msg
self._cv.notify()
Thank you, that was the problem, I eventually found that
A couple of remarks:
1. I think the code is neater if all manipulation with the condition is
done in the same class (actually in the same instance -- making this
instance into a monitor).
The reason I didn't do that is that I don' t want the Listener to sleep,
I maybe over simplified the example, I actually put them in a dictionary
as they come in, so in your example, if I have several threads waiting
on msgs it wouldn't work. I'm trying to make a webservice api thay will
also be turned into a java .jar for people that need java. Now that I
think about it, each session will have an instance of the object so msgs
shouldn' t get mixed up (one connection per user), so I could block in
the thread. I'll try your suggestion as I think it is cleaner.
class Listener(Thread):
def __init__(self, ws):
Thread.__init__(self)
self.interrupt = Event()
self.message = None
self._cv = Condition()
self.client = ws._client
self.client.RegisterHandler('message', self.onMessage)
def onMessage(self, conn, msg):
with self._cv:
self.message = msg
try:
self._cv.notify()
except RuntimeError:
print "self._cv has not acquired the lock yet"
def getMsg(self):
with self._cv:
while !self.message
self._cv.wait()
return self.message
class WS(object):
def __init__(self, username, password, res):
self._jid = xmpp.protocol.JID(username)
self._client = xmpp.Client(self._jid.getDomain())
#self._cv = Condition()
def getMsg(self, mid=None):
"""
"""
return self._listener.getMsg()
Of course I haven't tested this code as I don't have the context
modules.
2. I don't know if more than one message can be delivered in the same
instance. If yes, than your code will not work, and neither will the
code above as, the message instance variable is never cleared. So the
next getMsg will be happy to deliver the previous one.
You would have to clear it when returning this one.
Like I said above, in reality I have a dict not just a simple variable.
def getMsg(self):
with self._cv:
while !self.message
self._cv.wait()
msg = self.message
self.message = None
return msg
3. If the messages come in faster than they can be processed some will
be lost as they will overwrite the previous one in the self.message
variable. The solution is to use a threading.Queue to transfer the
messages from one thread to the other. This also saves you the hassle
of doing your own synchronisation like above. If you are not familiar
with synchronising multithreaded applications it is very easy to make
errors and even if you are it is quite easy to do them wrong. I have
been involved in distributed programming courses at university level
and I have seen many errors in this area.
I used a dict because the API can also be setup to be async and use
callbacks, so I had to be able to
access the msgs directly and quickly.
Gabriel
--
http://mail.python.org/mailman/listinfo/python-list
Re: list of all possible values
On 7/13/2009 9:33 AM Tim Chase said... For example: if my values are ['a', 'b', 'c'], then all possible lists of length 2 would be: aa, ab, ac, ba, bb, bc, ca, cb, cc. I created a recursive program to do it, but I was wondering if there was a better way of doing it (possibly with list comprehensions). Here's my recursive version: vals = ['a', 'b', 'c'] def foo(length): if length <=0: return [] if length == 1: return [[x] for x in vals] else: return [x + [y] for x in foo(length - 1) for y in vals] Sounds like you want one of the combinitoric generators found in itertools[1] -- in this case, the itertools.product() does what you describe. According to the docs, it was added in 2.6 so if you're running an older version, you'd have to back-port it Or on systems with list comps try: >>> V='abc' >>> ['%s%s'%(ii,jj) for ii in V for jj in V] ['aa', 'ab', 'ac', 'ba', 'bb', 'bc', 'ca', 'cb', 'cc'] >>> Emile -- http://mail.python.org/mailman/listinfo/python-list
How to override Marshaller/Unmarshaller in xmlrpc?
Is there a way to override marshaller in xmlrpc.client properly? For example, I want to hide bytes -> Binary transformation inside marshaller (i.e., just let user pass a bytes value to function and marshaller dumps it in base64 autmatically). Unfortunately, I cannot just make a derived class with necessary dump_* function and pass it to client and server; marshaller initialization seems to be hidden inside xmlrpc. So, here's the question - is there a nice way to do it (because there are some dirty ways like rewriting getparser() ans so on, but I don't like the idea of copypasting code from library)? Or should I reconsider my design instead? Thanks in advance. -- http://mail.python.org/mailman/listinfo/python-list
Re: Best Way to Handle All Exceptions
2009/7/13 seldan24 :
> Thank you both for your input. I want to make sure I get started on
> the right track. For this particular script, I should have included
> that I would take the exception contents, and pass those to the
> logging module. For this particular script, all exceptions are fatal
> and I would want them to be. I just wanted a way to catch them and
> log them prior to program termination.
The logging module has a function specifically to handle this case:
try:
# do something
except:
logging.exception("Uh oh...")
The exception() method will automatically add details of the exception
to the log message it creates; as per the docs, you should only call
it from within an exception handler.
Hope that helps,
Vil.
--
http://mail.python.org/mailman/listinfo/python-list
Re: list of all possible values
> Or on systems with list comps try: > V='abc' ['%s%s'%(ii,jj) for ii in V for jj in V] > ['aa', 'ab', 'ac', 'ba', 'bb', 'bc', 'ca', 'cb', 'cc'] Yeah, except that the length here is hard-coded. There's no way (as far as I can tell, at least), to make this generic with respect to list length. Thanks for the itertools suggestion, guys. I was hoping to improve my list-comprehension-fu, but that module was the next best thing. David On Mon, Jul 13, 2009 at 1:58 PM, Emile van Sebille wrote: > On 7/13/2009 9:33 AM Tim Chase said... >>> >>> For example: if my values are ['a', 'b', 'c'], then all possible lists >>> of length 2 would be: aa, ab, ac, ba, bb, bc, ca, cb, cc. >>> >>> I created a recursive program to do it, but I was wondering if there >>> was a better way of doing it (possibly with list comprehensions). >>> >>> Here's my recursive version: >>> >>> vals = ['a', 'b', 'c'] >>> >>> def foo(length): >>> if length <=0: >>> return [] >>> if length == 1: >>> return [[x] for x in vals] >>> else: >>> return [x + [y] for x in foo(length - 1) for y in vals] >> >> Sounds like you want one of the combinitoric generators found in >> itertools[1] -- in this case, the itertools.product() does what you >> describe. According to the docs, it was added in 2.6 so if you're running >> an older version, you'd have to back-port it > > > Emile > > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://mail.python.org/mailman/listinfo/python-list
Efficient binary search tree stored in a flat array?
I couldn't find a good algorithms forum on the Internet, so I guess I'll ask this question here instead: Is it possible to efficiently maintain a binary search tree in a flat array (i.e., without using pointers), as is typically done for a binary heap? It *is* possible, of course, to keep an ordered list in a flat array, and then do a binary search on the ordered array, but then insertion and deletion are O(n), rather than O(log n). It would also clearly be possible to store a balanced binary tree in a flat array, storing the children of the node at index i at indices 2*i and 2*i + 1, just as one does for a binary heap. But if you do that, I don't know if you could then do insertions and deletions in O(log n) time. One idea that came to mind, is that maybe it is possible using a "treap", which is a combination of a binary heap and a binary search tree. Insertions and deletions in binary heaps can be done in O(log n) in a flat array, but I don't know if this is also true for a treap, since you also have the binary search tree invariants to maintain, in addition to the binary heap invariants. For all I know, this might cause rotations to no longer be O(log n). |>ouglas -- http://mail.python.org/mailman/listinfo/python-list
Re: Automate rsync w/ authentication
> Bryan (B) wrote: >B> On Jul 10, 12:03 pm, mgi...@motorola.com (Gary Duzan) wrote: >>> In article >>> <3af970b1-b454-4d56-a33f-889ecfaca...@l28g2000vba.googlegroups.com>, >>> >>> Bryan wrote: >>> >>> >rsyncExec = '/usr/bin/ssh' >>> >source = 'r...@10.0.45.67:/home/bry/jquery.lookup' >>> >dest = '/home/bry/tmp' >>> >rshArg = '-e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"' >>> >args = [rsyncExec, '-a', '-v', '--dry-run', rshArg, source, dest] >>> >>> I think you want -e and the ssh command to be separate args. >>> Something like: >>> >>> rshArg = '/usr/bin/ssh -i /home/bry/keys/brybackup.key' >>> args = [rsyncExec, '-a', '-v', '--dry-run', '-e', rshArg, source, dest] >>> >>> or: >>> >>> rshArgs = [ '-e', '/usr/bin/ssh -i /home/bry/keys/brybackup.key' ] >>> args = [rsyncExec, '-a', '-v', '--dry-run'] + rshArgs + [ source, dest] >>> >>> Gary Duzan >>> Motorola H&NM >B> Separating the argument parts worked. Strangely though, I don't need >B> to do that for arguments such as --files-from='/path/to/file'. Also, >B> in this example code I had the rsync executable path pointing to the >B> ssh program, so no wonder I was getting the output of ssh! I should have seen that because I changed it in my own copy!!! --files-from='/path/to/file *is* one argument, in contrast to -e "/usr/bin/ssh -i /home/bry/keys/brybackup.key" which is two arguments. -- Piet van Oostrum URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4] Private email: p...@vanoostrum.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Threading.Condition problem
> Gabriel Rossetti (GR) wrote: >GR> Piet van Oostrum wrote: ... >GR> I wrote a small example that listens for xmpp msgs in a thread. The main >GR> program calls a function that blocks (using Condition.wait) until a msg >GR> has been received and then returns the msg. When a msg arrives, it is >GR> put in a variable in the thread's object, it then calls the notify() >GR> attr on the Condition object. For some reason, this doesn't work, the >GR> thread gets the msg, tries to notify the Condition object, fails because >GR> the lock has not been acquired yet and blocks. I tried ignoring the >GR> failure, thinking that since it has not been acquired yet then when it >GR> is, it will get the msg right away and never call Condition.wait, thus >GR> not causing any problems, but this does not work either. Does someone >GR> know what I am doing wrong? I attached the code to this msg. >>> >>> The code that puts the message in the variable should also acquire the >>> lock: >>> >>> >>> def onMessage(self, conn, msg): >>> with self._cv: >>> self.message = msg >>> self._cv.notify() >>> >GR> Thank you, that was the problem, I eventually found that >>> A couple of remarks: >>> >>> 1. I think the code is neater if all manipulation with the condition is >>> done in the same class (actually in the same instance -- making this >>> instance into a monitor). >>> >GR> The reason I didn't do that is that I don' t want the Listener to sleep, I >GR> maybe over simplified the example, I actually put them in a dictionary as >GR> they come in, so in your example, if I have several threads waiting on msgs >GR> it wouldn't work. I'm trying to make a webservice api thay will also be >GR> turned into a java .jar for people that need java. Now that I think about >GR> it, each session will have an instance of the object so msgs shouldn' t get >GR> mixed up (one connection per user), so I could block in the thread. I'll >GR> try your suggestion as I think it is cleaner. Sleeping as you call it is better than busy waiting. You must have some synchronisation to make it efficient. If you put the messages in a dictionary access to the dictionary must be protected. Having several threads waiting for the messages doesn't prevent you from using proper synchronisation. Maybe you must use notify_all instead of notify. -- Piet van Oostrum URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4] Private email: p...@vanoostrum.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Efficient binary search tree stored in a flat array?
Douglas Alan writes: > It would also clearly be > possible to store a balanced binary tree in a flat array, storing the > children of the node at index i at indices 2*i and 2*i + 1, just as > one does for a binary heap. But if you do that, I don't know if you > could then do insertions and deletions in O(log n) time. Probably not. Maybe you could organize a 2-3 tree like that, at the expense of some space. -- http://mail.python.org/mailman/listinfo/python-list
Re: multiprocessing and dictionaries
> Bjorn Meyer (BM) wrote:
>BM> Here is what I have been using as a test.
>BM> This pretty much mimics what I am trying to do.
>BM> I put both threading and multiprocessing in the example which shows
>BM> the output that I am looking for.
>BM> #!/usr/bin/env python
>BM> import threading
>BM> from multiprocessing import Manager, Process
>BM> name = ('test1','test2','test3')
>BM> data1 = ('dat1','dat2','dat3')
>BM> data2 = ('datA','datB','datC')
[snip]
>BM> def multiprocess_test(name,data1,data2, mydict):
>BM> for nam in name:
>BM> for num in range(0,3):
>BM> mydict.setdefault(nam, []).append(data1[num])
>BM> mydict.setdefault(nam, []).append(data2[num])
>BM> print 'Multiprocess test dic:',mydict
I guess what's happening is this:
d.setdefault(nam, []) returns a list, initially an empty list ([]). This
list gets appended to. However, this list is a local list in the
multi-process_test Process, therefore the result is not reflected in the
original list inside the manager. Therefore all your updates get lost.
You will have to do operations directly on the dictionary itself, not on
any intermediary objects. Of course with the threading the situation is
different as all operations are local.
This works:
def multiprocess_test(name,data1,data2, mydict):
print name, data1, data2
for nam in name:
for num in range(0,3):
mydict.setdefault(nam, [])
mydict[nam] += [data1[num]]
mydict[nam] += [data2[num]]
print 'Multiprocess test dic:',mydict
If you have more than one process operating on the dictionary
simultaneously you have to beware of race conditions!!
--
Piet van Oostrum
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: p...@vanoostrum.org
--
http://mail.python.org/mailman/listinfo/python-list
Re: Best Way to Handle All Exceptions
> seldan24 (s) wrote: >s> Hello, >s> I'm fairly new at Python so hopefully this question won't be too >s> awful. I am writing some code that will FTP to a host, and want to >s> catch any exception that may occur, take that and print it out >s> (eventually put it into a log file and perform some alerting action). >s> I've figured out two different ways to do this, and am wondering which >s> is the best (i.e. cleanest, 'right' way to proceed). I'm also trying >s> to understand exactly what occurs for each one. >s> The first example: >s> from ftplib import FTP >s> try: >s> ftp = FTP(ftp_host) >s> ftp.login(ftp_user, ftp_pass) >s> except Exception, err: >s> print err I think you should restrict yourself to those exceptions that are related to ftp. Do you want to catch an exception like a misspelling in one of the variables? from ftplib import FTP, all_errors try: ftp = FTP(ftp_host) ftp.login(ftp_user, ftp_pass) except all_errors, err: print err -- Piet van Oostrum URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4] Private email: p...@vanoostrum.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Efficient binary search tree stored in a flat array?
In article , Douglas Alan wrote: > >I couldn't find a good algorithms forum on the Internet, so I guess >I'll ask this question here instead: Is it possible to efficiently >maintain a binary search tree in a flat array (i.e., without using >pointers), as is typically done for a binary heap? > >It *is* possible, of course, to keep an ordered list in a flat array, >and then do a binary search on the ordered array, but then insertion >and deletion are O(n), rather than O(log n). Still, unless your list is large (more than thousands of elements), that's the way you should go. See the bisect module. Thing is, the speed difference between C and Python means the constant for insertion and deletion is very very small relative to bytecode speed. Keep in mind that Python's object/binding model means that you're shuffling pointers in the list rather than items. -- Aahz (a...@pythoncraft.com) <*> http://www.pythoncraft.com/ "If you think it's expensive to hire a professional to do the job, wait until you hire an amateur." --Red Adair -- http://mail.python.org/mailman/listinfo/python-list
Re: Best Way to Handle All Exceptions
Diez B. Roggisch wrote: The latter is - unfortunately - the better. This comes from python allowing all kinds of objects being thrown as exceptions, not only those extending from a common ancestor like Exception. Fixed in Python 3. exceptions, without exceptions, are instances of BaseException, either directly or as instances of subcleasses of BaseException. One of many reasons to switch when possible. tjr -- http://mail.python.org/mailman/listinfo/python-list
Tkinter / Entry widget problem
Hello. I searched the archives but couldn't find a solution to a problem related to the Entry widget in Tkinter. When creating a pop-up window in an app, which contains an Entry widget, I want this widget to contain some default string, to have all this default string selected (as if the user had manually selected everything), and to have the focus transferred to this widget. (The idea is then that if the window pops up, the user won't have to click or press Tab any more before being able to type what is needed in the textbox, overwriting what is written there already.) I thought this might be the way to go: entrybox=Entry(toplevel_parent_window) entrybox.insert(0,"Some default string") entrybox.select_range(0,END) entrybox.focus_set() entrybox.pack() But it doesn't seem to work - the focus is not transferred to the Entry widget, and the text does not appear to be selected (even though after this entrybox.selection_present() returns True). What am I doing wrong? andras -- http://mail.python.org/mailman/listinfo/python-list
Re: Efficient binary search tree stored in a flat array?
On Jul 13, 3:57 pm, a...@pythoncraft.com (Aahz) wrote: > Still, unless your list is large (more than thousands of elements), > that's the way you should go. See the bisect module. Thing is, the > speed difference between C and Python means the constant for insertion > and deletion is very very small relative to bytecode speed. Keep in > mind that Python's object/binding model means that you're shuffling > pointers in the list rather than items. Thank you. My question wasn't intended to be Python specific, though. I am just curious for purely academic reasons about whether there is such an algorithm. All the sources I've skimmed only seem to the answer the question via omission. Which is kind of strange, since it seems to me like an obvious question to ask. If I find the free time, I might try to work out myself whether it can be done with a treap. |>ouglas -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory error due to big input file
sityee kong wrote:
Hi All,
I have a similar problem that many new python users might encounter. I would
really appreciate if you could help me fix the error.
I have a big text file with size more than 2GB. It turned out memory error
when reading in this file. Here is my python script, the error occurred at
line -- self.fh.readlines().
import math
import time
class textfile:
def __init__(self,fname):
self.name=fname
self.fh=open(fname)
self.fh.readline()
self.lines=self.fh.readlines()
a=textfile("/home/sservice/nfbc/GenoData/CompareCalls3.diff")
lfile=len(a.lines)
def myfun(snp,start,end):
subdata=a.lines[start:end+1]
NEWmiss=0
OLDmiss=0
DIFF=0
for row in subdata:
k=row.split()
if (k[3]=="0/0") & (k[4]!="0/0"):
NEWmiss=NEWmiss+1
elif (k[3]!="0/0") & (k[4]=="0/0"):
OLDmiss=OLDmiss+1
elif (k[3]!="0/0") & (k[4]!="0/0"):
DIFF=DIFF+1
result.write(snp+" "+str(NEWmiss)+" "+str(OLDmiss)+" "+str(DIFF)+"\n")
result=open("Summary_noLoop_diff3.txt","w")
result.write("SNP NEWmiss OLDmiss DIFF\n")
start=0
snp=0
for i in range(lfile):
if (i==0): continue
after=a.lines[i].split()
before=a.lines[i-1].split()
if (before[0]==after[0]):
if (i!=(lfile-1)): continue
else:
end=lfile-1
myfun(before[0],start,end)
snp=snp+1
else:
end=i-1
myfun(before[0],start,end)
snp=snp+1
start=i
if (i ==(lfile-1)):
myfun(after[0],start,start)
snp=snp+1
result.close()
sincerely, phoebe
Others have pointed out that you have too little memory for a 2gig data
structure. If you're running on a 32bit system, chances are it won't
matter how much memory you add, a process is limited to 4gb, and the OS
typically takes about half of it, your code and other data takes some,
and you don't have 2gig left. A 64 bit version of Python, running on a
64bit OS, might be able to "just work."
Anyway, loading the whole file into a list is seldom the best answer,
except for files under a meg or so. It's usually better to process the
file in sequence. It looks like you're also making slices of that data,
so they could potentially be pretty big as well.
If you can be assured that you only need the current line and the
previous two (for example), then you can use a list of just those three,
and delete the oldest one, and add a new one to that list each time
through the loop.
Or, you can add some methods to that 'textfile' class that fetch a line
by index. Brute force, you could pre-scan the file, and record all the
file offsets for the lines you find, rather than storing the actual
line. So you still have just as big a list, but it's a list of
integers. Then when somebody calls your method, he passes an integer,
and you return the particular line. A little caching for performance,
and you're good to go.
Anyway, if you organize it that way, you can code the rest of the module
to not care whether the whole file is really in memory or not.
BTW, you should derive all your classes from something. If nothing
else, use object.
class textfile(object):
--
http://mail.python.org/mailman/listinfo/python-list
Pickling classes -- disappearing lists?
I'm trying to pickle an instance of a class. It mostly works just fine
-- I can save the pickle to a file, restore it from that file, and
it's mostly okay. The problem is, some lists seem to disappear. For
example (snipped and crunched from the code giving me trouble):
---
class InitGame:
value = False
journal = []
game.InitGame()
def Save():
global game
import cPickle, gzip, os
# Change some data
game.journal.append("A value")
game.value = True
pickles = {'game': game}
jar = gzip.open("pickefile", 'wb')
cPickle.dump(pickles, jar, 2)
jar.close()
def Load():
global game
import gzip, os, cPickle
jar = gzip.open("picklefile", 'r')
loaded = cPickle.load(jar)
jar.close()
game = loaded["game"]
---
Now, if I save a pickle, then load it back in, I'll get an instance of
InitGame called "game", and game.value will be true, but the list
"journal" will be empty.
Am I missing something obvious about the pickle spec? Am I doing
something wrong? Or should I be hunting for a deeper bug in the code?
I've noticed that pretty much anything that's a list isn't saving to
the pickle (or loading from the pickle... it's hard to tell which).
--
http://mail.python.org/mailman/listinfo/python-list
Re: Memory error due to big input file
> BTW, you should derive all your classes from something. If nothing > else, use object. > class textfile(object): Just out of curiousity... why is that? I've been coding in Python for a long time, and I never derive my base classes. What's the advantage to deriving them? -- http://mail.python.org/mailman/listinfo/python-list
PDF: finding a blank image
I have about 6000 PDF files which have been produced using a scanner with more being produced each day. The PDF files contain old paper records which have been taking up space. The scanner is set to detect when there is information on the backside of the page (duplex scan). The problem of course is it's not the always reliable and we wind up with a number of PDF files containing blank pages. What I would like to do is have python detect a "blank" pages in a PDF file and remove it. Any suggestions? Thanks, DrL -- http://mail.python.org/mailman/listinfo/python-list
Re: Pickling classes -- disappearing lists?
On Mon, Jul 13, 2009 at 2:14 PM, Aaron Scott wrote:
> I'm trying to pickle an instance of a class. It mostly works just fine
> -- I can save the pickle to a file, restore it from that file, and
> it's mostly okay. The problem is, some lists seem to disappear. For
> example (snipped and crunched from the code giving me trouble):
>
> ---
>
>
> class InitGame:
> value = False
> journal = []
>
>
> game.InitGame()
That line doesn't make sense with the code you've given...
> def Save():
> global game
> import cPickle, gzip, os
>
> # Change some data
> game.journal.append("A value")
> game.value = True
>
> pickles = {'game': game}
> jar = gzip.open("pickefile", 'wb')
> cPickle.dump(pickles, jar, 2)
> jar.close()
>
>
> def Load():
> global game
> import gzip, os, cPickle
> jar = gzip.open("picklefile", 'r')
> loaded = cPickle.load(jar)
> jar.close()
> game = loaded["game"]
>
>
> ---
>
> Now, if I save a pickle, then load it back in, I'll get an instance of
> InitGame called "game", and game.value will be true, but the list
> "journal" will be empty.
>
> Am I missing something obvious about the pickle spec? Am I doing
> something wrong? Or should I be hunting for a deeper bug in the code?
Your class definition isn't right. It makes 'value' and 'journal'
class variables (Java lingo: "static variables") as opposed to the
instance variables they should be. Here's a corrected version:
class InitGame(object):
def __init__(self):
#instance variables are created through self.foo assignments in __init__
self.value = False
self.journal = []
Cheers,
Chris
--
http://blog.rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: Memory error due to big input file
2009/7/13 Aaron Scott : >> BTW, you should derive all your classes from something. If nothing >> else, use object. >> class textfile(object): > > Just out of curiousity... why is that? I've been coding in Python for > a long time, and I never derive my base classes. What's the advantage > to deriving them? class Foo: uses the old object model. class Foo(object): uses the new object model. See http://docs.python.org/reference/datamodel.html (specifically section 3.3) for details of the differences. Vil. -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory error due to big input file
On Mon, Jul 13, 2009 at 2:51 PM, Vilya Harvey wrote: > 2009/7/13 Aaron Scott : >>> BTW, you should derive all your classes from something. If nothing >>> else, use object. >>> class textfile(object): >> >> Just out of curiousity... why is that? I've been coding in Python for >> a long time, and I never derive my base classes. What's the advantage >> to deriving them? > > class Foo: > > uses the old object model. > > class Foo(object): > > uses the new object model. > > See http://docs.python.org/reference/datamodel.html (specifically > section 3.3) for details of the differences. Note that Python 3.0 makes explicitly subclassing `object` unnecessary since it removes old-style classes; a class that doesn't explicitly subclass anything will implicitly subclass `object`. Cheers, Chris -- http://blog.rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Pickling classes -- disappearing lists?
> Your class definition isn't right. It makes 'value' and 'journal' > class variables (Java lingo: "static variables") as opposed to the > instance variables they should be. Here's a corrected version: > Woah, thanks. I can't believe I made such a stupid mistake. It's not like I'm a newcomer to Python, either. I'm can't believe I never noticed what I was doing. No more 2am coding for me. Thanks, Aaron -- http://mail.python.org/mailman/listinfo/python-list
Re: PDF: finding a blank image
DrLeif writes: > What I would like to do is have python detect a "blank" pages in a PDF > file and remove it. Any suggestions? The odds are good that even a blank page is being "rendered" within the PDF as having some small bits of data due to scanner resolution, imperfections on the page, etc.. So I suspect you won't be able to just look for a well-defined pattern in the resulting PDF or anything. Unless you're using OCR, the odds are good that the scanner is rendering the PDF as an embedded image. What I'd probably do is extract the image of the page, and then use image processing on it to try to identify blank pages. I haven't had the need to do this myself, and tool availability would depend on platform, but for example, I'd probably try ImageMagick's convert operation to turn the PDF into images (like PNGs). I think Gimp can also do a similar conversion, but you'd probably have to script it yourself. Once you have an image of a page, you could then use something like OpenCV to process the page (perhaps a morphology operation to remove small noise areas, then a threshold or non-zero counter to judge "blankness"), or probably just something like PIL depending on complexity of the processing needed. Once you identify a blank page, removing it could either be with pure Python (there have been other posts recently about PDF libraries) or with external tools (such as pdftk under Linux for example). -- David -- http://mail.python.org/mailman/listinfo/python-list
Re: PDF: finding a blank image
DrLeif wrote: I have about 6000 PDF files which have been produced using a scanner with more being produced each day. The PDF files contain old paper records which have been taking up space. The scanner is set to detect when there is information on the backside of the page (duplex scan). The problem of course is it's not the always reliable and we wind up with a number of PDF files containing blank pages. What I would like to do is have python detect a "blank" pages in a PDF file and remove it. Any suggestions? I'd check into ReportLab's commercial product, it may well be easily capable of that. If no success, you might contact PJ at Groklaw, she has dealt with a _lot_ of PDFs (and knows people who deal with PDFs in bulk). --Scott David Daniels scott.dani...@acm.org -- http://mail.python.org/mailman/listinfo/python-list
Re: PDF: finding a blank image
Perhaps your blank pages have a characteristic size. Or perhaps if you trim them with `convert' (ImageMagick) there is nothing left. On Mon, Jul 13, 2009 at 3:44 PM, DrLeif wrote: > I have about 6000 PDF files which have been produced using a scanner > with more being produced each day. The PDF files contain old paper > records which have been taking up space. The scanner is set to > detect when there is information on the backside of the page (duplex > scan). The problem of course is it's not the always reliable and we > wind up with a number of PDF files containing blank pages. > > What I would like to do is have python detect a "blank" pages in a PDF > file and remove it. Any suggestions? > > > Thanks, > DrL > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://mail.python.org/mailman/listinfo/python-list
Re: multiprocessing and dictionaries
On Monday 13 July 2009 13:12:18 Piet van Oostrum wrote:
> > Bjorn Meyer (BM) wrote:
> >
> >BM> Here is what I have been using as a test.
> >BM> This pretty much mimics what I am trying to do.
> >BM> I put both threading and multiprocessing in the example which shows
> >BM> the output that I am looking for.
> >
> >BM> #!/usr/bin/env python
> >
> >BM> import threading
> >BM> from multiprocessing import Manager, Process
> >
> >BM> name = ('test1','test2','test3')
> >BM> data1 = ('dat1','dat2','dat3')
> >BM> data2 = ('datA','datB','datC')
>
> [snip]
>
> >BM> def multiprocess_test(name,data1,data2, mydict):
> >BM> for nam in name:
> >BM> for num in range(0,3):
> >BM> mydict.setdefault(nam, []).append(data1[num])
> >BM> mydict.setdefault(nam, []).append(data2[num])
> >BM> print 'Multiprocess test dic:',mydict
>
> I guess what's happening is this:
>
> d.setdefault(nam, []) returns a list, initially an empty list ([]). This
> list gets appended to. However, this list is a local list in the
> multi-process_test Process, therefore the result is not reflected in the
> original list inside the manager. Therefore all your updates get lost.
> You will have to do operations directly on the dictionary itself, not on
> any intermediary objects. Of course with the threading the situation is
> different as all operations are local.
>
> This works:
>
> def multiprocess_test(name,data1,data2, mydict):
> print name, data1, data2
> for nam in name:
> for num in range(0,3):
> mydict.setdefault(nam, [])
> mydict[nam] += [data1[num]]
> mydict[nam] += [data2[num]]
> print 'Multiprocess test dic:',mydict
>
> If you have more than one process operating on the dictionary
> simultaneously you have to beware of race conditions!!
> --
> Piet van Oostrum
> URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
> Private email: p...@vanoostrum.org
Excellent. That works perfectly.
Thank you for your response Piet.
Bjorn
--
http://mail.python.org/mailman/listinfo/python-list
Re: Best Way to Handle All Exceptions
seldan24 writes: > I'm fairly new at Python so hopefully this question won't be too > awful. I am writing some code that will FTP to a host, and want to > catch any exception that may occur, take that and print it out > (eventually put it into a log file and perform some alerting action). You have been given a lot of advice so far about catching exceptions with the ‘try … except’ syntax. But reading your request, I think perhaps you don't want to catch exceptions at all. Instead of catching *all* exceptions at a specific point in your code, and then having nothing to do but end the program (possibly after some pre-exit action like logging), I think you would be better served by installing a custom top-level exception handler. When an exception is raised and uncaught, the interpreter calls sys.excepthook with three arguments, the exception class, exception instance, and a traceback object. In an interactive session this happens just before control is returned to the prompt; in a Python program this happens just before the program exits. The handling of such top-level exceptions can be customized by assigning another three-argument function to sys.excepthook. http://docs.python.org/library/sys#sys.excepthook> With a custom top-level exception handler in place, you should then restrict your use of ‘try: … except FooException: …’ to be as precise as possible, so that the ‘try’ block is as small as possible and the ‘FooException’ is as specific as possible. Any other exceptions that you don't specifically catch will then go through to your default handle-it-just-before-we-exit ‘sys.excepthook’ exception handler. -- \ “Nature hath given men one tongue but two ears, that we may | `\ hear from others twice as much as we speak.” —Epictetus, | _o__) _Fragments_ | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Best Way to Handle All Exceptions
On Jul 13, 12:31 pm, Piet van Oostrum wrote: > > seldan24 (s) wrote: > >s> Hello, > >s> I'm fairly new at Python so hopefully this question won't be too > >s> awful. I am writing some code that will FTP to a host, and want to > >s> catch any exception that may occur, take that and print it out > >s> (eventually put it into a log file and perform some alerting action). > >s> I've figured out two different ways to do this, and am wondering which > >s> is the best (i.e. cleanest, 'right' way to proceed). I'm also trying > >s> to understand exactly what occurs for each one. > >s> The first example: > >s> from ftplib import FTP > >s> try: > >s> ftp = FTP(ftp_host) > >s> ftp.login(ftp_user, ftp_pass) > >s> except Exception, err: > >s> print err > > I think you should restrict yourself to those exceptions that are > related to ftp. Do you want to catch an exception like a misspelling in > one of the variables? He quite reasonably could want that, such as if the program is designed to be run from a cron job, or in some other non-interactive way. Just because something is usually a bad idea doesn't mean it always is. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Best Way to Handle All Exceptions
On Mon, 13 Jul 2009 17:49:23 -0700, Carl Banks wrote: > On Jul 13, 12:31 pm, Piet van Oostrum wrote: >> > seldan24 (s) wrote: >> >s> Hello, >> >s> I'm fairly new at Python so hopefully this question won't be too s> >> >awful. I am writing some code that will FTP to a host, and want to s> >> >catch any exception that may occur, take that and print it out s> >> >(eventually put it into a log file and perform some alerting action). >> >s> I've figured out two different ways to do this, and am wondering >> >which s> is the best (i.e. cleanest, 'right' way to proceed). I'm >> >also trying s> to understand exactly what occurs for each one. s> The >> >first example: >> >s> from ftplib import FTP >> >s> try: >> >s> ftp = FTP(ftp_host) >> >s> ftp.login(ftp_user, ftp_pass) s> except Exception, err: >> >s> print err >> >> I think you should restrict yourself to those exceptions that are >> related to ftp. Do you want to catch an exception like a misspelling in >> one of the variables? > > He quite reasonably could want that, such as if the program is designed > to be run from a cron job, or in some other non-interactive way. Why is it okay for non-interactive programs to silently have incorrect behaviour? What's the point of a cron job that doesn't do what it is supposed to, because it has a bug which is silently swallowed? "I find it amusing when novice programmers believe their main job is preventing programs from crashing. ... More experienced programmers realize that correct code is great, code that crashes could use improvement, but incorrect code that doesn't crash is a horrible nightmare." http://www.pphsg.org/cdsmith/types.html -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Best Way to Handle All Exceptions
On Tue, 14 Jul 2009 10:09:06 +1000, Ben Finney wrote: > Instead of catching *all* exceptions at a specific point in your code, > and then having nothing to do but end the program (possibly after some > pre-exit action like logging), I think you would be better served by > installing a custom top-level exception handler. ... > Any other exceptions that you don't specifically catch will then go > through to your default handle-it-just-before-we-exit ‘sys.excepthook’ > exception handler. Isn't that risky though? Won't that potentially change the exception- handling behaviour of functions and classes he imports from other modules? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: The meaning of "=" (Was: tough-to-explain Python)
On Mon, 13 Jul 2009 23:22:36 +1200, Lawrence D'Oliveiro wrote: [stupidity omitted] > Nope, still doesn't work... Are we supposed to interpret that post as Dumb Insolence or just Dumb? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory error due to big input file
On Mon, 13 Jul 2009 14:20:13 -0700, Aaron Scott wrote: >> BTW, you should derive all your classes from something. If nothing >> else, use object. >> class textfile(object): > > Just out of curiousity... why is that? I've been coding in Python for a > long time, and I never derive my base classes. What's the advantage to > deriving them? "Old style" classes (those whose base classes aren't derived from anything) have a few disadvantages: (1) Properties don't work correctly: >>> class Parrot: # old-style class ... def __init__(self): ... self._x = 3 ... def _setter(self, value): ... self._x = value ... def _getter(self): ... print "Processing ..." ... return self._x + 1 ... x = property(_getter, _setter) ... >>> p = Parrot() >>> p.x Processing ... 4 >>> p.x = 2 >>> p.x 2 In general, anything that uses the descriptor protocol, not just property, will fail to work correctly with old-style classes. (2) Classes using multiple inheritance with diamond-shaped inheritance will be broken. (3) __slots__ is just an attribute. (4) super() doesn't work. And, depending on whether you consider this a disadvantage or an advantage: (5) Special methods like __len__ can be over-ridden on the instance, not just the class: >>> class K: ... def __len__(self): ... return 0 ... >>> k = K() >>> len(k) 0 >>> k.__len__ = lambda : 42 >>> len(k) 42 In their favour: (1) Less typing. (2) If you're not using descriptors, including property(), or multiple inheritance with diamond diagrams, they work fine. (3) They're (apparently) a tiny bit faster. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: How to check if any item from a list of strings is in a big string?
En Mon, 13 Jul 2009 10:11:09 -0300, denis escribió: Matt, how many words are you looking for, in how long a string ? Were you able to time any( substr in long_string ) against re.compile ( "|".join( list_items )) ? There is a known algorithm to solve specifically this problem (Aho-Corasick), a good implementation should perform better than R.E. (and better than the gen.expr. with the advantage of returning WHICH string matched) There is a C extension somewhere implementing Aho-Corasick. -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: explode()
On Sat, 11 Jul 2009 18:50:28 -0700, Chris Rebert
wrote:
>On Sat, Jul 11, 2009 at 7:41 PM, Fred Atkinson wrote:
>> What is the Python equivalent of the PHP explode() function?
>
>some_string.split("separator")
>
>Cheers,
>Chris
What would the equivalent of the PHP function implode be?
Thanks,
Fred
--
http://mail.python.org/mailman/listinfo/python-list
Re: explode()
On Mon, Jul 13, 2009 at 11:28 PM, Fred Atkinson wrote:
> On Sat, 11 Jul 2009 18:50:28 -0700, Chris Rebert
> wrote:
>
>>On Sat, Jul 11, 2009 at 7:41 PM, Fred Atkinson wrote:
>>> What is the Python equivalent of the PHP explode() function?
>>
>>some_string.split("separator")
>>
>>Cheers,
>>Chris
>
> What would the equivalent of the PHP function implode be?
"separator".join(a_list_of_strings)
I would recommend you read the Python manual's section on string methods:
http://docs.python.org/library/stdtypes.html#string-methods
Cheers,
Chris
--
http://blog.rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: explode()
On Mon, 13 Jul 2009 22:32:55 -0700, Chris Rebert
wrote:
>On Mon, Jul 13, 2009 at 11:28 PM, Fred Atkinson wrote:
>> On Sat, 11 Jul 2009 18:50:28 -0700, Chris Rebert
>> wrote:
>>
>>>On Sat, Jul 11, 2009 at 7:41 PM, Fred Atkinson wrote:
What is the Python equivalent of the PHP explode() function?
>>>
>>>some_string.split("separator")
>>>
>>>Cheers,
>>>Chris
>>
>> What would the equivalent of the PHP function implode be?
>
>"separator".join(a_list_of_strings)
>
>I would recommend you read the Python manual's section on string methods:
>http://docs.python.org/library/stdtypes.html#string-methods
>
>Cheers,
>Chris
Chris,
You always seem to be waiting when I post a question. Do you
ever sleep?
I wish the Python site was as well written as the PHP site. On
the PHP site, I can look up a command and they show not only the docs
on that command but a list of all other commands associated with it.
Thanks. Python is a whole new animal to me. I'm taking a
course in PHP and Python online right now. I've done PHP scripting
before though I don't claim to be a whiz on it. But I'd barely heard
of Python before I took this course.
The one thing I really dislike about Python over PHP is that
Python can usually only appear in the cgi directory (unless other
arragements are made with your hosting provider or if you reconfigure
Apache on your own server if you have your own). With PHP, I can put
them in any folder on my Web site without any problem.
Regards,
Fred
--
http://mail.python.org/mailman/listinfo/python-list
python _binary_ code
Hi Can anyone give me solution to create a python binary file (bytecode) other than pyc file .So my source code be secure.. I am working on ubuntu 9.04 with python2.6.. I Regards Sanju -- http://mail.python.org/mailman/listinfo/python-list
Re: explode()
On 2009-07-14, Fred Atkinson wrote: > The one thing I really dislike about Python over PHP is that > Python can usually only appear in the cgi directory (unless other > arragements are made with your hosting provider or if you reconfigure > Apache on your own server if you have your own). With PHP, I can put > them in any folder on my Web site without any problem. That is a server configuration and has nothing to do with Python directly. -- http://mail.python.org/mailman/listinfo/python-list
Re: python _binary_ code
On Mon, Jul 13, 2009 at 10:41 PM, sanju ps wrote: > Hi > > Can anyone give me solution to create a python binary file (bytecode) other > than pyc file .So my source code be secure.. I am working on ubuntu 9.04 > with python2.6.. I PyInstaller (http://www.pyinstaller.org/) will generate a self-contained executable, but it won't hide the bytecode (which can be decompiled w/ decompyle). It's generally accepted to be impossible (or at least more trouble than it's worth) to protect Python source. See http://stackoverflow.com/questions/261638/how-do-i-protect-python-code for excellent and more detailed answers. Cheers, Chris -- http://blog.rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Re: python _binary_ code
sanju ps schrieb: > Hi > > Can anyone give me solution to create a python binary file (bytecode) other > than pyc file .So my source code be secure.. I am working on ubuntu 9.04 > with python2.6.. I It's impossible to secure your code if it runs on an untrusted computer. This is true for all programming languages but it's even easier to decompile code of byte interpreted languages like Python, Java or .NET. Microsoft has tried to protect its programs, the film industry has tried it with the movies and RIAA, too. All have failed. You won't have more luck, though. :) Christian -- http://mail.python.org/mailman/listinfo/python-list
copy a file
Dear all, Can anyone tell me that suppose i want to copy few lines from one text file to another then how can i do that.Looking forward for soon reply. Amrita Kumari Research Fellow IISER Mohali Chandigarh INDIA -- http://mail.python.org/mailman/listinfo/python-list
