compilation problem of python on AIX 6.1
Hello I want to intsall python on my AIX 6.1 Box. I couldn't get any rpm for python 2.5.x for AIX 6.1. THen I decided to compile python 2.5.4 on my AIX box. I downloaded the python source code from www.python.org and tried to compile on my AIX both using gcc. step 1 ./configure --with-gcc configuration runs successfully step 2 make it gives me error case $MAKEFLAGS in *-s*) CC='gcc -pthread' LDSHARED='./Modules/ ld_so_aix gcc -pthread -bI:Modules/python.exp' OPT='-DNDEBUG -g - fwrapv -O3 -Wall -Wstrict-prototypes' ./python -E ./setup.py -q build;; *) CC='gcc -pthread' LDSHARED='./Modules/ld_so_aix gcc - pthread -bI:Modules/python.exp' OPT='-DNDEBUG -g -fwrapv -O3 -Wall - Wstrict-prototypes' ./python -E ./setup.py build;; esac running build running build_ext building 'bz2' extension gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall - Wstrict-prototypes -I. -I/avre/Python-2.5.4/./Include -I. -IInclude - I./Include -I/avre/Python-2.5.4/Include -I/avre/Python-2.5.4 -c /avre/ Python-2.5.4/Modules/bz2module.c -o build/temp.aix-6.1-2.5/avre/ Python-2.5.4/Modules/bz2module.o building '_tkinter' extension ./Modules/ld_so_aix gcc -pthread -bI:Modules/python.exp build/ temp.aix-6.1-2.5/avre/Python-2.5.4/Modules/_tkinter.o build/ temp.aix-6.1-2.5/avre/Python-2.5.4/Modules/tkappinit.o -L/usr/X11/lib - ltk8.4 -ltcl8.4 -lX11 -o build/lib.aix-6.1-2.5/_tkinter.so *** WARNING: renaming _tkinter since importing it failed: 0509-022 Cannot load module build/lib.aix-6.1-2.5. 0509-026 System error: A file or directory in the path name does not exist. plz suggest -- http://mail.python.org/mailman/listinfo/python-list
Re: multiprocessing and dictionaries
Bjorn Meyer bjorn.m.me...@gmail.com (BM) wrote: BM I am trying to convert a piece of code that I am using the thread module with BM to the multiprocessing module. BM The way that I have it set up is a chunk of code reads a text file and assigns BM a dictionary key multiple values from the text file. I am using locks to write BM the values to the dictionary. BM The way that the values are written is as follows: BMmydict.setdefault(key, []).append(value) BM The problem that I have run into is that using multiprocessing, the key gets BM set, but the values don't get appended. BM I've even tried the Manager().dict() option, but it doesn't seem to work. BM Is this not supported at this time or am I missing something? I think you should give more information. Try to make a *minimal* program that shows the problem and include it in your posting or supply a download link. -- Piet van Oostrum p...@cs.uu.nl URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4] Private email: p...@vanoostrum.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Question about generators
Cameron Pulsford cameron.pulsf...@gmail.com (CP) wrote: CP I read it on the haskell site in their sieves/prime wheel section, CP I guess I misunderstood something. (east to do over there...) I did CP verify it against established list of primes and other generators CP I've written that use more normal methods, but I only hand verified CP it. If it is used in a sieve then the non-primes will be sieved out. CP It is at least interesting though, I can probably extend it to CP check for primality by using a more normal sieve method. It might CP be pretty fast too because generally it does only generate primes, CP and the few non primes it does generate could be caught quickly CP using a scratching out technique. it does only generate primes = it does generate all primes. CP When I was initially looking at it there are some interesting CP patterns I might be able to extend into a generator that would CP yield only correct sets of numbers for the 6x + n pattern. As I wrote in my previous reply, in your use case the non-primes are harmless. But it is useful to reflect that in the naming of your identifiers. *And please, don't top post.* -- Piet van Oostrum p...@cs.uu.nl URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4] Private email: p...@vanoostrum.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Addind imports to a class namespace
Thank you for your replies. I have factored out the dependency and everything is solved. Cheers, Ryan -- http://mail.python.org/mailman/listinfo/python-list
Re: Infinite loops and synchronization
Vincent Gulinao vincent.guli...@gmail.com (VG) wrote: VG lst = list() VG (lst populated by async twisted deferred callbacks) VG while True: VGif len(lst) == SOME_NUMBER: VGreturn lst VG Q1: is this a common OK practice? I'm worried infinite loops hogs memory. VG Q2: operating on list from threads (mostly appends) must be safe, VG right (synchronization)? I am not familiar enough with twisted, but I think the principle is independent from twisted. This loop will not hog memory but it will hog CPU time. You should use a synchronisation construct like threading.Condition or a Semaphore. Here is my suggestion with a Condition: Global somewhere: lst_cond = Condition() In your loop: lst = list() # Why not lst = []? while True: # strange while/if combo if len(lst) == SOME_NUMBER: return lst Make that: with lst_cond: while len(lst) SOME_NUMBER: lst_cond.wait() return lst In the callback: with lst_cond: lst.append(new_value) lst_cond.notify() In case you don't have a python that supports the with statement (not even `from future') you should use: lst_cond.acquire() try: . finally: lst_cond.release() I think the solution with a semaphore is less elegant. global: sem = Semaphore() loop: for i in range(SOME_NUMBER): sem.acquire() return lst In callback: lst.append(new_value) sem.release() *Be careful: I haven't tested this code (not even syntax checked). So consider it pseudo code.* -- Piet van Oostrum p...@cs.uu.nl URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4] Private email: p...@vanoostrum.org -- http://mail.python.org/mailman/listinfo/python-list
Re: the ultimate logging facility for debugging code
On Jul 10, 10:39 am, Jean-Michel Pichavant jeanmic...@sequans.com
wrote:
Greetings,
Sorry for the dubious title :o). I was wandering if there is a standard
(or reliable) python module that implements the following feature:
http://code.activestate.com/recipes/198078/
Recipe 198078: Wrapping method calls (meta-class example)
A metaclass is used to wrap all (or just some) methods forlogging
purposes. The underlying mechanism can be used as well to check pre/post
conditions, attribute access,... The basic point is, that the actual
class must not be changed in any way to achive the desired effect.
I had to adapt the code to some of my needs, but now it's almost
working, by simply defining a metaclass for any of my class, I get the
class methods calls logged with the instance that called the method, the
method name, the call line in the file, the parameter **value** passed
to the method and the returned value.
It just saved me from hours of ipython interactive debugging with the
old print way, and to beauty of it: I don't have to write any
self._log('Please log me') line.
As it is so useful, I bet there is a module that is doing this in a more
reliable way that I did. Anyone has heard of it ?
Jean-Michel
Check out the Python Decorator Library:
http://wiki.python.org/moin/PythonDecoratorLibrary
and you should be able to adapt what you find there to your needs.
Regards,
Vinay Sajip
--
http://mail.python.org/mailman/listinfo/python-list
Re: Infinite loops and synchronization
On Jul 13, 6:06 am, Vincent Gulinao vincent.guli...@gmail.com wrote: lst = list() (lst populated by async twisted deferred callbacks) while True: if len(lst) == SOME_NUMBER: return lst Q1: is this a common OK practice? I'm worried infinite loops hogs memory. Q2: operating on list from threads (mostly appends) must be safe, right (synchronization)? Q1: I'll answer your question with another. What's so fundamentally different between your infinite loop and this one: while len(lst) != SOME_NUMBER: pass return lst which is not an infinite loop[1]. Why would yours be any worse in terms of memory than mine? Are you allocating anything that would hog memory? Of course, like Piet said, it *will* hog your CPU, so you want a time.sleep(.1) in there, at the least. Of course, the question is: why aren't you using a semaphore to let you know you can proceed, and make the provider increment the semaphore? [1] -- well, it can be, if len(lst) == SOME_NUMBER never comes about, and I'd hazard a guess that that's pretty much where your fear of memory hogging comes from: it's easy to allocate stuff and not deallocate it within a cycle, only to find the bounds on that cycle going awry. -- http://mail.python.org/mailman/listinfo/python-list
simple question about Dictionary type containing List objects
Hi group,
I have a dict object like
emails={'mycontacts': [ 'x...@gmail.com, 'y...@gmail.com',
'z...@gmail.com'], 'myname':['gganesh']}
I need to get the lenght of the list mycontacts ,like
mycontacts_numbers=3
help me to solve
Thanks
--
http://mail.python.org/mailman/listinfo/python-list
Re: simple question about Dictionary type containing List objects
On Mon, Jul 13, 2009 at 6:34 PM, gganesh ganesh@gmail.com wrote:
Hi group,
I have a dict object like
emails={'mycontacts': [ 'x...@gmail.com, 'y...@gmail.com',
'z...@gmail.com'], 'myname':['gganesh']}
I need to get the lenght of the list mycontacts ,like
mycontacts_numbers=3 http://mail.python.org/mailman/listinfo/python-list
len(emails['mycontacts'])
3
HTH,
Ching-Yun Xavier Ho, Technical Artist
Contact Information
Mobile: (+61) 04 3335 4748
Skype ID: SpaXe85
Email: cont...@xavierho.com
Website: http://xavierho.com/
--
http://mail.python.org/mailman/listinfo/python-list
Re: The meaning of = (Was: tough-to-explain Python)
In message h3bogf$oo...@panix3.panix.com, Aahz wrote:
In article h3bagu$52...@lust.ihug.co.nz,
Lawrence D'Oliveiro l...@geek-central.gen.new_zealand wrote:
In message h37gv5$r8...@panix3.panix.com, Aahz wrote:
It helps to remember that names and namespaces are in many
ways syntactic sugar for dicts or lists.
Interesting, though, that Python insists on maintaining a distinction
between c[x] and c.x, whereas JavaScript doesn't bother.
Why do you say insists?
class AttrDict:
def __getitem__(self, key):
return getattr(self, key)
OK, let's try it:
c = {}
c[x] = 3
c.x = 4
Traceback (most recent call last):
File stdin, line 1, in module
AttributeError: 'dict' object has no attribute 'x'
class AttrDict:
... def __getitem__(self, key):
... return getattr(self, key)
...
c.x = 4
Traceback (most recent call last):
File stdin, line 1, in module
AttributeError: 'dict' object has no attribute 'x'
Nope, still doesn't work...
--
http://mail.python.org/mailman/listinfo/python-list
Re: Infinite loops and synchronization
In message mailman.3048.1247462046.8015.python-l...@python.org, Vincent Gulinao wrote: Q1: is this a common OK practice? I'm worried infinite loops hogs memory. The problem is not that the loop is infinite, but that it busy-waits, hogging CPU. -- http://mail.python.org/mailman/listinfo/python-list
Re: MySQLdb + SSH Tunnel
In message mailman.3035.1247427709.8015.python-l...@python.org, Emile van Sebille wrote: ssh with something like... ssh -lroot -L3306:C:3306 B Watch out for other instances of mysql on A or B... You can use a non-default local port and specify that in your local connection parameters. Similarly you can tell the remote server to use a non-default port in its /etc/my.cnf. -- http://mail.python.org/mailman/listinfo/python-list
Re: Looking for a tool to checkfor python script backward compatibility
In message 4a59951a$0$10853$426a7...@news.free.fr, Baptiste Lepilleur wrote: I'm looking for a tool that could be used in a pre-commit step to check that only features available in a old python version are used, say python 2.3 for example. The only sure way would be to run the script under an appropriately old version of Python. -- http://mail.python.org/mailman/listinfo/python-list
Re: simple question about Dictionary type containing List objects
gganesh wrote:
I have a dict object like
emails={'mycontacts': [ 'x...@gmail.com, 'y...@gmail.com',
'z...@gmail.com'], 'myname':['gganesh']}
I need to get the lenght of the list mycontacts ,like
mycontacts_numbers=3
mycontacts = emails['mycontacts']
mycontacts_number = len(mycontacts)
A list doesn't change in behaviour when it is contained in a dictionary.
Uli
--
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
--
http://mail.python.org/mailman/listinfo/python-list
Re: send keys to focused window
In message mpg.24c0392b71e8a0eb989...@news.bintube.com, Broken wrote: I am new to Python, and I'm miserably failing to send specific keys to (say) notepad. I don't understand why you need to automate a GUI front-end, meant for human use, to a function that can be directly performed without that front-end anyway. -- http://mail.python.org/mailman/listinfo/python-list
Re: Check file is locked?
In message 652cca82-44a3-473f-b640- c2336a9cf...@v15g2000prn.googlegroups.com, Rajat wrote: ... my whole idea is to close the wordpad / notepad application so that I can delete the file and the directory where this file resides. Don't you think the user might have that application open for a reason? -- http://mail.python.org/mailman/listinfo/python-list
Re: A Bug By Any Other Name ...
In message 4a538a71$0$30236$9b4e6...@newsspool1.arcor-online.net, Stefan Behnel wrote: Lawrence D'Oliveiro wrote: I wonder how many people have been tripped up by the fact that ++n and --n fail silently for numeric-valued n. I doubt that there are many. Plus, you misspelled them from the more obvious n++ and n-- which *do* fail with a SyntaxError. Funny, you accuse me of misspelling, when it's _your_ version that fails with errors! -- http://mail.python.org/mailman/listinfo/python-list
Re: A Bug By Any Other Name ...
In message mailman.2787.1246986627.8015.python-l...@python.org, MRAB wrote: I wonder whether the problem with assignment in conditionals in C is due to the assignment operator being =. If it was :=, would the error still occur? One of the original C designers, Kernighan or Ritchie, admitted that he made the assignment operator = instead of := just to save typing one character, because he thought assignments were more common than tests for equality. -- http://mail.python.org/mailman/listinfo/python-list
Is it possible to use keyword arguments from C?
I am learning about the python C-api in order to contribute a feature to numpy. I see a discussion (http://docs.python.org/extending/ extending.html#keyword-parameters-for-extension-functions) on how to create a function or method in C that accepts kwargs, but is it possible to call such a method from C using kwargs rather than positional arguments? -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use keyword arguments from C?
On Jul 13, 9:38 am, Darren Dale dsdal...@gmail.com wrote: I am learning about the python C-api in order to contribute a feature to numpy. I see a discussion (http://docs.python.org/extending/ extending.html#keyword-parameters-for-extension-functions) on how to create a function or method in C that accepts kwargs, but is it possible to call such a method from C using kwargs rather than positional arguments? The answer was yes, this can be done. I found a short discussion at the bottom of section 1.6: http://docs.python.org/extending/extending.html#calling-python-functions-from-c -- http://mail.python.org/mailman/listinfo/python-list
python make dies :libtk8.5.so: cannot open shared object file: No such file or directory
Trying to build python-2.6.2 ./configure --prefix=/usr/local --exec-prefix=/usr/local LDFLAGS=-L/ usr/local (runs through happily, had to make some libs local) make runs most of the way until… building '_tkinter' extension gcc -pthread -fPIC -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall - Wstrict-prototypes -DWITH_APPINIT=1 -I/usr/X11R6/include -I. -I/tmp/ meld/Python-2.6.2/./Include -I. -IInclude -I./Include -I/usr/local/ include -I/tmp/meld/Python-2.6.2/Include -I/tmp/meld/Python-2.6.2 -c / tmp/meld/Python-2.6.2/Modules/_tkinter.c -o build/temp.linux-i686-2.6/ tmp/meld/Python-2.6.2/Modules/_tkinter.o gcc -pthread -fPIC -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall - Wstrict-prototypes -DWITH_APPINIT=1 -I/usr/X11R6/include -I. -I/tmp/ meld/Python-2.6.2/./Include -I. -IInclude -I./Include -I/usr/local/ include -I/tmp/meld/Python-2.6.2/Include -I/tmp/meld/Python-2.6.2 -c / tmp/meld/Python-2.6.2/Modules/tkappinit.c -o build/temp.linux-i686-2.6/ tmp/meld/Python-2.6.2/Modules/tkappinit.o gcc -pthread -shared build/temp.linux-i686-2.6/tmp/meld/Python-2.6.2/ Modules/_tkinter.o build/temp.linux-i686-2.6/tmp/meld/Python-2.6.2/ Modules/tkappinit.o -L/usr/X11R6/lib64 -L/usr/X11R6/lib -L/usr/local/ lib -ltk8.5 -ltcl8.5 -lX11 -o build/lib.linux-i686-2.6/_tkinter.so *** WARNING: renaming _tkinter since importing it failed: libtk8.5.so: cannot open shared object file: No such file or directory Failed to find the necessary bits to build these modules: _sqlite3 bsddb185 sunaudiodev To find the necessary bits, look in setup.py in detect_modules() for the module's name. Failed to build these modules: _tkinter running build_scripts # cd /usr/local/lib # ls -la | grep libtk8.5.so -r-xr-xr-x 1 root root 1112606 Jul 10 13:28 libtk8.5.so Am I missing something, it’s there? Regards, -Tony -- http://mail.python.org/mailman/listinfo/python-list
Re: python make dies :libtk8.5.so: cannot open shared object file: No such file or directory
Tony Lay wrote: # cd /usr/local/lib # ls -la | grep libtk8.5.so -r-xr-xr-x 1 root root 1112606 Jul 10 13:28 libtk8.5.so Am I missing something, it’s there? Is /usr/local/lib in your library search path? It looks like it isn't. Check /etc/ld.so.conf and /etc/ld.so.conf.d/. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: Best Way to Handle All Exceptions
seldan24 wrote: Hello, I'm fairly new at Python so hopefully this question won't be too awful. I am writing some code that will FTP to a host, and want to catch any exception that may occur, take that and print it out (eventually put it into a log file and perform some alerting action). I've figured out two different ways to do this, and am wondering which is the best (i.e. cleanest, 'right' way to proceed). I'm also trying to understand exactly what occurs for each one. The first example: from ftplib import FTP try: ftp = FTP(ftp_host) ftp.login(ftp_user, ftp_pass) except Exception, err: print err This works fine. I read through the documentation, and my understanding is that there is a built-in exceptions module in python, that is automatically available in a built-in namespace. Within that module is an 'Exception' class which would contain whatever exception is thrown. So, I'm passing that to the except, along with err to hold the value and then print it out. The second example: from ftplib import FTP import sys try: ftp = FTP(ftp_host) ftp.login(ftp_user, ftp_pass) except: print sys.exc_info() Here I, for the most part, get the same thing. I'm not passing anything to except and just printing out the exception using a method defined in the sys module. So, I'm new to Python... I've made it this far and am happy, but want to make sure I'm coding correctly from the start. Which method is the better/cleaner/more standard way to continue? Thanks for any help. The latter is - unfortunately - the better. This comes from python allowing all kinds of objects being thrown as exceptions, not only those extending from a common ancestor like Exception. You seem to have a sensible take on this, but anyway I'd like to mention that using these kinds of catch-all code is rarely justified, as it imposes the danger of not handling errors at all. So make sure the program spits out a sensible error-message, and then dies. Unless it's a server-process of course. Diez -- http://mail.python.org/mailman/listinfo/python-list
Re: Problem with two instances of PySerial
On Thu, 9 Jul 2009 08:40:18 -0500, Shine Jose shinejos...@gmail.com wrote: I achieve this by creating a separate worker thread to poll the serial port for arrival of data and then updating the required widget in main thread of program. For polling the serial port, I create a separate instance of pySerial object. However, I am unable to read any data in the worker thread. Can the reason for this be 2 instances of pySerial objects being connected to serial port. Yes. The reason I had this doubt was because Serial.Connect() in the worker thread did not throw any exception isOpen() method returns true. Under Unix, you are allowed to open a device or file as many times as you want (there are some common cases where there is a very useful thing to do). However, for something like a serial port, there's only one copy of each received data byte. Depending on exactly how you structure your code, the incoming data may be divided up amongst multiple readers, or one particular reader may get all of it. -- Grant Edwards grante Yow! I joined scientology at at a garage sale!! visi.com -- http://mail.python.org/mailman/listinfo/python-list
Re: need to write a assembly progrm
On 2009-07-12, Rhodri James rho...@wildebst.demon.co.uk wrote: The poorly punctuated paragraph sounds, to me, as if they mean they want (Windows assumed) EXE files that can be dropped onto remote/laptop machines without performing a full installation of Python... But yes... assembly programs does, to me, also mean things containing mnemonics for machine level opcodes. Given that to me, assembly programs does not mean .EXE files at all, not even a little bit, I'm prefering to refuse to guess :-) Ah, but guessing what a poster meant to ask and then answering the guessed question is a pretty popular sport here in c.l.p. The discussions resulting from guessed questions are often more informative than the answer to the intended question (if the OP ever returns to clarify his intent). -- Grant Edwards grante Yow! Bo Derek ruined at my life! visi.com -- http://mail.python.org/mailman/listinfo/python-list
Re: python make dies :libtk8.5.so: cannot open shared object file: No such file or directory
On Jul 13, 10:09 am, Christian Heimes li...@cheimes.de wrote: Tony Lay wrote: # cd /usr/local/lib # ls -la | grep libtk8.5.so -r-xr-xr-x 1 root root 1112606 Jul 10 13:28 libtk8.5.so Am I missing something, it’s there? Is /usr/local/lib in your library search path? It looks like it isn't. Check /etc/ld.so.conf and /etc/ld.so.conf.d/. Christian I added /usr/local/lib to ld.so.conf.d/python26.conf and ran ldconfig (thanks Chrisitan) Ran a make distclean (had to move the files and got a “file not found” from my previous working directory). Everything compiled and installed like a charm. Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: Best Way to Handle All Exceptions
seldan24 wrote: Hello, I'm fairly new at Python so hopefully this question won't be too awful. I am writing some code that will FTP to a host, and want to catch any exception that may occur, take that and print it out (eventually put it into a log file and perform some alerting action). I've figured out two different ways to do this, and am wondering which is the best (i.e. cleanest, 'right' way to proceed). I'm also trying to understand exactly what occurs for each one. The first example: from ftplib import FTP try: ftp = FTP(ftp_host) ftp.login(ftp_user, ftp_pass) except Exception, err: print err This works fine. I read through the documentation, and my understanding is that there is a built-in exceptions module in python, that is automatically available in a built-in namespace. Within that module is an 'Exception' class which would contain whatever exception is thrown. So, I'm passing that to the except, along with err to hold the value and then print it out. There isn't an exceptions module; exceptions are part of the language. The second example: from ftplib import FTP import sys try: ftp = FTP(ftp_host) ftp.login(ftp_user, ftp_pass) except: print sys.exc_info() This is called a bare exception handler. It's virtually never the right way to do it. Here I, for the most part, get the same thing. I'm not passing anything to except and just printing out the exception using a method defined in the sys module. So, I'm new to Python... I've made it this far and am happy, but want to make sure I'm coding correctly from the start. Which method is the better/cleaner/more standard way to continue? Thanks for any help. You should use the most specific exception possible because at lot of things could raise an exception: foo Traceback (most recent call last): File pyshell#0, line 1, in module foo NameError: name 'foo' is not defined try: foo except Exception, e: print *** Caught an exception *** print e *** Caught an exception *** name 'foo' is not defined -- http://mail.python.org/mailman/listinfo/python-list
Re: Automate rsync w/ authentication
On Jul 10, 12:03 pm, mgi...@motorola.com (Gary Duzan) wrote: In article 3af970b1-b454-4d56-a33f-889ecfaca...@l28g2000vba.googlegroups.com, Bryan bryanv...@gmail.com wrote: rsyncExec = '/usr/bin/ssh' source = 'r...@10.0.45.67:/home/bry/jquery.lookup' dest = '/home/bry/tmp' rshArg = '-e /usr/bin/ssh -i /home/bry/keys/brybackup.key' args = [rsyncExec, '-a', '-v', '--dry-run', rshArg, source, dest] I think you want -e and the ssh command to be separate args. Something like: rshArg = '/usr/bin/ssh -i /home/bry/keys/brybackup.key' args = [rsyncExec, '-a', '-v', '--dry-run', '-e', rshArg, source, dest] or: rshArgs = [ '-e', '/usr/bin/ssh -i /home/bry/keys/brybackup.key' ] args = [rsyncExec, '-a', '-v', '--dry-run'] + rshArgs + [ source, dest] Gary Duzan Motorola HNM Separating the argument parts worked. Strangely though, I don't need to do that for arguments such as --files-from='/path/to/file'. Also, in this example code I had the rsync executable path pointing to the ssh program, so no wonder I was getting the output of ssh! -- http://mail.python.org/mailman/listinfo/python-list
Re: Best Way to Handle All Exceptions
On Jul 13, 10:33 am, MRAB pyt...@mrabarnett.plus.com wrote: seldan24 wrote: Hello, I'm fairly new at Python so hopefully this question won't be too awful. I am writing some code that will FTP to a host, and want to catch any exception that may occur, take that and print it out (eventually put it into a log file and perform some alerting action). I've figured out two different ways to do this, and am wondering which is the best (i.e. cleanest, 'right' way to proceed). I'm also trying to understand exactly what occurs for each one. The first example: from ftplib import FTP try: ftp = FTP(ftp_host) ftp.login(ftp_user, ftp_pass) except Exception, err: print err This works fine. I read through the documentation, and my understanding is that there is a built-in exceptions module in python, that is automatically available in a built-in namespace. Within that module is an 'Exception' class which would contain whatever exception is thrown. So, I'm passing that to the except, along with err to hold the value and then print it out. There isn't an exceptions module; exceptions are part of the language. The second example: from ftplib import FTP import sys try: ftp = FTP(ftp_host) ftp.login(ftp_user, ftp_pass) except: print sys.exc_info() This is called a bare exception handler. It's virtually never the right way to do it. Here I, for the most part, get the same thing. I'm not passing anything to except and just printing out the exception using a method defined in the sys module. So, I'm new to Python... I've made it this far and am happy, but want to make sure I'm coding correctly from the start. Which method is the better/cleaner/more standard way to continue? Thanks for any help. You should use the most specific exception possible because at lot of things could raise an exception: foo Traceback (most recent call last): File pyshell#0, line 1, in module foo NameError: name 'foo' is not defined try: foo except Exception, e: print *** Caught an exception *** print e *** Caught an exception *** name 'foo' is not defined- Hide quoted text - - Show quoted text - Thank you both for your input. I want to make sure I get started on the right track. For this particular script, I should have included that I would take the exception contents, and pass those to the logging module. For this particular script, all exceptions are fatal and I would want them to be. I just wanted a way to catch them and log them prior to program termination. I can understand why folks should always specify exact exceptions where possible if they plan on doing something with them, but I just want to log and terminate (and alarm) when any exception, irregardless of what it is, occurs; that's why I'm using the catch-all approach. I most likely should have put in an exit() in my snippet; sorry about that. I did notice that Python allows different objects to be thrown. This threw me off a bit because, at first, I was expecting everything to come through as nice, easy tuples. It turns out that ftplib throws some class from the sockets module (my terminology may be off here). As for terminology, sorry about the 'exceptions' misuse. The Python documentation refers to 'exceptions' as a module, albeit built-in, so I used that language accordingly (link to doc: http://docs.python.org/library/exceptions.html?highlight=exceptions#module-exceptions). Anyway, thanks again for the help and advice, it is greatly appreciated. -- http://mail.python.org/mailman/listinfo/python-list
Re: Best Way to Handle All Exceptions
On Jul 13, 2:26 pm, seldan24 selda...@gmail.com wrote: The first example: from ftplib import FTP try: ftp = FTP(ftp_host) ftp.login(ftp_user, ftp_pass) except Exception, err: print err *If* you really do want to catch *all* exceptions (as mentioned already it is usually better to catch specific exceptions) this is the way to do it. To know why you should look at the class hierarchy on http://docs.python.org/library/exceptions.html. The reason is that you almost never want to be catching SystemExit, KeyboardInterrupt etc. catching them will give you trouble at some point (unless you really know what you're doing but then I would suggest you list them explicitly instead of using the bare except statement). While it is true that you could raise an object that is not a subclass from Exception it is very bad practice, you should never do that. And I've haven't seen an external module in the wild that does that in years and the stdlib will always play nice. Regards Floris -- http://mail.python.org/mailman/listinfo/python-list
Passing python list from C to python
I'm very new at wrapping Python/C, and I have run into some problems. I have one python module that provides me with a list (provideBuffer in provideBuff.py): Py_Initialize(); pName = PyString_FromString(provideBuff); pModule = PyImport_Import(pName); pFunc = PyObject_GetAttrString(pModule,provideBuffer); pValue = PyObject_CallObject(pFunc,NULL); pValue is now a PyList - i've even verified this with: int a = PyList_Check(pValue); printf(%d\n, a); However, I want to send this PyList to another python module, but I don't know how to do this. Initially I though I could just do like above, only swapping NULL with pValue, but that is not working. pName2 = PyString_FromString(C-embedding); pModule2 = PyImport_Import(pName2); pFunc2 = PyObject_GetAttrString(pModule2,buff); pValue2 = PyObject_CallObject(pFunc2,pValue); pValue2 is now False! So i guess i cannot pass pValue as an argument to PyObject_CallObject when i want to pass an python list as an argument. But how must a go about to make this work? -- http://mail.python.org/mailman/listinfo/python-list
list of all possible values
Hi guys. I was thinking about a problem I had: suppose I have a list of possible values. I want to to have a list of all possible lists of length n whose values are in that original list. For example: if my values are ['a', 'b', 'c'], then all possible lists of length 2 would be: aa, ab, ac, ba, bb, bc, ca, cb, cc. I created a recursive program to do it, but I was wondering if there was a better way of doing it (possibly with list comprehensions). Here's my recursive version: vals = ['a', 'b', 'c'] def foo(length): if length =0: return [] if length == 1: return [[x] for x in vals] else: return [x + [y] for x in foo(length - 1) for y in vals] print foo(3) Thanks, David -- http://mail.python.org/mailman/listinfo/python-list
Re: list of all possible values
For example: if my values are ['a', 'b', 'c'], then all possible lists of length 2 would be: aa, ab, ac, ba, bb, bc, ca, cb, cc. I created a recursive program to do it, but I was wondering if there was a better way of doing it (possibly with list comprehensions). Here's my recursive version: vals = ['a', 'b', 'c'] def foo(length): if length =0: return [] if length == 1: return [[x] for x in vals] else: return [x + [y] for x in foo(length - 1) for y in vals] Sounds like you want one of the combinitoric generators found in itertools[1] -- in this case, the itertools.product() does what you describe. According to the docs, it was added in 2.6 so if you're running an older version, you'd have to back-port it. -tkc [1] http://docs.python.org/library/itertools.html#itertools.product -- http://mail.python.org/mailman/listinfo/python-list
Re: Passing python list from C to python
On Jul 14, 1:47 am, hartley hartle...@gmail.com wrote: I'm very new at wrapping Python/C, and I have run into some problems. I have one python module that provides me with a list (provideBuffer in provideBuff.py): Py_Initialize(); pName = PyString_FromString(provideBuff); pModule = PyImport_Import(pName); pFunc = PyObject_GetAttrString(pModule,provideBuffer); pValue = PyObject_CallObject(pFunc,NULL); pValue is now a PyList - i've even verified this with: int a = PyList_Check(pValue); printf(%d\n, a); However, I want to send this PyList to another python module, Please explain send ... do you mean the C equivalent of the Python statement C_embedding.buff = the_pylist ? BTW C-embedding would trigger a syntax error in Python source; best to avoid ... but I don't know how to do this. Initially I though I could just do like above, only swapping NULL with pValue, but that is not working. pName2 = PyString_FromString(C-embedding); pModule2 = PyImport_Import(pName2); pFunc2 = PyObject_GetAttrString(pModule2,buff); Get?? Do you want Set? Is buff a Python function? Or is it the destination of the sending? Any good reason for not checking the return value for an error? [Rhetorical question; answer == No] pValue2 = PyObject_CallObject(pFunc2,pValue); CallObject?? You used this before because you had a function and wanted to call it because it returned you a value now you want to do one of (in Python terms) value = amodule.anattr value = getattr(amodule, anattr) or amodule.anattr = value setattr(amodule, anattr, value) pValue2 is now False! False?? Do you mean the C NULL? So i guess i cannot pass pValue as an argument to PyObject_CallObject when i want to pass an python list as an argument. But how must a go about to make this work? It's mainly just a matter of (1) knowing what you want to do (2) picking the API that does what you want (3) checking the returned value for error after every call. HTH, John -- http://mail.python.org/mailman/listinfo/python-list
Re: list of all possible values
David Gibb:
For example: if my values are ['a', 'b', 'c'], then all possible lists
of length 2 would be: aa, ab, ac, ba, bb, bc, ca, cb, cc.
from itertools import product
list(product(abc, repeat=2))
[('a', 'a'), ('a', 'b'), ('a', 'c'), ('b', 'a'), ('b', 'b'), ('b',
'c'), ('c', 'a'), ('c', 'b'), ('c', 'c')]
Bye,
bearophile
--
http://mail.python.org/mailman/listinfo/python-list
Re: multiprocessing and dictionaries
On Monday 13 July 2009 01:56:08 Piet van Oostrum wrote:
Bjorn Meyer bjorn.m.me...@gmail.com (BM) wrote:
BM I am trying to convert a piece of code that I am using the thread
module with BM to the multiprocessing module.
BM The way that I have it set up is a chunk of code reads a text file and
assigns BM a dictionary key multiple values from the text file. I am
using locks to write BM the values to the dictionary.
BM The way that the values are written is as follows:
BM mydict.setdefault(key, []).append(value)
BM The problem that I have run into is that using multiprocessing, the
key gets BM set, but the values don't get appended.
BM I've even tried the Manager().dict() option, but it doesn't seem to
work.
BM Is this not supported at this time or am I missing something?
I think you should give more information. Try to make a *minimal* program
that shows the problem and include it in your posting or supply a
download link.
--
Piet van Oostrum p...@cs.uu.nl
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: p...@vanoostrum.org
Here is what I have been using as a test.
This pretty much mimics what I am trying to do.
I put both threading and multiprocessing in the example which shows the output
that I am looking for.
#!/usr/bin/env python
import threading
from multiprocessing import Manager, Process
name = ('test1','test2','test3')
data1 = ('dat1','dat2','dat3')
data2 = ('datA','datB','datC')
def thread_test(name,data1,data2, d):
for nam in name:
for num in range(0,3):
d.setdefault(nam, []).append(data1[num])
d.setdefault(nam, []).append(data2[num])
print 'Thread test dict:',d
def multiprocess_test(name,data1,data2, mydict):
for nam in name:
for num in range(0,3):
mydict.setdefault(nam, []).append(data1[num])
mydict.setdefault(nam, []).append(data2[num])
print 'Multiprocess test dic:',mydict
if __name__ == '__main__':
mgr = Manager()
md = mgr.dict()
d = {}
m = Process(target=multiprocess_test, args=(name,data1,data2,md))
m.start()
t = threading.Thread(target=thread_test, args=(name,data1,data2,d))
t.start()
m.join()
t.join()
print 'Thread test:',d
print 'Multiprocess test:',md
Thanks
Bjorn
--
http://mail.python.org/mailman/listinfo/python-list
Germany Ups Terrorism Alert Before Election
With Germany going to the polls in a general election in three months, authorities are on high alert after detecting an increase in online warnings of terrorist attacks targeting the country. The German government held high-level talks with top security and intelligence chiefs in Berlin on Thursday to discuss the growing threat posed by Islamic extremists and to coordinate counterterrorism measures. Intelligence officials are alarmed by the rising number of videos posted online by militant Islamists who say they are specifically targeting Germany. Up to 13 videos are reported to have appeared on the Web since January, many of them referring to the deployment of German troops in Afghanistan. (See pictures of U.S. Marines opening a new offensive in Afghanistan.) for more http://www.terrorismsearch.blogspot.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Blocking XMPP API?
Lawrence D'Oliveiro wrote: In message mailman.2893.1247147391.8015.python-l...@python.org, Gabriel Rossetti wrote: I am looking for blocking XMPP API. I'm wanting to make a webservice that uses XMPP, but all the XMPP libs I find are non-blocking (i.e. w/ callbacks). So why not have the caller sleep or something until the callback is invoked? It's easier to turn an async API into a synchronous one than the other way round. Yes, that is what I ended up doing, thanks for the response though :-) Gabriel -- http://mail.python.org/mailman/listinfo/python-list
Memory error due to big input file
Hi All, I have a similar problem that many new python users might encounter. I would really appreciate if you could help me fix the error. I have a big text file with size more than 2GB. It turned out memory error when reading in this file. Here is my python script, the error occurred at line -- self.fh.readlines(). import math import time class textfile: def __init__(self,fname): self.name=fname self.fh=open(fname) self.fh.readline() self.lines=self.fh.readlines() a=textfile(/home/sservice/nfbc/GenoData/CompareCalls3.diff) lfile=len(a.lines) def myfun(snp,start,end): subdata=a.lines[start:end+1] NEWmiss=0 OLDmiss=0 DIFF=0 for row in subdata: k=row.split() if (k[3]==0/0) (k[4]!=0/0): NEWmiss=NEWmiss+1 elif (k[3]!=0/0) (k[4]==0/0): OLDmiss=OLDmiss+1 elif (k[3]!=0/0) (k[4]!=0/0): DIFF=DIFF+1 result.write(snp+ +str(NEWmiss)+ +str(OLDmiss)+ +str(DIFF)+\n) result=open(Summary_noLoop_diff3.txt,w) result.write(SNP NEWmiss OLDmiss DIFF\n) start=0 snp=0 for i in range(lfile): if (i==0): continue after=a.lines[i].split() before=a.lines[i-1].split() if (before[0]==after[0]): if (i!=(lfile-1)): continue else: end=lfile-1 myfun(before[0],start,end) snp=snp+1 else: end=i-1 myfun(before[0],start,end) snp=snp+1 start=i if (i ==(lfile-1)): myfun(after[0],start,start) snp=snp+1 result.close() sincerely, phoebe -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory error due to big input file
sityee kong wrote: Hi All, I have a similar problem that many new python users might encounter. I would really appreciate if you could help me fix the error. I have a big text file with size more than 2GB. It turned out memory error when reading in this file. Here is my python script, the error occurred at line -- self.fh.readlines(). [snip code] Your 'error' is that you're running it on a computer with insufficient memory. -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory error due to big input file
phoebe I have a big text file with size more than 2GB. It turned out phoebe memory error when reading in this file. Here is my python phoebe script, the error occurred at line -- self.fh.readlines(). phoebe import math phoebe import time phoebe class textfile: phoebe def __init__(self,fname): phoebe self.name=fname phoebe self.fh=open(fname) phoebe self.fh.readline() phoebe self.lines=self.fh.readlines() Don't do that. The problem is that you are trying to read the entire file into memory. Learn to operate a line (or a few lines) at a time. Try something like: a = open(/home/sservice/nfbc/GenoData/CompareCalls3.diff) for line in a: do your per-line work here -- Skip Montanaro - s...@pobox.com - http://www.smontanaro.net/ when i wake up with a heart rate below 40, i head right for the espresso machine. -- chaos @ forums.usms.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Threading.Condition problem
Piet van Oostrum wrote:
Gabriel Rossetti gabriel.rosse...@arimaz.com (GR) wrote:
GR Sorry if this appears twice, I sent it once with an attachment and it never
GR arrived so maybe the attachment is posing problems. I inlined the code this
GR time (at the bottom), thank you,
GR Gabriel
GR ## Original message
GR Hello everyone,
GR I wrote a small example that listens for xmpp msgs in a thread. The main
GR program calls a function that blocks (using Condition.wait) until a msg
GR has been received and then returns the msg. When a msg arrives, it is
GR put in a variable in the thread's object, it then calls the notify()
GR attr on the Condition object. For some reason, this doesn't work, the
GR thread gets the msg, tries to notify the Condition object, fails because
GR the lock has not been acquired yet and blocks. I tried ignoring the
GR failure, thinking that since it has not been acquired yet then when it
GR is, it will get the msg right away and never call Condition.wait, thus
GR not causing any problems, but this does not work either. Does someone
GR know what I am doing wrong? I attached the code to this msg.
The code that puts the message in the variable should also acquire the
lock:
def onMessage(self, conn, msg):
with self._cv:
self.message = msg
self._cv.notify()
Thank you, that was the problem, I eventually found that
A couple of remarks:
1. I think the code is neater if all manipulation with the condition is
done in the same class (actually in the same instance -- making this
instance into a monitor).
The reason I didn't do that is that I don' t want the Listener to sleep,
I maybe over simplified the example, I actually put them in a dictionary
as they come in, so in your example, if I have several threads waiting
on msgs it wouldn't work. I'm trying to make a webservice api thay will
also be turned into a java .jar for people that need java. Now that I
think about it, each session will have an instance of the object so msgs
shouldn' t get mixed up (one connection per user), so I could block in
the thread. I'll try your suggestion as I think it is cleaner.
class Listener(Thread):
def __init__(self, ws):
Thread.__init__(self)
self.interrupt = Event()
self.message = None
self._cv = Condition()
self.client = ws._client
self.client.RegisterHandler('message', self.onMessage)
def onMessage(self, conn, msg):
with self._cv:
self.message = msg
try:
self._cv.notify()
except RuntimeError:
print self._cv has not acquired the lock yet
def getMsg(self):
with self._cv:
while !self.message
self._cv.wait()
return self.message
class WS(object):
def __init__(self, username, password, res):
self._jid = xmpp.protocol.JID(username)
self._client = xmpp.Client(self._jid.getDomain())
#self._cv = Condition()
def getMsg(self, mid=None):
return self._listener.getMsg()
Of course I haven't tested this code as I don't have the context
modules.
2. I don't know if more than one message can be delivered in the same
instance. If yes, than your code will not work, and neither will the
code above as, the message instance variable is never cleared. So the
next getMsg will be happy to deliver the previous one.
You would have to clear it when returning this one.
Like I said above, in reality I have a dict not just a simple variable.
def getMsg(self):
with self._cv:
while !self.message
self._cv.wait()
msg = self.message
self.message = None
return msg
3. If the messages come in faster than they can be processed some will
be lost as they will overwrite the previous one in the self.message
variable. The solution is to use a threading.Queue to transfer the
messages from one thread to the other. This also saves you the hassle
of doing your own synchronisation like above. If you are not familiar
with synchronising multithreaded applications it is very easy to make
errors and even if you are it is quite easy to do them wrong. I have
been involved in distributed programming courses at university level
and I have seen many errors in this area.
I used a dict because the API can also be setup to be async and use
callbacks, so I had to be able to
access the msgs directly and quickly.
Gabriel
--
http://mail.python.org/mailman/listinfo/python-list
Re: list of all possible values
On 7/13/2009 9:33 AM Tim Chase said... For example: if my values are ['a', 'b', 'c'], then all possible lists of length 2 would be: aa, ab, ac, ba, bb, bc, ca, cb, cc. I created a recursive program to do it, but I was wondering if there was a better way of doing it (possibly with list comprehensions). Here's my recursive version: vals = ['a', 'b', 'c'] def foo(length): if length =0: return [] if length == 1: return [[x] for x in vals] else: return [x + [y] for x in foo(length - 1) for y in vals] Sounds like you want one of the combinitoric generators found in itertools[1] -- in this case, the itertools.product() does what you describe. According to the docs, it was added in 2.6 so if you're running an older version, you'd have to back-port it Or on systems with list comps try: V='abc' ['%s%s'%(ii,jj) for ii in V for jj in V] ['aa', 'ab', 'ac', 'ba', 'bb', 'bc', 'ca', 'cb', 'cc'] Emile -- http://mail.python.org/mailman/listinfo/python-list
How to override Marshaller/Unmarshaller in xmlrpc?
Is there a way to override marshaller in xmlrpc.client properly? For example, I want to hide bytes - Binary transformation inside marshaller (i.e., just let user pass a bytes value to function and marshaller dumps it in base64 autmatically). Unfortunately, I cannot just make a derived class with necessary dump_* function and pass it to client and server; marshaller initialization seems to be hidden inside xmlrpc. So, here's the question - is there a nice way to do it (because there are some dirty ways like rewriting getparser() ans so on, but I don't like the idea of copypasting code from library)? Or should I reconsider my design instead? Thanks in advance. -- http://mail.python.org/mailman/listinfo/python-list
Re: Best Way to Handle All Exceptions
2009/7/13 seldan24 selda...@gmail.com: Thank you both for your input. I want to make sure I get started on the right track. For this particular script, I should have included that I would take the exception contents, and pass those to the logging module. For this particular script, all exceptions are fatal and I would want them to be. I just wanted a way to catch them and log them prior to program termination. The logging module has a function specifically to handle this case: try: # do something except: logging.exception(Uh oh...) The exception() method will automatically add details of the exception to the log message it creates; as per the docs, you should only call it from within an exception handler. Hope that helps, Vil. -- http://mail.python.org/mailman/listinfo/python-list
Re: list of all possible values
Or on systems with list comps try: V='abc' ['%s%s'%(ii,jj) for ii in V for jj in V] ['aa', 'ab', 'ac', 'ba', 'bb', 'bc', 'ca', 'cb', 'cc'] Yeah, except that the length here is hard-coded. There's no way (as far as I can tell, at least), to make this generic with respect to list length. Thanks for the itertools suggestion, guys. I was hoping to improve my list-comprehension-fu, but that module was the next best thing. David On Mon, Jul 13, 2009 at 1:58 PM, Emile van Sebilleem...@fenx.com wrote: On 7/13/2009 9:33 AM Tim Chase said... For example: if my values are ['a', 'b', 'c'], then all possible lists of length 2 would be: aa, ab, ac, ba, bb, bc, ca, cb, cc. I created a recursive program to do it, but I was wondering if there was a better way of doing it (possibly with list comprehensions). Here's my recursive version: vals = ['a', 'b', 'c'] def foo(length): if length =0: return [] if length == 1: return [[x] for x in vals] else: return [x + [y] for x in foo(length - 1) for y in vals] Sounds like you want one of the combinitoric generators found in itertools[1] -- in this case, the itertools.product() does what you describe. According to the docs, it was added in 2.6 so if you're running an older version, you'd have to back-port it Emile -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Efficient binary search tree stored in a flat array?
I couldn't find a good algorithms forum on the Internet, so I guess I'll ask this question here instead: Is it possible to efficiently maintain a binary search tree in a flat array (i.e., without using pointers), as is typically done for a binary heap? It *is* possible, of course, to keep an ordered list in a flat array, and then do a binary search on the ordered array, but then insertion and deletion are O(n), rather than O(log n). It would also clearly be possible to store a balanced binary tree in a flat array, storing the children of the node at index i at indices 2*i and 2*i + 1, just as one does for a binary heap. But if you do that, I don't know if you could then do insertions and deletions in O(log n) time. One idea that came to mind, is that maybe it is possible using a treap, which is a combination of a binary heap and a binary search tree. Insertions and deletions in binary heaps can be done in O(log n) in a flat array, but I don't know if this is also true for a treap, since you also have the binary search tree invariants to maintain, in addition to the binary heap invariants. For all I know, this might cause rotations to no longer be O(log n). |ouglas -- http://mail.python.org/mailman/listinfo/python-list
Re: Automate rsync w/ authentication
Bryan bryanv...@gmail.com (B) wrote: B On Jul 10, 12:03 pm, mgi...@motorola.com (Gary Duzan) wrote: In article 3af970b1-b454-4d56-a33f-889ecfaca...@l28g2000vba.googlegroups.com, Bryan bryanv...@gmail.com wrote: rsyncExec = '/usr/bin/ssh' source = 'r...@10.0.45.67:/home/bry/jquery.lookup' dest = '/home/bry/tmp' rshArg = '-e /usr/bin/ssh -i /home/bry/keys/brybackup.key' args = [rsyncExec, '-a', '-v', '--dry-run', rshArg, source, dest] I think you want -e and the ssh command to be separate args. Something like: rshArg = '/usr/bin/ssh -i /home/bry/keys/brybackup.key' args = [rsyncExec, '-a', '-v', '--dry-run', '-e', rshArg, source, dest] or: rshArgs = [ '-e', '/usr/bin/ssh -i /home/bry/keys/brybackup.key' ] args = [rsyncExec, '-a', '-v', '--dry-run'] + rshArgs + [ source, dest] Gary Duzan Motorola HNM B Separating the argument parts worked. Strangely though, I don't need B to do that for arguments such as --files-from='/path/to/file'. Also, B in this example code I had the rsync executable path pointing to the B ssh program, so no wonder I was getting the output of ssh! I should have seen that because I changed it in my own copy!!! --files-from='/path/to/file *is* one argument, in contrast to -e /usr/bin/ssh -i /home/bry/keys/brybackup.key which is two arguments. -- Piet van Oostrum p...@cs.uu.nl URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4] Private email: p...@vanoostrum.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Threading.Condition problem
Gabriel Rossetti gabriel.rosse...@arimaz.com (GR) wrote: GR Piet van Oostrum wrote: ... GR I wrote a small example that listens for xmpp msgs in a thread. The main GR program calls a function that blocks (using Condition.wait) until a msg GR has been received and then returns the msg. When a msg arrives, it is GR put in a variable in the thread's object, it then calls the notify() GR attr on the Condition object. For some reason, this doesn't work, the GR thread gets the msg, tries to notify the Condition object, fails because GR the lock has not been acquired yet and blocks. I tried ignoring the GR failure, thinking that since it has not been acquired yet then when it GR is, it will get the msg right away and never call Condition.wait, thus GR not causing any problems, but this does not work either. Does someone GR know what I am doing wrong? I attached the code to this msg. The code that puts the message in the variable should also acquire the lock: def onMessage(self, conn, msg): with self._cv: self.message = msg self._cv.notify() GR Thank you, that was the problem, I eventually found that A couple of remarks: 1. I think the code is neater if all manipulation with the condition is done in the same class (actually in the same instance -- making this instance into a monitor). GR The reason I didn't do that is that I don' t want the Listener to sleep, I GR maybe over simplified the example, I actually put them in a dictionary as GR they come in, so in your example, if I have several threads waiting on msgs GR it wouldn't work. I'm trying to make a webservice api thay will also be GR turned into a java .jar for people that need java. Now that I think about GR it, each session will have an instance of the object so msgs shouldn' t get GR mixed up (one connection per user), so I could block in the thread. I'll GR try your suggestion as I think it is cleaner. Sleeping as you call it is better than busy waiting. You must have some synchronisation to make it efficient. If you put the messages in a dictionary access to the dictionary must be protected. Having several threads waiting for the messages doesn't prevent you from using proper synchronisation. Maybe you must use notify_all instead of notify. -- Piet van Oostrum p...@cs.uu.nl URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4] Private email: p...@vanoostrum.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Efficient binary search tree stored in a flat array?
On Jul 13, 3:57 pm, a...@pythoncraft.com (Aahz) wrote: Still, unless your list is large (more than thousands of elements), that's the way you should go. See the bisect module. Thing is, the speed difference between C and Python means the constant for insertion and deletion is very very small relative to bytecode speed. Keep in mind that Python's object/binding model means that you're shuffling pointers in the list rather than items. Thank you. My question wasn't intended to be Python specific, though. I am just curious for purely academic reasons about whether there is such an algorithm. All the sources I've skimmed only seem to the answer the question via omission. Which is kind of strange, since it seems to me like an obvious question to ask. If I find the free time, I might try to work out myself whether it can be done with a treap. |ouglas -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory error due to big input file
sityee kong wrote: Hi All, I have a similar problem that many new python users might encounter. I would really appreciate if you could help me fix the error. I have a big text file with size more than 2GB. It turned out memory error when reading in this file. Here is my python script, the error occurred at line -- self.fh.readlines(). import math import time class textfile: def __init__(self,fname): self.name=fname self.fh=open(fname) self.fh.readline() self.lines=self.fh.readlines() a=textfile(/home/sservice/nfbc/GenoData/CompareCalls3.diff) lfile=len(a.lines) def myfun(snp,start,end): subdata=a.lines[start:end+1] NEWmiss=0 OLDmiss=0 DIFF=0 for row in subdata: k=row.split() if (k[3]==0/0) (k[4]!=0/0): NEWmiss=NEWmiss+1 elif (k[3]!=0/0) (k[4]==0/0): OLDmiss=OLDmiss+1 elif (k[3]!=0/0) (k[4]!=0/0): DIFF=DIFF+1 result.write(snp+ +str(NEWmiss)+ +str(OLDmiss)+ +str(DIFF)+\n) result=open(Summary_noLoop_diff3.txt,w) result.write(SNP NEWmiss OLDmiss DIFF\n) start=0 snp=0 for i in range(lfile): if (i==0): continue after=a.lines[i].split() before=a.lines[i-1].split() if (before[0]==after[0]): if (i!=(lfile-1)): continue else: end=lfile-1 myfun(before[0],start,end) snp=snp+1 else: end=i-1 myfun(before[0],start,end) snp=snp+1 start=i if (i ==(lfile-1)): myfun(after[0],start,start) snp=snp+1 result.close() sincerely, phoebe Others have pointed out that you have too little memory for a 2gig data structure. If you're running on a 32bit system, chances are it won't matter how much memory you add, a process is limited to 4gb, and the OS typically takes about half of it, your code and other data takes some, and you don't have 2gig left. A 64 bit version of Python, running on a 64bit OS, might be able to just work. Anyway, loading the whole file into a list is seldom the best answer, except for files under a meg or so. It's usually better to process the file in sequence. It looks like you're also making slices of that data, so they could potentially be pretty big as well. If you can be assured that you only need the current line and the previous two (for example), then you can use a list of just those three, and delete the oldest one, and add a new one to that list each time through the loop. Or, you can add some methods to that 'textfile' class that fetch a line by index. Brute force, you could pre-scan the file, and record all the file offsets for the lines you find, rather than storing the actual line. So you still have just as big a list, but it's a list of integers. Then when somebody calls your method, he passes an integer, and you return the particular line. A little caching for performance, and you're good to go. Anyway, if you organize it that way, you can code the rest of the module to not care whether the whole file is really in memory or not. BTW, you should derive all your classes from something. If nothing else, use object. class textfile(object): -- http://mail.python.org/mailman/listinfo/python-list
Pickling classes -- disappearing lists?
I'm trying to pickle an instance of a class. It mostly works just fine
-- I can save the pickle to a file, restore it from that file, and
it's mostly okay. The problem is, some lists seem to disappear. For
example (snipped and crunched from the code giving me trouble):
---
class InitGame:
value = False
journal = []
game.InitGame()
def Save():
global game
import cPickle, gzip, os
# Change some data
game.journal.append(A value)
game.value = True
pickles = {'game': game}
jar = gzip.open(pickefile, 'wb')
cPickle.dump(pickles, jar, 2)
jar.close()
def Load():
global game
import gzip, os, cPickle
jar = gzip.open(picklefile, 'r')
loaded = cPickle.load(jar)
jar.close()
game = loaded[game]
---
Now, if I save a pickle, then load it back in, I'll get an instance of
InitGame called game, and game.value will be true, but the list
journal will be empty.
Am I missing something obvious about the pickle spec? Am I doing
something wrong? Or should I be hunting for a deeper bug in the code?
I've noticed that pretty much anything that's a list isn't saving to
the pickle (or loading from the pickle... it's hard to tell which).
--
http://mail.python.org/mailman/listinfo/python-list
Re: Memory error due to big input file
BTW, you should derive all your classes from something. If nothing else, use object. class textfile(object): Just out of curiousity... why is that? I've been coding in Python for a long time, and I never derive my base classes. What's the advantage to deriving them? -- http://mail.python.org/mailman/listinfo/python-list
PDF: finding a blank image
I have about 6000 PDF files which have been produced using a scanner with more being produced each day. The PDF files contain old paper records which have been taking up space. The scanner is set to detect when there is information on the backside of the page (duplex scan). The problem of course is it's not the always reliable and we wind up with a number of PDF files containing blank pages. What I would like to do is have python detect a blank pages in a PDF file and remove it. Any suggestions? Thanks, DrL -- http://mail.python.org/mailman/listinfo/python-list
Re: Pickling classes -- disappearing lists?
On Mon, Jul 13, 2009 at 2:14 PM, Aaron Scottaaron.hildebra...@gmail.com wrote:
I'm trying to pickle an instance of a class. It mostly works just fine
-- I can save the pickle to a file, restore it from that file, and
it's mostly okay. The problem is, some lists seem to disappear. For
example (snipped and crunched from the code giving me trouble):
---
class InitGame:
value = False
journal = []
game.InitGame()
That line doesn't make sense with the code you've given...
def Save():
global game
import cPickle, gzip, os
# Change some data
game.journal.append(A value)
game.value = True
pickles = {'game': game}
jar = gzip.open(pickefile, 'wb')
cPickle.dump(pickles, jar, 2)
jar.close()
def Load():
global game
import gzip, os, cPickle
jar = gzip.open(picklefile, 'r')
loaded = cPickle.load(jar)
jar.close()
game = loaded[game]
---
Now, if I save a pickle, then load it back in, I'll get an instance of
InitGame called game, and game.value will be true, but the list
journal will be empty.
Am I missing something obvious about the pickle spec? Am I doing
something wrong? Or should I be hunting for a deeper bug in the code?
Your class definition isn't right. It makes 'value' and 'journal'
class variables (Java lingo: static variables) as opposed to the
instance variables they should be. Here's a corrected version:
class InitGame(object):
def __init__(self):
#instance variables are created through self.foo assignments in __init__
self.value = False
self.journal = []
Cheers,
Chris
--
http://blog.rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: Memory error due to big input file
2009/7/13 Aaron Scott aaron.hildebra...@gmail.com: BTW, you should derive all your classes from something. If nothing else, use object. class textfile(object): Just out of curiousity... why is that? I've been coding in Python for a long time, and I never derive my base classes. What's the advantage to deriving them? class Foo: uses the old object model. class Foo(object): uses the new object model. See http://docs.python.org/reference/datamodel.html (specifically section 3.3) for details of the differences. Vil. -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory error due to big input file
On Mon, Jul 13, 2009 at 2:51 PM, Vilya Harveyvilya.har...@gmail.com wrote: 2009/7/13 Aaron Scott aaron.hildebra...@gmail.com: BTW, you should derive all your classes from something. If nothing else, use object. class textfile(object): Just out of curiousity... why is that? I've been coding in Python for a long time, and I never derive my base classes. What's the advantage to deriving them? class Foo: uses the old object model. class Foo(object): uses the new object model. See http://docs.python.org/reference/datamodel.html (specifically section 3.3) for details of the differences. Note that Python 3.0 makes explicitly subclassing `object` unnecessary since it removes old-style classes; a class that doesn't explicitly subclass anything will implicitly subclass `object`. Cheers, Chris -- http://blog.rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Pickling classes -- disappearing lists?
Your class definition isn't right. It makes 'value' and 'journal' class variables (Java lingo: static variables) as opposed to the instance variables they should be. Here's a corrected version: Woah, thanks. I can't believe I made such a stupid mistake. It's not like I'm a newcomer to Python, either. I'm can't believe I never noticed what I was doing. No more 2am coding for me. Thanks, Aaron -- http://mail.python.org/mailman/listinfo/python-list
Re: PDF: finding a blank image
DrLeif l.lensg...@gmail.com writes: What I would like to do is have python detect a blank pages in a PDF file and remove it. Any suggestions? The odds are good that even a blank page is being rendered within the PDF as having some small bits of data due to scanner resolution, imperfections on the page, etc.. So I suspect you won't be able to just look for a well-defined pattern in the resulting PDF or anything. Unless you're using OCR, the odds are good that the scanner is rendering the PDF as an embedded image. What I'd probably do is extract the image of the page, and then use image processing on it to try to identify blank pages. I haven't had the need to do this myself, and tool availability would depend on platform, but for example, I'd probably try ImageMagick's convert operation to turn the PDF into images (like PNGs). I think Gimp can also do a similar conversion, but you'd probably have to script it yourself. Once you have an image of a page, you could then use something like OpenCV to process the page (perhaps a morphology operation to remove small noise areas, then a threshold or non-zero counter to judge blankness), or probably just something like PIL depending on complexity of the processing needed. Once you identify a blank page, removing it could either be with pure Python (there have been other posts recently about PDF libraries) or with external tools (such as pdftk under Linux for example). -- David -- http://mail.python.org/mailman/listinfo/python-list
Re: PDF: finding a blank image
DrLeif wrote: I have about 6000 PDF files which have been produced using a scanner with more being produced each day. The PDF files contain old paper records which have been taking up space. The scanner is set to detect when there is information on the backside of the page (duplex scan). The problem of course is it's not the always reliable and we wind up with a number of PDF files containing blank pages. What I would like to do is have python detect a blank pages in a PDF file and remove it. Any suggestions? I'd check into ReportLab's commercial product, it may well be easily capable of that. If no success, you might contact PJ at Groklaw, she has dealt with a _lot_ of PDFs (and knows people who deal with PDFs in bulk). --Scott David Daniels scott.dani...@acm.org -- http://mail.python.org/mailman/listinfo/python-list
Re: How to check if any item from a list of strings is in a big string?
En Mon, 13 Jul 2009 10:11:09 -0300, denis denis-bz...@t-online.de escribió: Matt, how many words are you looking for, in how long a string ? Were you able to time any( substr in long_string ) against re.compile ( |.join( list_items )) ? There is a known algorithm to solve specifically this problem (Aho-Corasick), a good implementation should perform better than R.E. (and better than the gen.expr. with the advantage of returning WHICH string matched) There is a C extension somewhere implementing Aho-Corasick. -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: explode()
On Sat, 11 Jul 2009 18:50:28 -0700, Chris Rebert c...@rebertia.com wrote: On Sat, Jul 11, 2009 at 7:41 PM, Fred Atkinsonfatkin...@mishmash.com wrote: What is the Python equivalent of the PHP explode() function? some_string.split(separator) Cheers, Chris What would the equivalent of the PHP function implode be? Thanks, Fred -- http://mail.python.org/mailman/listinfo/python-list
Re: explode()
On Mon, Jul 13, 2009 at 11:28 PM, Fred Atkinsonfatkin...@mishmash.com wrote: On Sat, 11 Jul 2009 18:50:28 -0700, Chris Rebert c...@rebertia.com wrote: On Sat, Jul 11, 2009 at 7:41 PM, Fred Atkinsonfatkin...@mishmash.com wrote: What is the Python equivalent of the PHP explode() function? some_string.split(separator) Cheers, Chris What would the equivalent of the PHP function implode be? separator.join(a_list_of_strings) I would recommend you read the Python manual's section on string methods: http://docs.python.org/library/stdtypes.html#string-methods Cheers, Chris -- http://blog.rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Re: explode()
On Mon, 13 Jul 2009 22:32:55 -0700, Chris Rebert c...@rebertia.com wrote: On Mon, Jul 13, 2009 at 11:28 PM, Fred Atkinsonfatkin...@mishmash.com wrote: On Sat, 11 Jul 2009 18:50:28 -0700, Chris Rebert c...@rebertia.com wrote: On Sat, Jul 11, 2009 at 7:41 PM, Fred Atkinsonfatkin...@mishmash.com wrote: What is the Python equivalent of the PHP explode() function? some_string.split(separator) Cheers, Chris What would the equivalent of the PHP function implode be? separator.join(a_list_of_strings) I would recommend you read the Python manual's section on string methods: http://docs.python.org/library/stdtypes.html#string-methods Cheers, Chris Chris, You always seem to be waiting when I post a question. Do you ever sleep? I wish the Python site was as well written as the PHP site. On the PHP site, I can look up a command and they show not only the docs on that command but a list of all other commands associated with it. Thanks. Python is a whole new animal to me. I'm taking a course in PHP and Python online right now. I've done PHP scripting before though I don't claim to be a whiz on it. But I'd barely heard of Python before I took this course. The one thing I really dislike about Python over PHP is that Python can usually only appear in the cgi directory (unless other arragements are made with your hosting provider or if you reconfigure Apache on your own server if you have your own). With PHP, I can put them in any folder on my Web site without any problem. Regards, Fred -- http://mail.python.org/mailman/listinfo/python-list
python _binary_ code
Hi Can anyone give me solution to create a python binary file (bytecode) other than pyc file .So my source code be secure.. I am working on ubuntu 9.04 with python2.6.. I Regards Sanju -- http://mail.python.org/mailman/listinfo/python-list
Re: explode()
On 2009-07-14, Fred Atkinson fatkin...@mishmash.com wrote: The one thing I really dislike about Python over PHP is that Python can usually only appear in the cgi directory (unless other arragements are made with your hosting provider or if you reconfigure Apache on your own server if you have your own). With PHP, I can put them in any folder on my Web site without any problem. That is a server configuration and has nothing to do with Python directly. -- http://mail.python.org/mailman/listinfo/python-list
[issue6071] no longer possible to hash arrays
ivank i...@ludios.org added the comment: It no longer works with objects created with buffer() either: hashlib.sha1(buffer(x)).hexdigest() Traceback (most recent call last): File stdin, line 1, in module TypeError: object supporting the buffer API required -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6071 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6457] subprocess.Popen.communicate can lose data from output/error streams when broken input pipe occures
Amaury Forgeot d'Arc amaur...@gmail.com added the comment: Testing on Debian with latest trunk: - the proposed example code works very well (I get all data). - I added subprocess._has_poll = False, and some (sometimes all) data is lost. It seems that select() will return stdin even if it is not writable. On Windows of course, communicate() uses a blocking write, and always fail. The proposed patch ignore the errors when EPIPE is raised, and simply stops writing. -- keywords: +needs review, patch nosy: +amaury.forgeotdarc stage: - patch review Added file: http://bugs.python.org/file14493/broken_pipe.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6457 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1820] Enhance Object/structseq.c to match namedtuple and tuple api
Marc-Andre Lemburg m...@egenix.com added the comment:
Raymond Hettinger wrote:
Raymond Hettinger rhettin...@users.sourceforge.net added the comment:
In Py3.x, this fails:
%s.%s.%s-%s-%s % sys.version_info
The reason is that PyUnicode_Format() expects a real tuple, not a tuple
lookalike. The fix is to either have structseq inherit from tuple or to
modify PyUnicode_Format() to handle structseq:
if (PyCheck_StructSeq(args)) {
newargs = PyTuple_FromSequence(args);
if (newargs == NULL)
return NULL;
result = PyUncode_Format(format, newargs);
Py_DECREF(newargs);
return result;
}
-1
The special-casing of tuples vs. non-tuples for % is already
bad enough. Adding structseq as another special case doesn't
make that any better.
What's so hard about writing
%s.%s.%s-%s-%s % tuple(sys.version_info)
anyway ?
--
nosy: +lemburg
title: Enhance Object/structseq.c to match namedtuple and tuple api - Enhance
Object/structseq.c to match namedtuple and tuple api
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1820
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6473] hmac sha384/sha512 fails test vectors
Changes by Antoine Pitrou pit...@free.fr: -- priority: - critical stage: - needs patch versions: +Python 2.6, Python 2.7, Python 3.1, Python 3.2 -Python 2.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6473 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5753] CVE-2008-5983 python: untrusted python modules search path
Tomas Hoger tho...@redhat.com added the comment: Additional API has one disadvantage - it requires a modification of all affected applications embedding python, which is not likely to happen soon after the API is introduced. Therefore, it may still be worth reviewing current behaviour (that seemed to have had no documentation until recently, see issue #5144, and can probably still benefit from more warnings related to the embedded use) in this corner case (argv0 is bogus and contains no '/') to see if it may be worth changing in future python versions. As for command line flags, I presume you're referring to the 'wcscmp(argv0, L-c)' part of the patch. It's not more than a re-use of the pattern already used couple of times in the PySys_SetArgv, that got added via: http://svn.python.org/view?view=revrevision=39544 Again, it's an attempt to make sure this only changes behaviour in rather specific case. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5753 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6469] Function definition expressions feature
Chris Perkins chrisperkin...@gmail.com added the comment: Details and discussion: http://groups.google.com/group/python-ideas/browse_frm/thread/a2bb1ff3992d9c01 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6469 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6444] multiline exception logging via syslog handler
Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: Why can't you either use %r in the format string if you know you'll be printing out multiple lines, or define your own handler/formatter to do exactly what you want? I can't see how to provide what you need in a generally useful way in the stdlib. -- assignee: - vsajip nosy: +vsajip status: open - pending ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6444 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6458] With logging.config.fileConfig can't create logger without handler
Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: Just set up a handlers= line in the relevant sections. You don't have to actually specify any handlers. -- nosy: +vsajip resolution: - invalid status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6458 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6314] logging.basicConfig(level='DEBUG', ... and setLevel(DEBUG) result in no logging
Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: Fixed checked into trunk py3k as per msg90214. -- resolution: - fixed status: open - pending ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6314 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6457] subprocess.Popen.communicate can lose data from output/error streams when broken input pipe occures
Dariusz Walczak dariusz.walc...@gmail.com added the comment: Your patch works for me with one exception: On FreeBSD (python 2.6.1) OSError exception is raised by os.write instead of IOError exception (raised by self.stdin.write on Windows). The error code is same on both platforms. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6457 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6457] subprocess.Popen.communicate can lose data from output/error streams when broken input pipe occures
Amaury Forgeot d'Arc amaur...@gmail.com added the comment: Right, file.write raises IOError when os.write raises OSError :-( Updated the patch. -- Added file: http://bugs.python.org/file14494/broken_pipe-2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6457 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5753] CVE-2008-5983 python: untrusted python modules search path
Antoine Pitrou pit...@free.fr added the comment: Indeed, it would certainly be useful to review current behaviour and document it precisely; and then, perhaps change it in order to fix the current bug. The problem is that the current behaviour seems to have evolved quite organically, and it's not obvious who relies on what (as I said, Python has many users). I'm not myself motivated in doing such a research. Perhaps other developers can chime in. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5753 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5753] CVE-2008-5983 python: untrusted python modules search path
Antoine Pitrou pit...@free.fr added the comment: Besides, the new API makes the behaviour more explicit and puts the decision in the hands of the embedding developer (which certainly knows better than us what he wants to do). As the Python Zen says: In the face of ambiguity, refuse the temptation to guess. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5753 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6469] Function definition expressions feature
R. David Murray rdmur...@bitdance.com added the comment: Even assuming you'd gotten a rough consensus on python-ideas (which it does not appear that you have), a change of this nature would require a PEP, and is thus not suitable for the tracker. -- nosy: +r.david.murray priority: - normal resolution: - rejected stage: - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6469 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6433] multiprocessing: pool.map hangs on empty list
Eric Eisner e...@mit.edu added the comment: Can anyone review this patch? It is a very simple fix to catch this one edge case. As this can cause multiprocessing to enter a state where it needs to be externally killed, it would be good if this made 2.6.3. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6433 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6433] multiprocessing: pool.map hangs on empty list
Jesse Noller jnol...@gmail.com added the comment: It's on my list for this week. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6433 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6474] Inconsistent TypeError message on function calls with wrong number of arguments
New submission from Hagen Fürstenau hfuerste...@gmx.net: I think the following error messages are inconsistent and confusing: def f(a, b): pass ... f(a=1) Traceback (most recent call last): File stdin, line 1, in module TypeError: f() takes exactly 2 non-keyword positional arguments (1 given) f(b=1) Traceback (most recent call last): File stdin, line 1, in module TypeError: f() takes exactly 2 non-keyword positional arguments (0 given) Strictly speaking, no positional arguments are given in either case, so it should say (0 given) in both cases. On the other hand, the given keyword arguments are filled into the positional argument slots, so stating something like (1 missing) or (1 unspecified) in both cases seems to make more sense. Any opinions? -- components: Interpreter Core messages: 90485 nosy: hagen severity: normal status: open title: Inconsistent TypeError message on function calls with wrong number of arguments type: behavior versions: Python 2.7, Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6474 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6475] Documentation Tutorial List Comprehensions
New submission from Retro vinet...@gmail.com: There's a mistake in the code snippet: freshfruit = [' banana', ' loganberry ', 'passion fruit '] [weapon.strip() for weapon in freshfruit] ['banana', 'loganberry', 'passion fruit'] The second line should be: [fruit.strip() for fruit in freshfruit] ['banana', 'loganberry', 'passion fruit'] The old code snippet had weapons as items which many people didn't like, so the code snippet was changed into a friendlier version. Please fix this code snippet so that weapons are not involved here, even though the code is not broken. Thank you. -- assignee: georg.brandl components: Documentation messages: 90486 nosy: Retro, georg.brandl severity: normal status: open title: Documentation Tutorial List Comprehensions versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6475 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6475] Documentation Tutorial List Comprehensions
Ezio Melotti ezio.melo...@gmail.com added the comment: That example was introduced in r16743 as: spcs = [ Apple, Banana , Coco nut ] print [s.strip() for s in spcs] ['Apple', 'Banana', 'Coco nut'] and was changed in r16831 to: freshfruit = [' banana', ' loganberry ', 'passion fruit '] [weapon.strip() for weapon in freshfruit] ['banana', 'loganberry', 'passion fruit'] I think that the fresh fruits can really be considered as dangerous weapons - especially the banana. wink -- nosy: +ezio.melotti priority: - low resolution: - rejected status: open - pending ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6475 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6475] Documentation Tutorial List Comprehensions
Mark Dickinson dicki...@gmail.com added the comment: I think that the fresh fruits can really be considered as dangerous weapons. Indeed. Google for monty python self-defence against fresh fruit. -- nosy: +marketdickinson status: pending - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6475 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6475] Documentation Tutorial List Comprehensions
Changes by Mark Dickinson dicki...@gmail.com: -- status: open - pending ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6475 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1563] asyncore and asynchat incompatible with Py3k str and bytes
Jorrit Posthuma j.posth...@gmail.com added the comment: It's not 'that' clear you should only work with bytes on a socket. Especially not when using a wrapper like asynchat. They are supposed to make it easier, and passing a string is easier. Maybe it's possible to do a default byte conversion when the user is working with strings. -- nosy: +JPosthuma ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1563 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1563] asyncore and asynchat incompatible with Py3k str and bytes
Jean-Paul Calderone exar...@divmod.com added the comment: It's not 'that' clear you should only work with bytes on a socket. It's pretty clear to me. :) That's what sockets can deal with - bytes. If you want to transfer something other than bytes via a socket, then you need to convert it to bytes. In the case of unicode, there are many different choices which can be made for how to do this conversion. asyncore cannot know what the correct choice is in any particular situation, so it shouldn't try to make it. The attached patch forces the application to make this choice, fortunately. However, since push_str is only one line, I'm not sure what the attraction is. Why is push_str(foo, bar) preferable to push(foo.encode(bar))? Maybe it's possible to do a default byte conversion when the user is working with strings. This definitely isn't reasonable and should not be done. It's also not what the last proposed patch does, so it doesn't seem to the direction the other interested parties have been working in. -- nosy: +exarkun ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1563 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1563] asyncore and asynchat incompatible with Py3k str and bytes
Jorrit Posthuma j.posth...@gmail.com added the comment: On 13 jul 2009, at 17:33, Jean-Paul Calderone wrote: Jean-Paul Calderone exar...@divmod.com added the comment: It's not 'that' clear you should only work with bytes on a socket. It's pretty clear to me. :) That's what sockets can deal with - bytes. It's allso clear to me, but there are people that don't know that. If you want to transfer something other than bytes via a socket, then you need to convert it to bytes. In the case of unicode, there are many different choices which can be made for how to do this conversion. asyncore cannot know what the correct choice is in any particular situation, so it shouldn't try to make it. The attached patch forces the application to make this choice, fortunately. However, since push_str is only one line, I'm not sure what the attraction is. Why is push_str(foo, bar) preferable to push(foo.encode(bar))? It's not, I was more thinking of push_str(foo), where it uses a default encoding. I think some people don't know what unicode or encoding is, or don't want to worry about it. Maybe it's possible to do a default byte conversion when the user is working with strings. This definitely isn't reasonable and should not be done. It's also not what the last proposed patch does, so it doesn't seem to the direction the other interested parties have been working in. It's not an attack ;), i'm pretty new to Python, and it was just something that i noticed (after changing from 2 to 3) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1563 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1563] asyncore and asynchat incompatible with Py3k str and bytes
Jean-Paul Calderone exar...@divmod.com added the comment: It's not, I was more thinking of push_str(foo), where it uses a default encoding. I think some people don't know what unicode or encoding is, or don't want to worry about it. Nice as it sounds, the problem with this is that those people cannot write correct programs without understanding this. A push_str API which implicitly encodes unicode to str would just be a point for bugs to be introduced. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1563 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6418] unittest.TestProgram change breaks nose
Michael Foord mich...@voidspace.org.uk added the comment: Committed in revision 6418. Still needs fixing in trunk (note the patch itself can't be applied as it is missing a couple of 'self's). Having problems testing this fix without causing spurious output on stderr. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6418 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6418] unittest.TestProgram change breaks nose
Michael Foord mich...@voidspace.org.uk added the comment: Oops - I meant revision 73997 of course. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6418 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6476] MSVCRT's spawnve/spawnvpe are not thread safe
New submission from Jose Fonseca jose.r.fons...@gmail.com: MSVCRT's implementation of _spawnve, _spawnvpe, or any version that takes an environ as paramater is not thread-safe, because it stores a temporary environment string into a global variable. _spawnve, _spawnvpe, and friends call a function named _cenvarg which concatenate the environment strings into a global variable called _aenvptr, which gets free'd and zero'd after each invocation. This was causing random build failures in scons when parallel build (-j) was enabled. The sample application evidences this problem. It also includes a simple workaround in python, by acquiring a global lock around os.spawnve, and simulating P_WAIT with P_NOWAIT to avoid holding the global lock while the child process is running. I believe something along these lines should be done for CPython on Windows. -- components: Interpreter Core files: spawnve.py messages: 90495 nosy: jfonseca severity: normal status: open title: MSVCRT's spawnve/spawnvpe are not thread safe type: crash versions: Python 2.4, Python 2.5, Python 2.6 Added file: http://bugs.python.org/file14495/spawnve.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6476 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6477] Pickling of NoneType raises PicklingError
New submission from July Tikhonov july.t...@gmail.com: Python 3.2a0 (py3k:73749M, Jul 1 2009, 23:17:59) [GCC 4.3.2 [gcc-4_3-branch revision 141291]] on linux2 Type help, copyright, credits or license for more information. import pickle [40072 refs] pickle.dumps(type(None)) Traceback (most recent call last): File stdin, line 1, in module File /usr/local/lib/python3.2/pickle.py, line 1358, in dumps Pickler(f, protocol, fix_imports=fix_imports).dump(obj) _pickle.PicklingError: Can't pickle class 'NoneType': attribute lookup builtins.NoneType failed [40137 refs] -- components: Library (Lib) messages: 90496 nosy: July severity: normal status: open title: Pickling of NoneType raises PicklingError type: behavior versions: Python 2.6, Python 2.7, Python 3.0, Python 3.1, Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6477 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6478] time.tzset does not reset _strptime's locale time cache
New submission from Mihai Ibanescu mihai.ibane...@gmail.com: If one changes from one timezone to another within the same python process, and if one tries to parse a time string that includes the timezone, the library malfunctions. See attached script for a reproducer. The problem is that, even though time.tzset() is called, the LocaleTime persisted in the TimeRE global is not reset. In my example, the EDT timezone name, compiled from the previous TZ variable, is not valid anymore in the 'Pacific/Fiji' timezone. To witness the behavior, run the attached script with no arguments. It will parse the time in the America/New_York timezone just fine. Then run it with an argument (like python ttime.py 1). It will first prime the _strptime cache in the Pacific/Fiji timezone, then attempt to parse the same time string in the America/New_York timezone. Finally, you can change the if 0 to if 1 for working around the problem. This has been verified in 2.4.4 and 2.6.1 (did not have a chance to verify it against python 2.6.2 yet). -- components: Library (Lib) files: ttime.py messages: 90497 nosy: mibanescu severity: normal status: open title: time.tzset does not reset _strptime's locale time cache type: behavior versions: Python 2.4, Python 2.6 Added file: http://bugs.python.org/file14496/ttime.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6478 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6476] MSVCRT's spawnve/spawnvpe are not thread safe
Amaury Forgeot d'Arc amaur...@gmail.com added the comment: Indeed. But shouldn't you use the subprocess module instead? -- nosy: +amaury.forgeotdarc resolution: - wont fix status: open - pending ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6476 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6479] platform.py writes to hard coded platform dependant dev/null
New submission from John Burnett pyt...@johnburnett.com: The functions _syscmd_uname and _syscmd_file are hard-coded to pipe stderr to /dev/null, as opposed to os.devnull. On Windows, this has the effect of creating a file called null to a local dev directory (if the directory exists). Attached is a fix. While the _syscmd_uname change isn't entirely necessary on Windows due to its sys.platform check, I changed both functions for consistency (and I'm not sure what other platforms might not have a /dev/null either). -- components: Library (Lib) files: devnull.patch keywords: patch messages: 90499 nosy: john.burnett severity: normal status: open title: platform.py writes to hard coded platform dependant dev/null type: behavior versions: Python 2.5, Python 2.6, Python 2.7, Python 3.0, Python 3.1, Python 3.2 Added file: http://bugs.python.org/file14497/devnull.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6479 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6479] platform.py writes to hard coded platform dependant dev/null
Changes by Benjamin Peterson benja...@python.org: -- assignee: - lemburg nosy: +lemburg ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6479 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6479] platform.py writes to hard coded platform dependant dev/null
Marc-Andre Lemburg m...@egenix.com added the comment: On which Windows platform would that be ? Both internal helper functions bypass using an external command for Windows and DOS platforms. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6479 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
