Re: Copying a ZipExtFile
On Oct 28, 8:33 pm, ryles ryle...@gmail.com wrote: As for why the bytesToRead calculation in ZipExtFile.read() results in a long, I've not yet looked at it closely. Simple, actually: In ZipExtFile.__init__(): self.bytes_read = 0L In ZipExitFile.read(): bytesToRead = self.compress_size - self.bytes_read 13 - 0L == 13L -- http://mail.python.org/mailman/listinfo/python-list
Re: Copying a ZipExtFile
On Oct 23, 1:15 pm, Moore, Mathew L moor...@battelle.org wrote:
Hello all,
A newbie here. I was wondering why the following fails on Python 2.6.2
(r262:71605) on win32. Am I doing something inappropriate?
Interestingly, it works in 3.1, but would like to also get it working in 2.6.
Thanks in advance,
--Matt
import io
import shutil
import tempfile
import zipfile
with tempfile.TemporaryFile() as f:
# (Real code retrieves archive via urllib2.urlopen().)
zip = zipfile.ZipFile(f, mode='w')
zip.writestr('unknowndir/src.txt', 'Hello, world!')
zip.close();
# (Pretend we just downloaded the zip file.)
f.seek(0)
# Result of urlopen() is not seekable, but ZipFile requires a
# seekable file. Work around this by copying the file into a
# memory stream.
with io.BytesIO() as memio:
shutil.copyfileobj(f, memio)
zip = zipfile.ZipFile(file=memio)
# Can't use zip.extract(), because I want to ignore paths
# within archive.
src = zip.open('unknowndir/src.txt')
with open('dst.txt', mode='wb') as dst:
shutil.copyfileobj(src, dst)
The last line throws an Error:
Traceback (most recent call last):
File test.py, line 25, in module
shutil.copyfileobj(src, dst)
File C:\Python26\lib\shutil.py, line 27, in copyfileobj
buf = fsrc.read(length)
File C:\Python26\lib\zipfile.py, line 594, in read
bytes = self.fileobj.read(bytesToRead)
TypeError: integer argument expected, got 'long'
It should hopefully work if you use cStringIO/StringIO instead of
BytesIO.
I think the issue is essentially that StringIO.read() will accept a
long object while the backport of bytesio to to 2.6 does an explicit
check for int:
py StringIO.StringIO(foo).read(long(1))
'f'
py io.BytesIO(foo).read(long(1))
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: integer argument expected, got 'long'
Should this be amended? Perhaps someone on core can consider it.
As for why the bytesToRead calculation in ZipExtFile.read() results in
a long, I've not yet looked at it closely.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Building Python on Solaris 10?
On Oct 28, 3:46 pm, Judy Booth j...@invalid.invalid wrote:
Can anyone point me towards some instructions for building Python on
Solaris 10?
We need this for some of our test scripts and so far we cannot get this
to build.
We have tried both Python 2.6.4 and 3.1.1 and both fail with messages
like this:
Include/pyport.h:685:2: #error LONG_BIT definition appears wrong for
platform (bad gcc/glibc config?).
*** Error code 1
make: Fatal error: Command failed for target `Modules/python.o'
The configure command used is:
./configure --with-universal-archs=64-bit --enable-universalsdk
LDFLAGS=-s
-L/usr/local/lib -L/opt/ssl/lib -lgcc --prefix=/opt/python
--prefix=${PREFIX}
and the compiler version is gcc 3.4.3.
A search on Google showed this up as a problem several years ago and
suggested that there might be a problem with the way configure is working
on Solaris.
If anyone can help with this that would be great.
Thanks,
Judy
We never heard back from the OP of this thread, but you might try
building with cc instead of gcc:
http://groups.google.com/group/comp.lang.python/browse_thread/thread/bf109ba2c86c3715
--
http://mail.python.org/mailman/listinfo/python-list
Re: Working threads progress
On Oct 28, 7:02 pm, mattia ger...@gmail.com wrote: Now, I would like to know the activity done (e.g. every two seconds) so I create another thread that checks the queue size (using .qsize()). Have you any suggestion to improve the code? It's not uncommon to pass each thread a second queue for output, which in your case might be tuples of (item, result). You can read from this queue in a single thread, and since the results now accumulate incrementally, you can easily collect more detailed status/progress information (if that's what you're looking for). -- http://mail.python.org/mailman/listinfo/python-list
Re: Anyone have python 3.1.1 installed on Solaris 10 ? (sparc or x86)
On Oct 19, 1:00 pm, Dave Crouse dc...@crouse.us wrote: Anyone have python 3.1.1 installed on Solaris 10 ? (sparc or x86) I've tried several times on sparc, I keep getting: gcc -lintl -o python \ Modules/python.o \ libpython3.1.a -lsocket -lnsl -lintl -lrt -ldl -lm Undefined first referenced symbol in file libintl_bind_textdomain_codeset libpython3.1.a(_localemodule.o) libintl_gettext libpython3.1.a(_localemodule.o) libintl_textdomain libpython3.1.a(_localemodule.o) libintl_dcgettext libpython3.1.a(_localemodule.o) libintl_bindtextdomain libpython3.1.a(_localemodule.o) libintl_dgettext libpython3.1.a(_localemodule.o) ld: fatal: Symbol referencing errors. No output written to python collect2: ld returned 1 exit status *** Error code 1 make: Fatal error: Command failed for target `python' Ideas ? (I'm not a c programmerso bear with me ;) ) Have you tried building with cc instead of gcc? I've seen a few botched gcc installations on Solaris (though maybe it's just my own environment). Try adding --without-gcc to the ./configure line, and before that export CC=cc (or however you set environment variables with your own shell). -- http://mail.python.org/mailman/listinfo/python-list
Re: restriction on sum: intentional bug?
On Oct 16, 6:53 pm, ryles ryle...@gmail.com wrote: Then you probably haven't read through these discussions: sum and strings:http://mail.python.org/pipermail/python-list/2006-August/subject.html summing a bunch of numbers:http://mail.python.org/pipermail/python-dev/2003-April/subj You meant: sum and strings: http://mail.python.org/pipermail/python-list/2006-August/subject.html summing a bunch of numbers: http://mail.python.org/pipermail/python-dev/2003-April/subject.html -- http://mail.python.org/mailman/listinfo/python-list
Re: Installation question 2.5.4
On Sat, Oct 17, 2009 at 9:55 PM, JimR kd...@arrl.net wrote: Thanks. As it turned out, I needed /usr/local/python instead of /usr/local as the prefix. After setting that, all worked as it should. -- http://mail.python.org/mailman/listinfo/python-list
Re: restriction on sum: intentional bug?
On Oct 16, 11:39 am, Alan G Isaac alan.is...@gmail.com wrote: I expected this to be fixed in Python 3: sum(['ab','cd'],'') Traceback (most recent call last): File stdin, line 1, in module TypeError: sum() can't sum strings [use ''.join(seq) instead] Then you probably haven't read through these discussions: sum and strings: http://mail.python.org/pipermail/python-list/2006-August/subject.html summing a bunch of numbers: http://mail.python.org/pipermail/python-dev/2003-April/subj -- http://mail.python.org/mailman/listinfo/python-list
Re: How about adding slice notation to iterators/generators?
On Oct 16, 5:28 pm, Eloff dan.el...@gmail.com wrote: By giving iterators a default, overridable, __getitem__ that is just syntactic sugar for islice, they would share a slightly larger interface subset with the builtin container types. In a duck-typed language like python, that's almost always a good thing. You could use iterators in more situations which expect something more like a list. As long as it breaks no rationally existing code, I can think of no good reason why not to do this in a future python. I think Python programmers have learned to expect certain things from objects that support __getitem__. For example, indexing and slicing is repeatable on the same object: a[1] == a[1] a[1:4] == a[1:4] If you saw the above code would you want to think twice above whether or not these expressions were true? Iterators don't have a general concept of get item like types such as strings, lists, etc. They have a concept of get next item. So, with your proposal, i[1] != i[1] and i[1:4] != i[1:4]. Not only that, it's also common for types with __getitem__ to have __len__, which we obviously can't provide. So, ultimately, although it could afford some small conveniences, I think trying to mix iterators with __getitem__ would cause too much confusion. The use of islice() is both readable and explicit. It's very clear to the reader that you're working with iterators and that items will be consumed (something that's not reversible). -- http://mail.python.org/mailman/listinfo/python-list
Re: strange behaviour when inheriting from tuple
On Oct 11, 3:04 am, metal metal...@gmail.com wrote: Environment: PythonWin 2.5.4 (r254:67916, Apr 27 2009, 15:41:14) [MSC v.1310 32 bit (Intel)] on win32. Portions Copyright 1994-2008 Mark Hammond - see 'Help/About PythonWin' for further copyright information. Evil Code: class Foo: def __init__(self, *args): print args Foo(1, 2, 3) # (1, 2, 3), good class Bar(tuple): def __init__(self, *args): print args Bar(1, 2, 3) # TypeError: tuple() takes at most 1 argument (3 given) what the heck? I even didn't call tuple.__init__ yet When subclassing immutable types you'll want to override __new__, and should ensure that the base type's __new__ is called: __ class MyTuple(tuple): __ def __new__(cls, *args): __ return tuple.__new__(cls, args) __ print MyTuple(1, 2, 3) (1, 2, 3) See http://www.python.org/download/releases/2.2.3/descrintro/#__new__ -- http://mail.python.org/mailman/listinfo/python-list
Re: Script to complete web form fields
On Oct 10, 9:39 pm, Feyo dkatkow...@gmail.com wrote: How can I use Python to complete web form fields automatically? My work web-based email time-out is like 15 seconds. Every time I need to access my calendar, address book, or email, I have to type in my username and password. I'm just tired of it. I found the ClientForm module and have been working with it. Can do what I want in a unit testing sense to have responses sent and returned, but what I want to do is much simpler. Any ideas? Thanks. See this recent thread for ideas: http://groups.google.com/group/comp.lang.python/browse_thread/thread/58b7513596ab648c However, why not take a more direct approach? Speak to management and see if your company can correct this. It's obviously hindering productivity, and that's something they may have overlooked. There should be a more reasonable compromise between efficiency and security. It's worth a shot. -- http://mail.python.org/mailman/listinfo/python-list
Re: code in a module is executed twice (cyclic import problems) ?
On Oct 10, 7:36 pm, Stef Mientki stef.mien...@gmail.com wrote: hello, I always thought code in a module was only executed once, but doesn't seem to be true. I'm using Python 2.5. And this is the example: == A.py == My_List = [] == B.py == from A import * My_List.append ( 3 ) print 'B', My_List import C == C.py == from A import * from B import * print 'C', My_List Now when you start with B.py as the main program, this is the resulting output: B [3] B [3, 3] C [3, 3] Why is the B.py executed twice ? thanks, Stef FYI, there was actually a related discussion about this just recently: http://groups.google.com/group/comp.lang.python/browse_thread/thread/e24be42ecbee7cad -- http://mail.python.org/mailman/listinfo/python-list
Re: How to find number of line that is currently executing?
On Oct 9, 11:46 pm, Dr. Phillip M. Feldman pfeld...@verizon.net wrote: I would like to put a statement on line N of my program that prints the line number that is currently executing. inspect.currentframe().f_lineno http://docs.python.org/library/inspect.html -- http://mail.python.org/mailman/listinfo/python-list
Re: Need feedback on subprocess-using function
On Oct 4, 9:46 pm, Nobody nob...@nowhere.com wrote: On Sat, 03 Oct 2009 13:21:00 +, gb345 wrote: I'm relatively new to Python, and I'm trying to get the hang of using Python's subprocess module. As an exercise, I wrote the Tac class below, which can prints output to a file in reverse order, by piping it through the Unix tac utility. (The idea is to delegate the problem of managing the memory for an arbitrarily large task to tac.) self.pipe = subprocess.Popen(['tac'], stdout=out, stdin=subprocess.PIPE, stderr=subprocess.PIPE) This works OK, as far as I can tell, but I'm not sure that I've dotted all the i's and crossed all the t's... E.g., I had to add the line p.stdin.close() to the close method when I when I ran into sporadic deadlock at the p.stderr.read() statement. Are there other similar problems lurking in this code? Yep. If the process writes more than a buffer-full of data to stderr, it will deadlock. tac will block trying to write to stderr, and won't be reading its stdin, so your program will block trying to write to tac. This is why the POSIX popen() call only lets you attach a pipe to stdin or stdout but not both. If you want a double-ended slave process, you need to use polling or non-blocking I/O or asynchronous I/O or multiple threads. I'm not aware of any single solution which works on all platforms. The easiest way around this problem is to redirect stderr to a temporary file and read in the file's contents in the close() method. There is also Popen.communicate(): http://docs.python.org/library/subprocess.html#subprocess.Popen.communicate -- http://mail.python.org/mailman/listinfo/python-list
Re: Python shared lib
On Oct 4, 11:04 am, a...@pythoncraft.com (Aahz) wrote: I've got a dim memory that there's a reason for this -- you might try searching the python-dev archives and/or bugs.python.org. Found mention of it in this discussion: http://bugs.python.org/issue771998 I disagree that --enable-shared should be the default. It is difficult to maintain, as finding shared libraries is always a PITA (in particular on Solaris). Its only real use is for embedding, in even in the case of embedding, it is possible to avoid using shared libraries. -- http://mail.python.org/mailman/listinfo/python-list
Re: weak reference to bound method
On Oct 2, 4:54 am, Ole Streicher ole-usenet-s...@gmx.net wrote: Hi group, I am trying to use a weak reference to a bound method: class MyClass(object): def myfunc(self): pass o = MyClass() print o.myfunc bound method MyClass.myfunc of __main__.MyClass object at 0xc675d0 import weakref r = weakref.ref(o.myfunc) print r() None This is what I do not understand. The object o is still alive, and therefore the bound method o.myfunc shall exists. Why does the weak reference claim that it is removed? And how can I hold the reference to the method until the object is removed? Is this a bug or a feature? (Python 2.6) Best regards Ole Have a look at: http://mindtrove.info/articles/python-weak-references -- http://mail.python.org/mailman/listinfo/python-list
Re: question about the GC implementation
On Oct 2, 11:20 am, Francis Moreau francis.m...@gmail.com wrote: Hello, I'm looking at gcmodule.c and in move_unreachable() function, the code assumes that if an object has its gc.gc_refs stuff to 0 then it *may* be unreachable. How can an object tagged as unreachable could suddenly become reachable later ? Thanks It looks like you're not reading through all of the comments: GC_TENTATIVELY_UNREACHABLE move_unreachable() then moves objects not reachable (whether directly or indirectly) from outside the generation into an unreachable set. Objects that are found to be reachable have gc_refs set to GC_REACHABLE again. Objects that are found to be unreachable have gc_refs set to GC_TENTATIVELY_UNREACHABLE. It's tentatively because the pass doing this can't be sure until it ends, and GC_TENTATIVELY_UNREACHABLE may transition back to GC_REACHABLE. Only objects with GC_TENTATIVELY_UNREACHABLE still set are candidates for collection. If it's decided not to collect such an object (e.g., it has a __del__ method), its gc_refs is restored to GC_REACHABLE again. -- http://mail.python.org/mailman/listinfo/python-list
Re: Granularity of OSError
On Sep 19, 9:22 pm, MRAB pyt...@mrabarnett.plus.com wrote: The point is that it's sometimes a good idea to do a cheap check first before attempting an operation that's 'expensive' even when it fails. Strongly agree. Furthermore, with LBYL it's often easier to give a user clearer error messages for common usage errors, rather than waiting for an exception in a much lower-level place where it's less clear to them what the cause is. -- http://mail.python.org/mailman/listinfo/python-list
Re: on package import, have it conditionally import a subpackage
On Sep 19, 4:06 pm, Gabriel Rossetti gabriel.rosse...@arimaz.com
wrote:
Hello everyone,
I'd like to ba able to import a package and have it's __init__
conditionally import a subpackage. Suppose that you have this structure :
mybase/
mybase/__init__.py
mybase/mypkg
mybase/mypkg/__init__.py
mybase/mypkg/module0.py
mybase/mypkg/type1
mybase/mypkg/type1/__init__.py
mybase/mypkg/type1/module1.py
mybase/mypkg/type1/module2.py
mybase/mypkg/type2
mybase/ mypkg/type2/__init__.py
mybase/ mypkg/type2/module1.py
mybase/ mypkg/type2/module2.py
I'd like to do something like th following in my code
import mybase
mybase.setType(type1)
from mybase.mypkg import module0, module1, module2
module0.__file__
'mybase/mypkg/module0.pyc'
module1.__file__
'mybase/mypkg/type1/module1.pyc'
module2.__file__
'mybase/mypkg/type1/module2.pyc'
but I can't figure out how to do this.
Thank you,
Gabriel
You might try something like the following:
$ export PYTHONPATH=/home/ryles
$ cat mybase_test.py
import mybase
mybase.type = type2
from mybase.mypkg import module0, module1, module2
print module0.__file__
print module1.__file__
print module2.__file__
$ python mybase_test.py
/home/ryles/mybase/mypkg/module0.pyc
/home/ryles/mybase/mypkg/type2/module1.pyc
/home/ryles/mybase/mypkg/type2/module2.pyc
$ ls -l mybase/
-rw-r--r-- 1 ryles None 44 Sep 20 16:59 __init__.py
drwxr-xr-x+ 4 ryles None 0 Sep 20 17:06 mypkg
$ cat mybase/__init__.py
default_type = type1
type = default_type
$ ls -l mybase/mypkg/
-rw-r--r-- 1 ryles None 307 Sep 20 17:06 __init__.py
-rw-r--r-- 1 ryles None 0 Sep 20 16:51 module0.py
drwxr-xr-x+ 2 ryles None 0 Sep 20 17:02 type1
drwxr-xr-x+ 2 ryles None 0 Sep 20 17:02 type2
$ ls -l mybase/mypkg/type1/
-rw-r--r-- 1 ryles None 0 Sep 20 16:49 __init__.py
-rw-r--r-- 1 ryles None 0 Sep 20 16:49 module1.py
-rw-r--r-- 1 ryles None 0 Sep 20 16:51 module2.py
$ ls -l mybase/mypkg/type2/
-rw-r--r-- 1 ryles None 0 Sep 20 16:48 __init__.py
-rw-r--r-- 1 ryles None 0 Sep 20 16:51 module1.py
-rw-r--r-- 1 ryles None 0 Sep 20 16:49 module2.py
$ cat mybase/mypkg/__init__.py
# These modules are hard-coded.
from mybase.mypkg import module0
# These modules are resolved dynamically.
from mybase import type
for module_name in [module1, module2]:
full_name = mybase.mypkg. + type + '.' + module_name
globals()[module_name] = __import__(full_name, {}, {},
[full_name])
$
--
http://mail.python.org/mailman/listinfo/python-list
Re: Are min() and max() thread-safe?
On Sep 17, 7:02 am, Carl Banks pavlovevide...@gmail.com wrote: On Sep 16, 9:33 pm, Steven D'Aprano ste...@remove.this.cybersource.com.au wrote: def minmax(seq): result = [None, None] t1 = MMThread(seq, min, result, 0) t2 = MMThread(seq, max, result, 1) t1.start() t2.start() # Block until all threads are done. while any([t1.isAlive(), t2.isAlive()]): time.sleep(0) Why not use t1.join(); t2.join() here? Is there any benefit to do it this way instead? Carl Banks The benefit is that SIGINT (Control-C) will be handled. Not sure why sleep(0) is used, though, rather than something less busy like sleep (0.5). -- http://mail.python.org/mailman/listinfo/python-list
Re: Multiple inheritance - How to call method_x in InheritedBaseB from method_x in InheritedBaseA?
On Sep 9, 1:47 am, The Music Guy music...@alphaios.net wrote: I should also mention--and I should have realized this much sooner--that each of the BaseN classes are themselves each going to have at least one common base which will define method_x, so each BaseN will be calling that if it defines its own method_x. Again, sorry I didn't mention that sooner. For some reason it didn't occur to me that it would be important. I feel dumb now... :P Here is the updated current example: class CommonBase(object): def method_x(self, a, b c): ... no call to superclass' method_x is needed here ... class MyMixin(object): def method_x(self, a, b, c): get_other_base(self).method_x(self, a, b, c) ... class BaseA(CommonBase): def method_x(self, a, b, c): super(BaseA, self).method_x(a, b, c) ... class BaseB(CommonBaset): ... class BaseC(CommonBase): def method_x(self, a, b, c): super(BaseC, self).method_x(a, b, c) ... class FooX(MyMixin, BaseA): ... class FooY(MyMxin, BaseB): ... class FooZ(MyMixin, BaseC): ... OK, how about this? class CommonBase(object): def method_x(self, a, b, c): print CommonBase class MyMixin(object): def method_x(self, a, b, c): print MyMixin, super(MyMixin, self).method_x(a, b, c) class BaseA(CommonBase): def method_x(self, a, b, c): print BaseA, super(BaseA, self).method_x(a, b, c) class BaseB(CommonBase): def method_x(self, a, b, c): print BaseB, super(BaseB, self).method_x(a, b, c) class BaseC(CommonBase): def method_x(self, a, b, c): print BaseC, super(BaseC, self).method_x(a, b, c) class FooX(MyMixin, BaseA): def method_x(self, a, b, c): print FooX, super(FooX, self).method_x(a, b, c) class FooY(MyMixin, BaseB): def method_x(self, a, b, c): print FooY, super(FooY, self).method_x(a, b, c) class FooZ(MyMixin, BaseC): # Supposing this class doesn't require a specialized method_x. pass FooX().method_x(1, 2, 3) # Prints 'FooX MyMixin BaseA CommonBase' FooY().method_x(1, 2, 3) # Prints 'FooY MyMixin BaseB CommonBase FooZ().method_x(1, 2, 3) # Prints 'MyMixin BaseC CommonBase' --- Each method_x implementation uses super() to ensure that the next base class method_x is called. The exception is CommonBase, which is presumed to be the hierarchy root. http://www.python.org/download/releases/2.2.3/descrintro/#cooperation http://www.python.org/download/releases/2.3/mro -- http://mail.python.org/mailman/listinfo/python-list
Re: Multiple inheritance - How to call method_x in InheritedBaseB from method_x in InheritedBaseA?
On Sep 9, 7:48 pm, The Music Guy music...@alphaios.net wrote: Btw, Carl, please forgive me if I frustrate you, because I'm trying my best not to. I'm trying to keep track of what I did and what you did and what Ryles and Scott did, while at the same time trying to keep a firm grasp of exactly what it is I'm trying to acheive. Besides that, I'm not a professional programmer--just a hobbyist. And of course, there's all the other IRL things I'm trying to deal with simultaneously with this...hopefully you can see how I would make mistakes. Glad to see your project is afoot. Multiple inheritance is tricky stuff. Try to keep class hierarchies simple and prefer composition when feasible. Have a look at the links I suggested in my previous post, particularly the new-style classes paper. It's tough reading but if you get through it you'll gain a lot. And don't worry about Carl, he's a Calvin Klein model and, as you know, sometimes they can get a bit moody ;) -- http://mail.python.org/mailman/listinfo/python-list
Re: Is #!/usr/bin/env python the better shebang line ?
On Sep 6, 10:01 am, Timothy Madden terminato...@gmail.com wrote: Hello Sorry if this has been discussed before, my search did not find it. My questions is if I should use #!/usr/bin/env python as the shebang line in a portable and open python script and if it does help with portability and usage. Note that there is one limitation of /usr/bin/env on many systems: you cannot supply any arguments to the interpreter. e.g. #!/usr/bin/env python -u The above will fail on many systems as env will literally try to find a file named python -u. http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/237208 Another, more obvious, case where you might avoid /usr/bin/env is when you know (or might expect in the future) that there is another Python version on the system which your script will not work with. Then it's often convenient to hard code the interpreter path. This comes up mostly in environments where you're not the administrator and have another Python version installed in a custom place which you (and any co-developers) have in their PATH. For example, if you wanted to share your script with a non-team member, then having /usr/bin/env in your script would force them to have to change their PATH first. Then, supposing /usr/bin/env is better than /usr/bin/python and I use it, is it not the case that many editors and code analysing programs, that want to know the script language for syntax highlighting, code completion, tags and source code browsing, etc, will loose their functionality because of this shebang line ? Is this not a good general reason to use the old one ? I personally use vim, and it will recognize either shebang, or if none is present (as in a typical importable module) it will notice the file suffix, .py. It's not an issue. -- http://mail.python.org/mailman/listinfo/python-list
Re: simple string question
There's probably a more general method covering all the escape sequences, but for just \n: your_string = your_string.replace(\\n, \n) py s = hello\\r\\n py s 'hello\\r\\n' py s.decode(string_escape) 'hello\r\n' py -- http://mail.python.org/mailman/listinfo/python-list
Re: Multiple inheritance - How to call method_x in InheritedBaseB from method_x in InheritedBaseA?
On Sep 4, 6:01 am, The Music Guy music...@alphaios.net wrote: I have a peculiar problem that involves multiple inheritance and method calling. I have a bunch of classes, one of which is called MyMixin and doesn't inherit from anything. MyMixin expects that it will be inherited along with one of several other classes that each define certain functionality. It defines method_x, which it assumes will also be defined in the other class that MyMixin ends up getting inherited with. For example, class MyMixin(object): def method_x(self, a, b, c): ... class BaseA(object): def method_x(self, a, b, c): ... class BaseB(object): def method_x(self, a, b, c): ... class BaseC(object): def method_x(self, a, b, c): ... class FooX(MyMixin, BaseA): ... class FooY(MyMxin, BaseB): ... class FooZ(MyMixin, BaseC): ... This all appears fine at first, but there is a problem: Each Foo's method_x must call the method_x of MyMixin as well as the method_x of each respective Foo's second base class. One cannot simply call FooN.method_x, because that will only call MyMixin.method_x and not that of the other base. One might be tempted to amend MyMixin's method_x so that it calls the parent's method_x before doing anything else: class MyMixin(object): def method_x(self, a, b, c): super(MyMixin, self).method_x(a, b, c) ... ...but of course, that will fail with an AttributeError because MyMixin's only superclass is object, which does not have a method_x. The only way I can think to solve the problem would be to implement a method_x for each Foo that calls the method_x for each of the bases: class FooX(MyMixin, BaseA): def method_x(self, a, b, c): MyMixin.method_x(self, a, b, c) BaseA.method_x(self, a, b, c) class FooY(MyMxin, BaseB): def method_x(self, a, b, c): MyMixin.method_x(self, a, b, c) BaseB.method_x(self, a, b, c) class FooZ(MyMixin, BaseC): def method_x(self, a, b, c): MyMixin.method_x(self, a, b, c) BaseC.method_x(self, a, b, c) The problem with this solution is that method_x has to be explicitly created for each Foo, even though they all do just about the same thing, which kind of defeats the purpose of using multiple inheritance in this situation. Besides that, I just don't like it! So, does anyone have an idea about how to remedy this, or at least work around it? Hopefully I've interpreted your requirement correctly. In this particular case a workaround is to use super() in your base classes and then have your concrete classes inherit from the base *before* the mixin: class MyMixin(object): def method_x(self, a, b, c): print MyMixin class BaseA(object): def method_x(self, a, b, c): print BaseA super(BaseA, self).method_x(a, b, c) class FooX(BaseA, MyMixin): pass FooX().method_x(1, 2, 3) BaseA MyMixin And if you want to define method_x() in FooX, then simply have that use super() as well. -- http://mail.python.org/mailman/listinfo/python-list
Re: executable path finding
On Aug 31, 12:37 pm, koranthala koranth...@gmail.com wrote: On Aug 31, 9:07 pm, Diez B. Roggisch de...@nospam.web.de wrote: koranthala wrote: Hi, I am creating a python application using py2exe. I am facing a problem which I am not sure how to solve. The application contains many other files associated with it - like icons, config files etc. The executable can be in any directory. If the user creates a shortcut to the executable to run in desktop, the program fails - saying that the icon image cannot be found. But I can run the application properly, if I run it from application directory - where all the other icons, config files etc are kept. How is such issues usually solved? I do not want to hardcode the paths of icons and store the icons in those paths. I guess, we can change the directory to the application directory before running the application. (in __init__.py ???) But how do we find the current directory in that case? I am completely at sea in solving this. This looks to be a very common issue. How is this usually solved? You can get the location of a module via module.__file__ This can be used to find a file relative to e.g. the toplevel module/package of your application. The pkg_resources-module of setuptools encapsulates that even into a stream and file-name api. Diez Thank you Diez. It was what I was looking for. Also see: http://docs.python.org/library/pkgutil.html -- http://mail.python.org/mailman/listinfo/python-list
Re: Q: multiprocessing.Queue size limitations or bug...
On Aug 26, 4:56 am, Michael Riedel mrie...@inova-semiconductors.de
wrote:
Sorry for being not more specific but I'm not absolutely certain whether
I encountered a bug or did anything wrong:
The (stupid) code below results in a stall forever or not at 'p0.join()'
depending on the value of TROUBLE_MAKER.
Any help, thoughts, comments?
Thank you for your time.
Michael
# --
from multiprocessing import Process, Queue
# bit vector size
BVS=8
#
TROUBLE_MAKER=12 # for greater values p0.join() is never satisfied...
def evaluate(q, id, start=0, stop=2**BVS):
cmin = {0: []}
for mask0 in range(start, stop):
for mask1 in range(0, 2**BVS):
for mask2 in range(mask1, TROUBLE_MAKER):
cmin[0].append((mask0, mask1, mask2))
print 'process %d finished (dict layout: %d/%d)...' % (id,
len(cmin), len(cmin[0]))
q.put(cmin.copy())
q.close()
if __name__ == '__main__':
q0 = Queue()
q1 = Queue()
q2 = Queue()
q3 = Queue()
part = 2**BVS/4
p0 = Process(target=evaluate, args=(q0, 0, 0*part, 1*part),
name='worker_0')
p1 = Process(target=evaluate, args=(q1, 1, 1*part, 2*part),
name='worker_1')
p2 = Process(target=evaluate, args=(q2, 2, 2*part, 3*part),
name='worker_2')
p3 = Process(target=evaluate, args=(q3, 3, 3*part, 4*part),
name='worker_3')
p0.start()
print 'process 0 started...'
p1.start()
print 'process 1 started...'
p2.start()
print 'process 2 started...'
p3.start()
print 'process 3 started...'
# main process stalls at p0.join() for bigger TROUBLE_MAKER
p0.join()
p1.join()
p2.join()
p3.join()
res0 = q0.get()
res1 = q1.get()
res2 = q2.get()
res3 = q3.get()
print 'results fetched...'
# --
--
There is a warning related to this in the documentation:
http://docs.python.org/library/multiprocessing.html#pipes-and-queues
Basically, you should reverse the order of the get() and join() calls.
multiprocessing does a pretty nice job of abstracting away the low-
level details of IPC, but there are still some gotchas. As you've
noticed, your program will deadlock when there is a large enough
amount of data being put into the queue. This is related to a hidden
thread that exists inside each of your child processes. The thread is
responsible for taking your queue items from an internal buffer and
then writing them into a pipe that your parent process will read from
when get() is called. The pipe mechanism is what allows the two
processes to pass information, and is supported directly by the
Operating System. However, the pipe has a limited capacity, and when
it is full, the writer thread is stuck waiting for the reader to read
enough from the pipe so that it can finish its write. The problem is
that your parent process (reader) is not actually calling get() to
drain the pipe. Instead it's stuck in join() waiting for the writer to
complete.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Object's nesting scope
On Aug 26, 8:51 am, zaur szp...@gmail.com wrote: Hi folk! What do you think about idea of object's nesting scope in python? Let's imaging this feature, for example, in this syntax: obj=expression: body or expression: body That's means that result object of expression evaluation is used as nested scope for body evaluation. So is this idea useful? Zaur FYI, there was recently a similar idea being discussed on the python- ideas list: http://groups.google.com/group/python-ideas/browse_thread/thread/1a13cd9a5189b01c?hl=en -- http://mail.python.org/mailman/listinfo/python-list
Re: Protecting against callbacks queuing up?
On Aug 23, 8:14 pm, Esben von Buchwald find@paa.google wrote: I thought that this code would do the trick, but it obviously doesn't help at all, and i can't understand why... def doCallback(self): if self.process_busy==False: self.process_busy=True self.data_callback() self.process_busy=False doCallback is defined as a callback function for the accelerometer instance, and data_callback does some calculations and show them on the display of the phone. What to do? Thanks... As Dennis pointed out, it sounds like doCallback() is called serially. That is, you can think of doCallback() as being called in a loop, rather than entered multiple times simultaneously. Your 'busy' flag is never actually checked until after it has already been reset to False. If the data accessed by the callback as a unique copy and has a timestamp then a simple approach which avoids multithreading would be to have your callback ensure that any 'old' data is simply ignored. This will allow it to quickly catch up to a 'new' event. -- http://mail.python.org/mailman/listinfo/python-list
Re: Setuptools - help!
On Aug 7, 3:04 pm, Peter Chant rempete...@petezilla.co.uk wrote: Robert Kern wrote: You need to put main.py into the pphoto package. $ mkdir pphoto/ $ mv main.py pphoto/ $ touch pphoto/__init__.py Thanks, it worked. Any ideas how to run the resulting scripts without installing or running as root? Pete --http://www.petezilla.co.uk If you are using Python 2.6+ you have your own site-packages directory, e.g. ~/.local/lib/python2.6/site-packages. So, if this package is only for your own use, you can use easy_install's --install- dir option to have setuptools install it there, and can also set -- script-dir to ~/bin, where your console script will go. This, and other options are discussed here: http://peak.telecommunity.com/DevCenter/EasyInstall#custom-installation-locations -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 2.6 still not giving memory back to the OS...
On Aug 15, 7:55 am, Chris Withers ch...@simplistix.co.uk wrote:
Hi All,
I thought this was fixed back in Python 2.5, but I guess not?
So, I'm playing in an interactive session:
from xlrd import open_workbook
b = open_workbook('some.xls',pickleable=0,formatting_info=1)
At this point, top shows the process usage for python to be about 500Mb.
That's okay, I'd expect that, b is big ;-)
del b
However, it still does now, maybe the garbage collector needs a kick?
import gc
gc.collect()
702614
Nope, still 500Mb. What gives? How can I make Python give the memory its
no longer using back to the OS?
Okay, so maybe this is something to do with it being an interactive
session? So I wrote this script:
from xlrd import open_workbook
import gc
b = open_workbook('some.xls',pickleable=0,formatting_info=1)
print 'opened'
raw_input()
del b
print 'deleted'
raw_input()
gc.collect()
print 'gc'
raw_input()
The raw inputs are there so I can check the memory usage in top.
Even after the gc, Python still hasn't given the memory back to the OS :-(
What am I doing wrong?
Chris
--
Simplistix - Content Management, Batch Processing Python Consulting
-http://www.simplistix.co.uk
Repeat the 'b = open_workbook ...' line in the interpreter several
times and track the memory usage after each execution. I noticed this
behavior on linux, but saw that after repeating it a few times memory
usage finally stop going up. Probably just the behavior of the
allocator.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Splitting a string into substrings of equal size
On Aug 14, 8:22 pm, candide cand...@free.invalid wrote:
Suppose you need to split a string into substrings of a given size (except
possibly the last substring). I make the hypothesis the first slice is at the
end of the string.
A typical example is provided by formatting a decimal string with thousands
separator.
What is the pythonic way to do this ?
For my part, i reach to this rather complicated code:
# --
def comaSep(z,k=3, sep=','):
z=z[::-1]
x=[z[k*i:k*(i+1)][::-1] for i in range(1+(len(z)-1)/k)][::-1]
return sep.join(x)
# Test
for z in [75096042068045, 509, 12024, 7, 2009]:
print z+ -- , comaSep(z)
# --
outputting :
75096042068045 -- 75,096,042,068,045
509 -- 509
12024 -- 12,024
7 -- 7
2009 -- 2,009
Thanks
py s='1234567'
py ','.join(_[::-1] for _ in re.findall('.{1,3}',s[::-1])[::-1])
'1,234,567'
py # j/k ;)
--
http://mail.python.org/mailman/listinfo/python-list
Re: Splitting a string into substrings of equal size
On Aug 15, 6:28 pm, MRAB pyt...@mrabarnett.plus.com wrote: for z in [75096042068045, 509, 12024, 7, 2009]: print re.sub(r(?=.)(?=(?:...)+$), ,, z) 75,096,042,068,045 509 12,024 7 2,009 The call replaces a zero-width match with a comma, ie inserts a comma, if certain conditions are met: (?=.) Look behind for 1 character. There must be at least one previous character. This ensures that a comma is never inserted at the start of the string. I could also have used (?!^). Actually, it doesn't check whether the first character is a -. That's left as an exercise for the reader. :-) (?=(?:...)+$) Look ahead for a multiple of 3 characters, followed by the end of the string. Wow, well done. An exceptional recipe from Python's unofficial regex guru. And thanks for sharing the explanation. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python and behavior
On Aug 13, 8:36 pm, goldtech goldt...@worldpost.com wrote: Could you explain or link me to an explanation of this? http://docs.python.org/tutorial/datastructures.html#more-on-conditions Give the whole tutorial a good read. -- http://mail.python.org/mailman/listinfo/python-list
Re: run all scripts in sub-directory as subroutines?
On Aug 11, 5:12 pm, guthrie guth...@mum.edu wrote: This works fine, but in the sub-modules the sys.path appropriately returns the same as from the parent, I want them to know their own file names. How?? I can pass it to them, but wondered if there is a more self-sufficient way for a module to know from where it was invoked. Instead of sys.path, you may be interested in the __file__ attribute will give you a module's filename: http://docs.python.org/reference/simple_stmts.html#index-1031 You could also be more explicit in matching *.py files in your subdirectory by using glob: http://docs.python.org/library/glob.html -- http://mail.python.org/mailman/listinfo/python-list
Re: How to do relpath implementation on 2.5
On Aug 8, 9:08 pm, Brian Allen Vanderburg II brianvanderbu...@aim.com wrote: I've coded my own 'relpath' implementation for 2.5 (shown below) and I want to make sure it follows as closely as it should to 2.6 and later. I've got a question regarding that. When attempting to convert to a relative path and it is not possible for some reason (different drive or UNC share), should that be an error or should it return the absolute path of the target? I'm using Debian so I don't have 2.6 available right now for testing. Usually Python source code is easily browsed at: http://svn.python.org/view/ Unfortunately the server is down right now. The implementation in 2.6 raises a ValueError exception. Here is the function from ntpath.py: def relpath(path, start=curdir): Return a relative version of a path if not path: raise ValueError(no path specified) start_list = abspath(start).split(sep) path_list = abspath(path).split(sep) if start_list[0].lower() != path_list[0].lower(): unc_path, rest = splitunc(path) unc_start, rest = splitunc(start) if bool(unc_path) ^ bool(unc_start): raise ValueError(Cannot mix UNC and non-UNC paths (%s and %s) % (path, start)) else: raise ValueError(path is on drive %s, start on drive %s % (path_list[0], start_list[0])) # Work out how much of the filepath is shared by start and path. for i in range(min(len(start_list), len(path_list))): if start_list[i].lower() != path_list[i].lower(): break else: i += 1 rel_list = [pardir] * (len(start_list)-i) + path_list[i:] if not rel_list: return curdir return join(*rel_list) -- http://mail.python.org/mailman/listinfo/python-list
Re: unique-ifying a list
On Aug 7, 4:53 pm, kj no.em...@please.post wrote: Suppose that x is some list. To produce a version of the list with duplicate elements removed one could, I suppose, do this: x = list(set(x)) but I expect that this will not preserve the original order of elements. OrderedSet is most likely on the way, but for now Python 3.1 and 2.7 have OrderedDict. For 3.0 and 2.6 the recipe is here: http://code.activestate.com/recipes/576669 With OrderedDict you can do: OrderedDict.fromkeys(x).keys() # Returns an iterator in 3.x, a list in 2.x. -- http://mail.python.org/mailman/listinfo/python-list
Re: Web page data and urllib2.urlopen
On Aug 5, 4:30 pm, Massi massi_...@msn.com wrote: Hi everyone, I'm using the urllib2 library to get the html source code of web pages. In general it works great, but I'm having to do with a financial web site which does not provide the souce code I expect. As a matter of fact if you try: import urllib2 res = urllib2.urlopen(http://www.marketwatch.com/story/mondays- biggest-gaining-and-declining-stocks-2009-07-27) page = res.read() print page you will see that the printed code is very different from the one given, for example, by mozilla. Since I have really little knowledge in html I can't even understand if this is a python or html problem. Can anyone give me some help? Thanks in advance. Check if setting your user agent to Mozilla results in a different page: http://diveintopython.org/http_web_services/user_agent.html -- http://mail.python.org/mailman/listinfo/python-list
Re: Is this a bug in multiprocessing or in my script?
On Aug 4, 10:37 pm, erikcw erikwickst...@gmail.com wrote: Traceback (most recent call last): File scraper.py, line 144, in module print pool.map(scrape, range(10)) File /usr/lib/python2.6/multiprocessing/pool.py, line 148, in map return self.map_async(func, iterable, chunksize).get() File /usr/lib/python2.6/multiprocessing/pool.py, line 422, in get raise self._value TypeError: expected string or buffer This is almost certainly due to your scrape call raising an exception. In the parent process, multiprocessing will detect if one of its workers have terminated with an exception and then re-raise it. However, only the exception and not the original traceback is made available, which is making debugging more difficult for you. Here's a simple example which demonstrates this behavior: *** from multiprocessing import Pool *** def evil_on_8(x): ... if x == 8: raise ValueError(I DONT LIKE THE NUMBER 8) ... return x + 1 ... *** pool = Pool(processes=4) pool.map(evil_on_8, range(5)) [1, 2, 3, 4, 5] *** pool.map(evil_on_8, range(10)) # 8 will cause evilness. Traceback (most recent call last): File stdin, line 1, in module File /bb/real/3ps/lib/python2.6/multiprocessing/pool.py, line 148, in map return self.map_async(func, iterable, chunksize).get() File /bb/real/3ps/lib/python2.6/multiprocessing/pool.py, line 422, in get raise self._value ValueError: I DONT LIKE THE NUMBER 8 *** My recommendation is that you wrap your scrape code inside a try/ except and log any exception. I usually do this with logging.exception (), or if logging is not in use, the traceback module. After that you can simply re-raise it. -- http://mail.python.org/mailman/listinfo/python-list
Re: Wrapping prstat on Solaris
On Jul 27, 10:26 am, s...@pobox.com wrote: At work we currently use top to monitor ongoing system utilization on our Solaris systems. As time has moved on though, use of top has become problematic. Our admins want us to switch to prstat, Sun's top-like command. It works fine however doesn't emit a timestamp at each display interval, so it's essentially impossible looking at a prstat output file to determine when the particular screen was emitted. If figured, No problem. I'll just write a little wrapper. Alas, that is proving harder than I thought. I can run prstat directing output to a file and it seems to blast out a block of lines every N seconds, but when run from inside a Python script I can't seem to make the damn thing not massively buffer its output. Accordingly, my script doesn't really see the output as its emitted, so the timestamp line it prepends to a block of output is off. I'm currently using subprocess.Popen like so: os.environ[TERM] = dumb cmd = prstat -c %s % .join(sys.argv[1:]) pipe = Popen(cmd, shell=True, bufsize=1, stdout=PIPE).stdout I've tried these other variants as well: * prefacing the prstat command with unbuffer (the tcl/expect thingamabob) * explicitly redirect the prstat output to /dev/stdout * setting bufsize to 0 * used os.popen instead of Subprocess.Popen Nothing seems to help. I always seem to see about 30 seconds of data at once (I'm currently using a 5-second interval and 10 lines of output). I would have expected that prstat would simply flush stdout after each block of output. Any ideas about how to get prstat to cooperate better? Thanks, -- Skip Montanaro - s...@pobox.com -http://www.smontanaro.net/ That's more than a dress. That's an Audrey Hepburn movie. -- Jerry Maguire Hey Skip, You haven't told us how you are actually reading from prstat's output pipe, which may be the cause. For instance, if you are doing for line in pipe: print line ... then this could cause your buffering issue. Instead try using readline(): while True: line = pipe.readline() ... -- http://mail.python.org/mailman/listinfo/python-list
Re: The longest word
On Jul 28, 7:55 am, NiklasRTZ nikla...@gmail.com wrote: Sincere thanks for strengthening python's superior flexibility. Same function also works around an exploding index problem returning results for longest word where otherwise a word with whitespace crashes the index: Babelfish? -- http://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't `from pkg import mod' work after `del pkg.mod'?
On Jul 25, 8:57 am, Piet van Oostrum p...@cs.uu.nl wrote:
ryles ryle...@gmail.com (r) wrote:
r According tohttp://www.python.org/doc/essays/packages.html:
r The import statement first tests whether the item is defined in the
r package; if not, it assumes it is a module and attempts to load it.
r However, I've noticed that once a module is imported using the
r `from pkg import mod' syntax, if its name is deleted from the
r package namespace then subsequent imports with `from' will fail.
This is incorrectly stated. Also on the initial import it will fail, not
just on subsequent imports.
piet$ python
Python 2.6.2 (r262:71600, Apr 16 2009, 09:17:39)
[GCC 4.0.1 (Apple Computer, Inc. build 5250)] on darwin
Type help, copyright, credits or license for more information.
from pkg import _impl
_impl # this is from a print statement in _impl to show that it did import
Traceback (most recent call last):
File stdin, line 1, in module
ImportError: cannot import name _impl
Well, since __init__.py is executed first, I would regard it as
causing the initial (successful) import, and yours being the
subsequent one, so that the initial import does not fail, the
subsequent one does. Wording aside, though, I think we are on the same
page here.
According to the documentation
http://docs.python.org/library/functions.html#__import__ the statement
from pkg import _impl
_temp = __import__('pkg', globals(), locals(), ['_impl'], -1)
_impl = _temp._impl
This fails because _temp (the imported module) doesn't have a binding
for _impl because you deleted it.
But supposing we hadn't deleted it, and __init__.py didn't import
_impl, there would still be no binding. Yet the importer would still
import and make the module available (presumably it knows to do this
since it was not already in sys.modules). The question really is that
since in our example pkg._impl is still in sys.modules, why won't the
importer add it to the pkg namespace again as it previous had? I would
imagine this is either by design, or is a case that was never
considered.
for _impl because you deleted it. By the way, if you have a `normal'
package that doesn't do anything with the name _impl, after the import
the pkg module namespace will have got a binding for _impl although it is not
in your code. It will have been put there by the import code.
Yes, this was noted further down in the original post with an example.
--
http://mail.python.org/mailman/listinfo/python-list
Re: difference in printing to screen Mac / Windows
On Jul 18, 7:03 am, Tim Chase python.l...@tim.thechases.com wrote: Lastly, you can force all standard-output in your program to be unbuffered without the -u parameter: And if you're using -u a lot, the PYTHONUNBUFFERED environment variable can also be set (but not empty), so that python adds the option automatically. -- http://mail.python.org/mailman/listinfo/python-list
Why doesn't `from pkg import mod' work after `del pkg.mod'?
According to http://www.python.org/doc/essays/packages.html: The import statement first tests whether the item is defined in the package; if not, it assumes it is a module and attempts to load it. However, I've noticed that once a module is imported using the `from pkg import mod' syntax, if its name is deleted from the package namespace then subsequent imports with `from' will fail. Here is an example of this type of scenario: $ ls -l pkg total 8.0K -rw-rw-r-- 1 ? general 0 Jul 24 20:21 _impl.py -rw-rw-r-- 1 ? general 147 Jul 24 20:28 __init__.py -rw-rw-r-- 1 ? general 0 Jul 24 20:33 public2.py -rw-rw-r-- 1 ? general 208 Jul 24 20:32 public.py $ cat pkg/__init__.py from pkg import _impl # Add functions which refer to objects in _impl # ... # Don't currently require this in the namespace anymore. del _impl $ cat pkg/public.py # Implement something with the aid of _impl. from pkg import public2 # OK, fine. from pkg import _impl# The module exists, but this will fail. # Could do import pkg._impl (or a relative import) $ python Python 2.6.2 (r262:71600, Jul 16 2009, 14:04:28) [GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2 Type help, copyright, credits or license for more information. from pkg import public Traceback (most recent call last): File stdin, line 1, in module File pkg/public.py, line 4, in module from pkg import _impl# The module exists, but this will fail. ImportError: cannot import name _impl import sys sys.modules[pkg._impl] module 'pkg._impl' from 'pkg/_impl.py' from pkg import _impl Traceback (most recent call last): File stdin, line 1, in module ImportError: cannot import name _impl import pkg._impl # Giving up! I had previously noted that once a module is imported from a package it is automatically added to the package namespace: import pkg dir(pkg) ['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__'] from pkg import public2 dir(pkg) ['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', 'public2'] However, I didn't expect python to give up on importing those names again once removed. Can anyone provide more details on what's occurring here? Is this behavior intentional (a feature), or, is it in your opinion, a defect? -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Equivalent for dd fold
On Jul 15, 1:14 pm, seldan24 selda...@gmail.com wrote: On Jul 15, 12:47 pm, Michiel Overtoom mot...@xs4all.nl wrote: seldan24 wrote: what can I use as the equivalent for the Unix 'fold' command? def fold(s,len): while s: print s[:len] s=s[len:] s=A very long string indeed. Really that long? Indeed. fold(s,10) Output: A very lon g string i ndeed. Rea lly that l ong? Indee d. Greetings, -- The ability of the OSS process to collect and harness the collective IQ of thousands of individuals across the Internet is simply amazing. - Vinod Valloppillilhttp://www.catb.org/~esr/halloween/halloween4.html Wow, I feel like a dork. I should have done more research prior to posting. Anyway, thanks for the advice. The trouble with Python is that things make 'too much' sense. Loving this language. You might also find the textwrap module useful: http://docs.python.org/library/textwrap.html -- http://mail.python.org/mailman/listinfo/python-list
Re: how to set timeout while colling a soap method?
On Jul 13, 9:07 am, dzizes dzizes...@gmail.com wrote:
Hello!
I wrote some some python code for executing a soap method:
import SOAPpy
from SOAPpy import WSDL
_server = WSDL.Proxy(some wsdl)
r=_server.generuj(some parameters...)
print r.encode('cp1250')
It works fine. However, the execution time of this soap method might be
long. Therefore, I would like to set a timeout like 1 minute, after which I
would post a timeoute message.
Any ideas?
Greats,
--
View this message in
context:http://www.nabble.com/how-to-set-timeout-while-colling-a-soap-method-...
Sent from the Python - python-list mailing list archive at Nabble.com.
I don't believe SOAPpy supports this directly. You can set the timeout
globally before you make your SOAP call:
import socket
socket.setdefaulttimeout(60)
http://docs.python.org/library/socket.html#socket.setdefaulttimeout
--
http://mail.python.org/mailman/listinfo/python-list
Re: Searching equivalent to C++ RAII or deterministic destructors
You can go ahead and implement a __del__() method. It will often work in CPython, but you get no guarantees, especially when you have reference cycles and with other Python implementations that don't use refcounting. And for resources whose lifetime is greater than your process (e.g. a file), you can also use the atexit module to help ensure they are cleaned/reclaimed at shutdown. One way to do this is to maintain a weak dictionary (from the weakref module) to your objects and install a handler which iterates this. This is useful for not only dealing with reference cycles, but for objects which may exist at the time the interpreter exits. These may not be deleted. http://docs.python.org/reference/datamodel.html#object.__del__ -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi thread reading a file
On Jul 2, 6:10 am, Gabriel Genellina gagsl-...@yahoo.com.ar wrote: En Wed, 01 Jul 2009 12:49:31 -0300, Scott David Daniels scott.dani...@acm.org escribió: These loops work well with the two-argument version of iter, which is easy to forget, but quite useful to have in your bag of tricks: def convert(in_queue, out_queue): for row in iter(in_queue.get, None): # ... convert row out_queue.put(converted_line) Yep, I always forget about that variant of iter() -- very handy! Yes, at first glance using iter() here seems quite elegant and clever. You might even pat yourself on the back, or treat yourself to an ice cream cone, as I once did. There is one subtle distinction, however. Please allow me to demonstrate. import Queue queue = Queue.Queue() queue.put(1) queue.put(la la la) queue.put(None) list(iter(queue.get, None)) [1, 'la la la'] # Cool, it really works! I'm going to change all my old code to use this... new and *improved* ... # And then one day your user inevitably does something like this. ... class A(object): ... def __init__(self, value): ... self.value = value ... ... def __eq__(self, other): ... return self.value == other.value ... queue.put(A(1)) queue.put(None) # And then this happens inside your 'generic' code (which probably even passed your unit tests). ... list(iter(queue.get, None)) Traceback (most recent call last): File stdin, line 1, in module File stdin, line 5, in __eq__ AttributeError: 'NoneType' object has no attribute 'value' # Oh... yeah. I really *did* want 'is None' and not '== None' which iter() will do. Sorry guys! Please don't let this happen to you too ;) -- http://mail.python.org/mailman/listinfo/python-list
Re: What are the limitations of cStringIO.StringIO() ?
This reminds me. Would anyone object to adding a sentence to http://docs.python.org/library/stringio.html#module-cStringIO just to mention that attributes cannot be added to a cStringIO, like such: % import cStringIO, StringIO % StringIO.StringIO().value = 1 % cStringIO.StringIO().value = 1 Traceback (most recent call last): File stdin, line 1, in module AttributeError: 'cStringIO.StringO' object has no attribute 'value' -- http://mail.python.org/mailman/listinfo/python-list
Re: Adding an object to the global namespace through f_globals is that allowed ?
On Jul 2, 1:25 am, Terry Reedy tjre...@udel.edu wrote: The next statement works, but I'm not sure if it will have any dramatical side effects, other than overruling a possible object with the name A def some_function ( ...) : A = object ( ...) sys._getframe(1).f_globals [ Name ] = A global name name = A or is name is a string var globals()[name] = A It wasn't explicit, but I think Stef meant that the global should be added to the caller's environment, which was the reason for sys._getframe(). Is this environment only intended for interactive use? I wonder if you might just set things up in a PYTHONSTARTUP script instead. -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi thread reading a file
On Jul 2, 10:20 pm, Paul Rubin http://phr...@nospam.invalid wrote: ryles ryle...@gmail.com writes: # Oh... yeah. I really *did* want 'is None' and not '== None' which iter() will do. Sorry guys! Please don't let this happen to you too ;) None is a perfectly good value to put onto a queue. I prefer using a unique sentinel to mark the end of the stream: sentinel = object() I agree, this is cleaner than None. We're still in the same boat, though, regarding iter(). Either it's 'item == None' or 'item == object ()', and depending on the type, __eq__ can introduce some avoidable risk. FWIW, even object() has its disadvantages. Namely, it doesn't work for multiprocessing.Queue which pickles and unpickles, thus giving you a new object. One way to deal with this is to define a Stopper class and type check objects taken from the queue. This is not news to anyone who's worked with multiprocessing.Queue, though. -- http://mail.python.org/mailman/listinfo/python-list
Re: multiprocessing: pool with blocking queue
On Jul 2, 11:09 am, masher vertesp...@gmail.com wrote: My questions, then, is: Is there a more elegant/pythonic way of doing what I am trying to do with the current Pool class? Another thing you might try is to subclass Pool and add an apply_async () wrapper which would wait for _taskqueue.qsize() to reach the desired size. You would probably do this wait in a loop with a small sleep. This approach would avoid needing a second Queue, but you would also add some delay to your producer due to the sleep (something you're not currently worried about). The minimum sleep may be something like 1 ms (it's really system dependent), but the time it takes for a thread blocked on a mutex to wake up is often more on the order of microseconds, which you have with your blocking queue. I doubt this offers you much satisfaction, though. If the verdict is no, I'll be happy to file a bug report. Yeah, I think it's a worth a try. -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi thread reading a file
On Jul 2, 11:55 pm, Paul Rubin http://phr...@nospam.invalid wrote: Yeah, it should allow supplying a predicate instead of using == on a value. How about (untested): from itertools import * ... for row in takewhile(lambda x: x is sentinel, starmap(in_queue.get, repeat((: ... Yeah, it's a small recipe I'm sure a lot of others have written as well. My old version: def iterwhile(callable_, predicate): Like iter() but with a predicate instead of a sentinel. return itertools.takewhile(predicate, repeatfunc(callable_)) where repeatfunc is as defined here: http://docs.python.org/library/itertools.html#recipes I wish all of these little recipes made their way into itertools or a like module; itertools seems a bit tightly guarded. -- http://mail.python.org/mailman/listinfo/python-list
Re: Direct interaction with subprocess - the curse of blocking I/O
On Jun 29, 5:43 pm, Scott David Daniels scott.dani...@acm.org wrote: and I personally wouldn't have it any other way. Simulating a shell with hooks on its I/O should be so complicated that a script kiddie has trouble writing a Trojan. +1 QOTW -- http://mail.python.org/mailman/listinfo/python-list
Re: Making code run in both source tree and installation path
On Jun 29, 12:20 pm, Javier Collado javier.coll...@gmail.com wrote: Hello, I would like to be able to run the main script in a python project from both the source tree and the path in which it's installed on Ubuntu. The script, among other things, imports a package which in turns makes use of some data files that contains some metadata that is needed in xml format. The source tree has an structure such as this one: setup.py debian/ (packaging files) src/ (source code) src/lib (package files) src/data (data files) src/bin (main script) However, when the project is installed using setup.py install, the directory structure is approximately this way: /usr/local/bin (main script) /usr/local/share/project_name (data files) /usr/local/lib/python2.x/dist-packages/project_name (library files) And when installing the code through a package, the structure is the same one, but removing local. Hence, the data files aren't always in the same relative directories depending on we're executing code from the source tree or from the installation. To make it possible to run the code from both places, I've seen different approaches: - distutils trick in setup.py to modify the installed script (i.e. changing a global variable value) so that it has a reference to the data files location. - Heuristic in the package code to detect when it's being executed from the source tree and when it has been the installed - Just using an environment variable that the user must set according to his needs I guess that there are other options, for example, maybe using buildout. What would you say it's the best/more elegant option to solve this problem? Best regards, Javier It's kludgey, but one option may be to try and use __file__ to figure out where the script is installed. Something like os.path.dirname (os.path.abspath(__file__)) could tell you if it's in src/ or in the bin/ directory, and then data files could be found in the appropriate place. I like the distutils/variable option better. Your script is more likely to still behave correctly when copied to another directory. Plus its code definitely remains cleaner. -- http://mail.python.org/mailman/listinfo/python-list
Re: generator expression works in shell, NameError in script
Does the generator expression have its own little namespace or so ? Yes, generators create a new scope: http://docs.python.org/reference/expressions.html#grammar-token-generator_expression -- http://mail.python.org/mailman/listinfo/python-list
Re: generator expression works in shell, NameError in script
On Jun 18, 9:56 am, nn prueba...@latinmail.com wrote: On Jun 18, 8:38 am, guthrie grguth...@gmail.com wrote: On Jun 17, 6:38 pm, Steven Samuel Cole steven.samuel.c...@gmail.com wrote: Still don't really understand why my initial code didn't work, though... Your code certainly looks reasonable, and looks to me like it should work. The comment of partial namespace is interesting, but unconvincing (to me) - but I am not a Python expert! It would certainly seem that within that code block it is in the local namespace, not removed from that scope to be in another. Seems that it should either be a bug, or else is a design error in the language! Just as in the above noted WTF - non-intuitive language constructs that surprise users are poor design. This is certainly an odd one. This code works fine under 2.6 but fails in Python 3.1. class x: ... lst=[2] ... gen=[lst.index(e) for e in lst] ... Traceback (most recent call last): File stdin, line 1, in module File stdin, line 3, in x File stdin, line 3, in listcomp NameError: global name 'lst' is not defined I believe it works in 2.x because unlike generator expressions, list comprehensions do not create a new scope. However, in 3.0 list comprehensions are actually treated as list(generator). http://docs.python.org/reference/expressions.html http://www.python.org/dev/peps/pep-0289 -- http://mail.python.org/mailman/listinfo/python-list
Re: Unhandled exception in thread
On Jun 11, 3:52 pm, Mads Michelsen madsmaill...@gmail.com wrote: Here's the deal. The script in question is a screen scraping thing which downloads and parses the html in the background using a separate thread (the 'thread' module), while the main program serves up the data to the user, allowing some modicum of interaction. Sometimes, not always (though I cannot see a pattern), when I quit the script, the following error message is printed: Unhandled exception in thread started by Error in sys.excepthook: Original exception was: Are you using daemon threads? There are some issues with CPython when exiting with daemon threads: http://bugs.python.org/issue1722344 http://bugs.python.org/issue1856 http://bugs.python.org/issue4106 It's possible you are encountering this kind of problem. I have had to deal with issue4106, the workaround being to explicitly join the thread of multiprocessing.Queue before exiting. I would follow Scott's advice and explicitly request and wait for your threads to exit before terminating the process. -- http://mail.python.org/mailman/listinfo/python-list
Re: unpickling a stream
On May 26, 3:25 pm, mso...@linuxmail.org wrote: Hello, I want to send a stream of pickled objects over a socket. Is there a standard way of ensuring that only complete objects are unpickled on the receiving side. client pseudo code: loop forever: receive some bytes on the socket if we have received a complete pickled object: == How is this done? unpickle the object This isn't for a project, just trying to learn some more about the pickle module. Mike If you'd like, you can turn a socket into a file-like object using its makefile() method. An Unpickler will know when to stop reading from the file (there's a STOP character in the pickle format). David Beazley gives an example of this approach in Part 7 of Generator Tricks for Systems Programmers: http://www.dabeaz.com/generators/Generators.pdf -- http://mail.python.org/mailman/listinfo/python-list
Re: slice iterator?
On May 8, 11:17 pm, Ross ross.j...@gmail.com wrote: I have a really long list that I would like segmented into smaller lists. Let's say I had a list a = [1,2,3,4,5,6,7,8,9,10,11,12] and I wanted to split it into groups of 2 or groups of 3 or 4, etc. Is there a way to do this without explicitly defining new lists? If the above problem were to be split into groups of 3, I've tried something like: start = 0 stop = 3 for i in range(len(a)): segment = a[start:stop] print segment start += stop stop += stop Unfortunately start and stop don't increment using this code. Ideally, my outcome would be [1,2,3] [4,5,6] [7,8,9] [10,11,12] There is also an iterator version here: http://code.activestate.com/recipes/303279 -- http://mail.python.org/mailman/listinfo/python-list
