Re: Queue cleanup
John Nagle na...@animats.com writes: Unoptimized reference counting, which is what CPython does, isn't all that great either. The four big bottlenecks in Python are boxed numbers, attribute lookups, reference count updates, and the GIL. The performance hit of having to lock the refcounts before update has been the historical reason for keeping the GIL. The LOCK prefix takes something like 100 cycles on an x86. Is optimizing the refcount updates going to anywhere near make up for that? Python's with statement as an approach to RAII has seemed ok to me. I can't think of a time when I've really had to use a finalizer for something with dynamic extent. They've always seemed like a code smell to me. -- http://mail.python.org/mailman/listinfo/python-list
Re: Subclassing by monkey-patching
Jason jason.hee...@gmail.com writes: [...] Is there a way I can write the subclass but then somehow... extend an existing instance all at once rather than monkeypatch methods on one by one? So I could take an existing instance of a FileMonitor and make it an instance of my subclass? This would even allow me to override the gio.File.monitor_directory() method to take the monitor returned by the original method and decide whether to make it recursive based on a parameter passed to monitor_directory(). There is a straightforward way: class A(object): ... def __init__(self, x): ... self.x = x ... class B(A): ... def x2(self): ... return self.x**2 ... a = A(42) # Create an instance of A a.__class__ = B # Change its class to B a.x2() 1764 However, I've never really tried it so I don't know what ways it could break. HTH -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Subclassing by monkey-patching
Arnaud Delobelle wrote: Jason jason.hee...@gmail.com writes: [...] Is there a way I can write the subclass but then somehow... extend an existing instance all at once rather than monkeypatch methods on one by one? So I could take an existing instance of a FileMonitor and make it an instance of my subclass? This would even allow me to override the gio.File.monitor_directory() method to take the monitor returned by the original method and decide whether to make it recursive based on a parameter passed to monitor_directory(). There is a straightforward way: class A(object): ... def __init__(self, x): ... self.x = x ... class B(A): ... def x2(self): ... return self.x**2 ... a = A(42) # Create an instance of A a.__class__ = B # Change its class to B a.x2() 1764 However, I've never really tried it so I don't know what ways it could break. I think this is fine for classes written Python but won't work here: m = gio.File(.).monitor_directory() C = type(m) class CC(C): ... def whatever(self): ... print 42 ... m.__class__ = CC Traceback (most recent call last): File stdin, line 1, in module TypeError: __class__ assignment: '__main__.GInotifyDirectoryMonitor' deallocator differs from 'CC' A possible alternative may be a class that wraps a FileMonitor instead of subclassing it. Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: Question about Reading Files
genxtech wrote: Hello. I am still really new to python and I have a project where I am trying to use the data files from another program and write a new program with new user interface and all. My first step was to open one of the files in 'rb' mode and print the contents, but I am unfamiliar with the format. Here is what was printed to the terminal: b'URES\x04\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0f \x00\x00\x00\x03\t\x00c\x01\x00\x00\x0c#\x00\x00\x02\x1b\x00\x00\x00Y \x00\x00\x00\x08\x98\x00\x00t\x00\x00\x00\x01\'\x01\x00\x00z$ \x00\x00\x04,\xa7\x00\x00\xa1%\x00\x00\x05\x0b\x00\x00\x00o$\x00\x00\n \x11\x00\x00\x00\xcd\xcc\x00\x00\x0b\xf8\x00\x00\x00\xde\xcc \x00\x00\x0c\x19\x00\x00' I am using Python 3.1 on a Fedora 13 box if that makes any difference. Any advise on how to decode the data would be greatly appreciated. What's the name of the file? What's the program that uses the file? If the source code is available, what library does that program use to read the file? Peter -- http://mail.python.org/mailman/listinfo/python-list
Python 2.7 module path problems on OS X
Dear List, I have a horrible feeling that this is in some way related to the new user installation directory in 2.7,or some problem with the framework built, but I'm having great trouble with the module search path on 2.7. I usually install modules to install_lib = ~/Library/Python/$py_version_short/site-packages I assumed that this would continue to work on 2.7. Unfortunately, it seems not to. The installation is fine, but sys.path only includes '/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip', '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old', '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload', '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages' Whereas on 2.6 on my system the default is the more extensive: '/Users/nicholas/Library/Python/2.6/site-packages/docutils-0.5-py2.6.egg', '/Users/nicholas/Library/Python/2.6/site-packages/py2app-0.4.3-py2.6.egg', '/Users/nicholas/Library/Python/2.6/site-packages/macholib-1.2.1-py2.6.egg', '/Users/nicholas/Library/Python/2.6/site-packages/modulegraph-0.7.3-py2.6.egg', '/Users/nicholas/Library/Python/2.6/site-packages/altgraph-0.6.7-py2.6.egg', '/Users/nicholas/Library/Python/2.6/site-packages/bdist_mpkg-0.4.4-py2.6.egg', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python26.zip', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-darwin', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-mac', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-mac/lib-scriptpackages', '/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-tk', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-old', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload', '/Library/Python/2.6/site-packages', '/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/PyObjC', '/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/wx-2.8-mac-unicode', '/Users/nicholas/Library/Python/2.6/site-packages' I'm sorry to ask such a trivial question on the list, but I'm used to python installations Just working! Can anyone suggest a good fix? Best wishes, Nicholas -- http://mail.python.org/mailman/listinfo/python-list
Re: Subclassing by monkey-patching
On Sep 5, 3:53 pm, Peter Otten __pete...@web.de wrote: m = gio.File(.).monitor_directory() C = type(m) 'C' will not necessarily be 'gio.FileMonitor' — I think the internals of the GIO methods might further subclass it in some way depending on what underlying monitors are available. A possible alternative may be a class that wraps a FileMonitor instead of subclassing it. I've been avoiding this because it involves a lot of boilerplate: the signals needs to be replicated and passed through, same for all GObject properties, same for the usual methods. I'd basically have to re-write the entire class in Python, and then tack on my methods. Otherwise I have to write two sets of methods for anything that touches this wrapped object. Still, if it's the only way, sure. — Jason -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 2.7 module path problems on OS X
In article aanlkti=zzgomi5nhyzca7x9rbwu==oqzrzr4s876k...@mail.gmail.com, Nicholas Cole nicholas.c...@gmail.com wrote: I have a horrible feeling that this is in some way related to the new user installation directory in 2.7,or some problem with the framework built, but I'm having great trouble with the module search path on 2.7. I usually install modules to install_lib = ~/Library/Python/$py_version_short/site-packages There are differences in defaults between different Python versions and instances of the same version as built by different providers. The pattern above is the default location for site-packages for Apple-supplied Pythons in recent versions of OS X, in particular the default /usr/bin/python2.6 in OS X 10.6 and /usr/bin/python2.5 in 10.5. I assumed that this would continue to work on 2.7. Unfortunately, it seems not to. The installation is fine, but sys.path only includes '/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip', '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib- scriptpackages', '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old', '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload', '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages ' OTOH, the Python instances installed by the python.org OS X installers (or ones built similarly) default to using the site-packages directory in the framework directory as shown in that last path. Whereas on 2.6 on my system the default is the more extensive: [...] I'm sorry to ask such a trivial question on the list, but I'm used to python installations Just working! Can anyone suggest a good fix? I'm not sure why you think it is broken. The Apple 2.6 and the python.org 2.7 have different site-package directories in different locations. That is to be expected. The Apple-supplied Python comes with some additional packages pre-installed, like setuptools, PyObjC, and wx. But you would need to install new versions of these for 2.7 anyway. f you use easy_install (aka setuptools or Distribute), keep in mind that every instance of Python needs to have its own version. So, for 2.7, you should follow the installation instructions for either setuptools or Distribute and install a version for 2.7 using python2.7. Its easy_install command will be in the framework bin directory, /Library/Frameworks/Python.framework/Versions/2.7/bin, which is the default location for scripts to be installed. You should ensure that that directory appears on your shell PATH before /usr/bin (where the Apple-supplied easy_install is); if you used the default settings for the python.org installer, it will have attempted to modify your shell startup files, like .bash_profile, to do that for you. Then you can use that easy_install (also available as easy_install-2.7) to install whatever packages you need for 2.7. Or you can use pip. Or just manually installing using the package's setup.py script. All of them use the Python instance's version of Distutils and it will do the right thing for each instance as long as you don't try to influence things by using non-default settings or setting Python-related environment variables, like PYTHONPATH. -- Ned Deily, n...@acm.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 2.7 module path problems on OS X
On Sun, Sep 5, 2010 at 10:20 AM, Ned Deily n...@acm.org wrote: I'm not sure why you think it is broken. The Apple 2.6 and the python.org 2.7 have different site-package directories in different locations. That is to be expected. The Apple-supplied Python comes with some additional packages pre-installed, like setuptools, PyObjC, and wx. But you would need to install new versions of these for 2.7 anyway. Sorry. I wasn't clear. Of course I understand that each version of python needs its own additional packages, and I was expecting to re-install them. But I don't want to install system-wide -- I'd like to have my own, user-specific, site-packages. On previous versions of python, I've installed into ~/Library/Python/$py_version_short/site-packages under my home directory. In my recollection, this has worked in the past for python installed from python.org, but perhaps that recollection is wrong. At any rate, it doesn't work with python 2.7. Best wishes, Nicholas -- http://mail.python.org/mailman/listinfo/python-list
Re: Network and socket programming in python
Am Sat, 04 Sep 2010 21:29:49 -0700 schrieb shivram: i want to learn network and socket programming but i would like to do this in python.Reason behind this is that python is very simple and the only language i know . anybody can suggest me which book should i pick. the book should have following specification-- 1)not tedious to follow 2)lots of example 3)starts with some networking stuff and then get into codes thanks in advance with regards. Ever heard of Google? python network tutorial / python socket tutorial should get you started. HTH. Martin -- http://mail.python.org/mailman/listinfo/python-list
High CPU utilization with Pylab
I am trying to plot a runtime graph using pylab. But the very purpose of functionality is lost because of high CPU percentage hogged by plotting the graph. Here is the piece of code which I have written. def timerfunc(ulcm, dlcm): count=0 xaxis=[] yaxis=[] ion() while 1: xaxis.append(count) dr=(((dlcm.sum)/dlcm.timer)*8) dlcm.counter=0 yaxis.append(dr) dlcm.sum=0 h=plot(xaxis,yaxis,'b') draw() count=count+1 time.sleep(dlcm.timer/1000.0) if (count100): xlim(count-100, count) Here is the function which plots the graph. Is there something which does not makes sense .? -Madhur -- http://mail.python.org/mailman/listinfo/python-list
Re: Network and socket programming in python
On Sat, Sep 4, 2010 at 9:29 PM, shivram shivramsha...@gmail.com wrote: i want to learn network and socket programming but i would like to do this in python.Reason behind this is that python is very simple and the only language i know . anybody can suggest me which book should i pick. the book should have following specification-- 1)not tedious to follow 2)lots of example 3)starts with some networking stuff and then get into codes thanks in advance with regards. Foundations of Python Network Programming by John Goerzen. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Question about Reading Files
On 2010-09-05, genxtech jrmy.l...@gmail.com wrote: I am using Fedora 13. When I run the file command the response is that it is a 'data' file. If there are any tips on how to programatically figure out the format, I would greatly appreciate it. I tried python-magic from the fedora repositories, but was unable to get it working. However there was another project at github [1] which seems to be working, at least with python 2.6. An easy way to test whether it works is by unpacking it, going to the directory, opening the interactive shell and running 'import magic' and 'magic.from_file(testdata/test.pdf)' See help(magic) for more help [1] http://github.com/ahupp/python-magic -- http://mail.python.org/mailman/listinfo/python-list
Re: High CPU utilization with Pylab
Il Sun, 5 Sep 2010 03:05:54 -0700 (PDT), Madhur ha scritto: Is there something which does not makes sense .? 1. How, a computer, is supposed to handle your infinite loop? 2. Running ghraphs shuold be handled in totally different way: look at the stripcharting demo in this zip http://agni.phys.iit.edu/~kmcivor/wxmpl/downloads/wxmpl-1.3.1.zip -- http://mail.python.org/mailman/listinfo/python-list
Re: Subclassing by monkey-patching
Jason wrote: On Sep 5, 3:53 pm, Peter Otten __pete...@web.de wrote: m = gio.File(.).monitor_directory() C = type(m) 'C' will not necessarily be 'gio.FileMonitor' — I think the internals of the GIO methods might further subclass it in some way depending on what underlying monitors are available. That would only be relevant if it had worked, I think ;) A possible alternative may be a class that wraps a FileMonitor instead of subclassing it. I've been avoiding this because it involves a lot of boilerplate: the signals needs to be replicated and passed through, same for all GObject properties, same for the usual methods. I'd basically have to re-write the entire class in Python, and then tack on my methods. Otherwise I have to write two sets of methods for anything that touches this wrapped object. Still, if it's the only way, sure. Does it have to be gio.FileMonitor? pyinotify can automatically add new subdirectories out of the box. Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: Speed-up for loops
David Cournapeau courn...@gmail.com wrote in message news:mailman.455.1283665528.29448.python-l...@python.org... On Thu, Sep 2, 2010 at 7:02 PM, Michael Kreim mich...@perfect-kreim.de wrote: imax = 10 a = 0 for i in xrange(imax): a = a + 10 print a Unfortunately my Python Code was much slower [than Matlab] and I do not understand why. Getting the above kind of code fast requires the interpreter to be clever enough so that it will use native machine operations on a int type instead of converting back and forth between internal representations. Writing for i in xrange(10) you'd think would give it a clue, but it makes no difference. Matlab since version 6 I believe, has a JIT to do just that. There is no mature JIT-like implementation of python which will give you the same speed up for this exact case today. Or do I have to live with the fact that Matlab beats Python in this example? Yes. Without a JIT, python cannot hope to get the same kind of speeds for this kind of examples. That being said, neither matlab nor matlab are especially good at doing what you do in your example - for this exact operation, doing it in C or other compiled languages will be at least one order of magnitude faster. One order of magnitude (say 10-20x slower) wouldn't be so bad. That's what you might expect for a dynamically typed, interpreted language. But on my machine this code was more like 50-200x slower than C, for unaccelerated Python. Generally, you use matlab's vectorized operations, and in that case, numpy gives you similar performances (sometimes faster, sometimes slower, but in the same ballpark in general). That would simply be delegating Python to a scripting language. It would be nice if you could directly code low-level algorithms in it without relying on accelerators, and not have to wait two and a half minutes (or whatever) for a simple test to complete. -- bartc -- http://mail.python.org/mailman/listinfo/python-list
Re: Network and socket programming in python
On Sep 5, 3:29 pm, geremy condra debat...@gmail.com wrote: On Sat, Sep 4, 2010 at 9:29 PM, shivram shivramsha...@gmail.com wrote: i want to learn network and socket programming but i would like to do this in python.Reason behind this is that python is very simple and the only language i know . anybody can suggest me which book should i pick. the book should have following specification-- 1)not tedious to follow 2)lots of example 3)starts with some networking stuff and then get into codes thanks in advance with regards. Foundations of Python Network Programming by John Goerzen. Geremy Condra thank you so much for your concern -- http://mail.python.org/mailman/listinfo/python-list
Python [repair_cycorder_mov.py]
Trying to learn Python for a specific purpose to repair a quicktime file that's corrupted, is it even possible to create a proof-of- concept python script that generates a valid 'moov' atom from a corrupt .mov video ?, file size 1.3gb (Kodak camera failed to finalize file). [IMG]http://a.imageshack.us/img337/5503/cycorder.jpg[/IMG] By [URL=http://profile.imageshack.us/user/ctops].[/URL] [IMG]http://a.imageshack.us/img375/9254/finallm.jpg[/IMG] By [URL=http://profile.imageshack.us/user/ctops].[/URL] FORM METHOD=LINK ACTION=https://docs.google.com/leaf? id=0B52sCu28BET8ZjlkNTcwODAtYjFmOC00M2VlLTkxMTAtODVmZWM3NWNhNDBisort=namelayout=listnum=50INPUT TYPE=submit VALUE=Source Quicktime mov file 22mb /FORM -- http://mail.python.org/mailman/listinfo/python-list
Python script for repairing .mov file
Trying to learn Python for a specific purpose to repair a quicktime file that's corrupted, is it even possible to create a proof-of- concept python script that generates a valid 'moov' atom from a corrupt .mov video ?, file size 1.3gb (Kodak camera failed to finalize file). Mac-- http://a.imageshack.us/img337/5503/cycorder.jpg Windows-- http://a.imageshack.us/img375/9254/finallm.jpg Source sample Quicktime mov file 22mb; https://docs.google.com/leaf? id=0B52sCu28BET8ZjlkNTcwODAtYjFmOC00M2VlLTkxMTAtODVmZWM3NWNhNDBisort=name layout=listnum=50 Source thread; http://modmyi.com/forums/windows-specific-modding-discussion/587461-editing-mov-files-corrupt-cycorder-videos-8.html -- http://mail.python.org/mailman/listinfo/python-list
Python script for repairing .mov file ?
Trying to learn Python for a specific purpose to repair a quicktime file that's corrupted, is it even possible to create a proof-of- concept python script that generates a valid 'moov' atom from a corrupt .mov video ?, file size 1.3gb (Kodak camera failed to finalize file). Mac-- http://a.imageshack.us/img337/5503/cycorder.jpg Windows-- http://a.imageshack.us/img375/9254/finallm.jpg Source sample Quicktime mov file 22mb; https://docs.google.com/leaf?id=0B52sCu28BET8ZjlkNTcwODAtYjFmOC00M2VlLTkxMTAtODVmZWM3NWNhNDBisort=namelayout=listnum=50 Source thread; http://modmyi.com/forums/windows-specific-modding-discussion/587461-editing= -mov-files-corrupt-cycorder-videos-8.html -- http://mail.python.org/mailman/listinfo/python-list
Re: Python [repair_cycorder_mov.py]
On Sun, 05 Sep 2010 05:44:16 -0700, ctops.legal wrote: Trying to learn Python for a specific purpose to repair a quicktime file that's corrupted, is it even possible to create a proof-of- concept python script that generates a valid 'moov' atom from a corrupt .mov video ?, file size 1.3gb (Kodak camera failed to finalize file). Yes, Python is a Turing complete programming language. Anything you can do in any programming language, you can do in Python. By the way, there's no need to send three messages in 10 minutes asking the same question, and adding FORM METHOD links to your post will probably just get it flagged as spam by many people. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
ssh browser? where?
why not ssh browser traffic? why use SSL certificate authorities which can't be trusted in the first place? Is SSH not proven to be secure? To this day I have not seen ssh module for say Apache web server, why not? I understand this maybe wrong list to ask this question, but I love you guys so much at python and I think your smart... :) Thank you, -Alex Goretoy -- http://mail.python.org/mailman/listinfo/python-list
Re: ssh browser? where?
On Sunday 05 September 2010, it occurred to alex goretoy to exclaim: why not ssh browser traffic? why use SSL certificate authorities which can't be trusted in the first place? Is SSH not proven to be secure? To this day I have not seen ssh module for say Apache web server, why not? I understand this maybe wrong list to ask this question, but I love you guys so much at python and I think your smart... :) Umn. Yes, wrong list. Still, SSH is secure shell. Shell ≠ WWW. TLS/SSL is secure. Once you have an encrypted connection to the correct party, the information you send over that connection is just as illegible to the outsider as it'd be over an SSH connection. If you're connected to someone other than you think you are connected to, then all the encryption in the world won't help you. (man-in-the-middle attack) With SSH, this isn't that big a problem: you only connect to a machine over SSH if you have an account there. Since this tends to be a relatively small number of machines, it's relatively easy to manually check the server's key fingerprint. The authenticity of the server's key must be checked to prevent an attack, and the status quo with SSH is that the user does it, which is arguably the most secure model anyway. On the web, you might connect to any number of servers. Manually checking every single key would be a gargantuan and certainly not very rewarding task. THAT's where certificate authorities come in. They're a reasonable way to automatically check the validity of a server's key, so you don't have to. Now, in theory, you should verify the integrity of the certificate authorities yourself before trusting their judgement, but in practice, you leave that daunting task to your operating system and/or web browser vendor. There are a number of weak spots, but there are also rather strict audits going on all over the place. If we were to use SSH on the web, which is certainly not the point of SSH, we'd still need some kind of certificate authority to make the whole system workable. -- http://mail.python.org/mailman/listinfo/python-list
Re: ssh browser? where?
If we were to use SSH on the web, which is certainly not the point of SSH, we'd still need some kind of certificate authority to make the whole system workable. Yeah, you are correct. I thought about that after having posted these questions. Even though it was SSH there still would be wiggle room for a mitm. I had a dream last night that my future employer was asking me what SSH was. lol I appreciate your response and making it clear as day. Still though, I think there maybe something for testing here. I personally verify and scope out my cert manually. Well for the sites I visit often. But I understand not everyone will be likely to do this. Thank you, -Alex Goretoy -- http://mail.python.org/mailman/listinfo/python-list
Re: Speed-up for loops
On Fri, 03 Sep 2010 21:17:44 +0100, BartC wrote:
I'm not sure the Python developers were interested in getting fast
loops.
For-loops which iterate between two numbers are amongst the easiest
things to make fast in a language. Yet originally you had to use:
for i in range(N):
I don't have any versions of Python prior to version 1.5, but as far back
as that there was always a choice between creating a list with range()
and a lazy iterator with xrange().
which (if I understood correctly) actually created a list of N objects,
populated it with the values 0, 1, 2...N-1 (presumably using a more
sensible loop), then iterated between the values of the list!
By more sensible, do you mean in C code? If so, then you are correct.
So Python had the distinction of being one of the slowest languages in
which to do nothing (ie. running an empty loop).
Nonsense.
[st...@sylar ~]$ time python test.py
real0m3.441s
user0m2.969s
sys 0m0.024s
[st...@sylar ~]$ time perl test.pl
real0m3.490s
user0m2.722s
sys 0m0.011s
[st...@sylar ~]$ time ruby test.rb
real0m11.875s
user0m6.740s
sys 0m3.995s
The difference between an empty loop in Python and Perl is insignificant,
and much faster than Ruby (at least the specific version of Ruby
installed on my machine, 1.8.6).
And if you want to see the code I ran:
[st...@sylar ~]$ cat test.*
# perl
for ($i = 0; $i 10_000_000; ++$i) {
1;
}
# python
for i in xrange(1000):
1
# ruby
for i in 0...1000
1
end
Just for comparisons' sake:
[st...@sylar ~]$ gpc empty_test.p
[st...@sylar ~]$ time ./a.out
real0m0.106s
user0m0.070s
sys 0m0.004s
[st...@sylar ~]$ cat empty_test.p
program main(input, output);
var
i: integer;
begin
for i := 0 to 1000 do
begin
end;
end.
Of course, a real optimizing compiler would realise that the Pascal code
did nothing at all, and compile it all away to an empty a.out file...
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python script for repairing .mov file ?
On 5 September 2010 14:54, ctops.legal ctops.le...@gmail.com wrote: Trying to learn Python for a specific purpose to repair a quicktime file that's corrupted, is it even possible to create a proof-of- concept python script that generates a valid 'moov' atom from a corrupt .mov video ?, file size 1.3gb (Kodak camera failed to finalize file). Why would you use Python for this? Would ffmpeg or mencoder not be more suitable? Anyway, there's pymedia, but I have no experience with it. I'm not sure what I should see in the links. The Soure thread one is broken by the way (404 error). Almar -- http://mail.python.org/mailman/listinfo/python-list
Re: Speed-up for loops
On Sun, 05 Sep 2010 12:28:47 +0100, BartC wrote: Getting the above kind of code fast requires the interpreter to be clever enough so that it will use native machine operations on a int type instead of converting back and forth between internal representations. Writing for i in xrange(10) you'd think would give it a clue, but it makes no difference. CPython takes a very conservative view towards runtime optimizations. Optimizations don't happen for free, you know, they have costs. Memory is one cost, but human time and effort is another. But if you want a JIT compiler, see Psycho, or try PyPy, which is very exciting and I hope will one day be ready to take over from CPython as the first choice for production use. [...] One order of magnitude (say 10-20x slower) wouldn't be so bad. That's what you might expect for a dynamically typed, interpreted language. But on my machine this code was more like 50-200x slower than C, for unaccelerated Python. I'd say that 50-200 times slower than C is exactly what I'd expect from a dynamically typed language like Python without any fancy JIT tricks. Getting such a language to within an order of magnitude of C is quite an achievement. Generally, you use matlab's vectorized operations, and in that case, numpy gives you similar performances (sometimes faster, sometimes slower, but in the same ballpark in general). That would simply be delegating Python to a scripting language. That's sheer unadulterated nonsense. In any case, Python was designed as a glue language, specifically to be a high-level user-friendly language for gluing components written in C together. That's what Python *is* -- it provides a bunch of primitives, written in C (or Java, or dot-Net, pick whichever implementation you like) and manipulated in a friendly, safe language. Calling numpy for fast vectorized operations is *exactly* the right solution, if you need high-performance maths calculations. Use the right tool for the job, don't insist that your spanner should double as a screwdriver. It would be nice if you could directly code low-level algorithms in it without relying on accelerators, and not have to wait two and a half minutes (or whatever) for a simple test to complete. Yes, and it would be nice if my clothes washed and ironed themselves, but they don't. Somebody has to do the work. Are you volunteering to write the JIT compiler for CPython? Will you contribute to the PyPy project, or help maintain Psycho, or are you just bitching? The simple fact is that there are far more important things for Python developers to spend their time and effort on than optimizations like this. If such an optimization comes out of the PyPy project, I'll be cheering them on -- but it's a lot of effort for such a trivial gain. The example given by the Original Poster is contrived. Nobody sensibly writes an integer multiplication as a repeated addition like that, and any real code that would need to be put in a for loop is almost certainly going to be too complicated for the JIT compiler to benefit greatly. The loop overhead itself will almost certainly be overwhelmed by the work done in the loop: [st...@sylar ~]$ time python -c a = 0 for i in xrange(1000): a += 10 real0m6.906s user0m5.820s sys 0m0.022s which is about double the time for an empty loop of the same size. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
doctest annoyance/puzzle
I'm writing a literate programming document, example.txt, which mixes text and code in ReST format: This is my excellent module for making spam. It has one public function, ``make_spam``, which takes a single argument for how much spam to make: from module import make_spam make_spam(3) 'spam spam spam' If you ask for too little spam, you get a bonus: make_spam(-3) 'spam spam spam GLORIOUS SPAM!!!' I run the doctests with: python2.6 -m doctest examples.txt and the first example passes, but the second fails with NameError: make_spam not defined. I have a work-around, I do an import at the start of *every* doctest block, but that's ugly. More complex (realistic) examples sometimes need setup code, which I'd like to survive from one block to the next: it = iter(make_spam(5).split()) next(it) 'spam' text goes here next(it) 'spam' but of course that also fails. This surprises me, because doctests in functions or classes can mix descriptive text between tests without this problem. Am I doing something wrong, or is doctest fundamentally broken when executing tests in a text file? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
personal firewall software is blocking the connection
I'am getting this annoying message all the time when using IDLE. ... personal firewall software is blocking the connection When I am editing a source *.py file and I click on F5 (run) I get the above message. What can I do? I am using Norton Internet Security. I try to find if 127.0.0.1 is blocked, but I do not know where to look. Can anybody help? -- http://mail.python.org/mailman/listinfo/python-list
IDLE: personal firewall software is blocking the connection
On 5 sep, 18:00, vsoler vicente.so...@gmail.com wrote: I'am getting this annoying message all the time when using IDLE. ... personal firewall software is blocking the connection When I am editing a source *.py file and I click on F5 (run) I get the above message. What can I do? I am using Norton Internet Security. I try to find if 127.0.0.1 is blocked, but I do not know where to look. Can anybody help? -- http://mail.python.org/mailman/listinfo/python-list
Re: Python [repair_cycorder_mov.py]
On Sun, 2010-09-05 at 14:00 +, Steven D'Aprano wrote: By the way, there's no need to send three messages in 10 minutes asking the same question, and adding FORM METHOD links to your post will probably just get it flagged as spam by many people. Apparently it has, as I only got this response. -- http://mail.python.org/mailman/listinfo/python-list
Re: Speed-up for loops
Steven D'Aprano, 05.09.2010 17:00: Of course, a real optimizing compiler would realise that the Pascal code did nothing at all, and compile it all away to an empty a.out file... Which is just one of the reasons why this kind if benchmark provides no insight into anything that should have an impact on a programmer's daily job. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: doctest annoyance/puzzle
Hi Steven, On 2010-09-05 17:30, Steven D'Aprano wrote: I run the doctests with: python2.6 -m doctest examples.txt and the first example passes, but the second fails with NameError: make_spam not defined. I run my doctests by calling doctest.testfile(filename) for each file in a loop. This way, names and their associated objects survice from one code block to the next. I just read that the way you use doctest should behave the same, according to the documentation. In case of a text file I just tested, it does; all tests pass despite the text snippets between the code blocks. What do you get if you test your text file by explicitly calling doctest.testfile? Stefan -- http://mail.python.org/mailman/listinfo/python-list
Extracting item from list of tuples?
I'm using httplib, and want to get the Location header from the response. The getheaders() method gives you back a list of (name, value) tuples. It would be a lot more convenient if it gave you back a dict, but it is what it is. Anyway, I came up with: location = [t[1] for t in headers if t[0] == 'location'][0] which works, but is getting on towards the cryptic end of things. Is there a better idiom for this? -- http://mail.python.org/mailman/listinfo/python-list
Re: Speed-up for loops
Steven D'Aprano st...@remove-this-cybersource.com.au wrote in message news:4c83b425$0$28657$c3e8...@news.astraweb.com... On Sun, 05 Sep 2010 12:28:47 +0100, BartC wrote: It would be nice if you could directly code low-level algorithms in it without relying on accelerators, and not have to wait two and a half minutes (or whatever) for a simple test to complete. Yes, and it would be nice if my clothes washed and ironed themselves, but they don't. Somebody has to do the work. Are you volunteering to write the JIT compiler for CPython? Will you contribute to the PyPy project, or help maintain Psycho, or are you just bitching? I've thought about it (writing an independent interpreter). But I don't know enough of the language, and a lot of it I don't understand (eg. OOP). Besides, I think the language itself deliberately makes it difficult to get it up to speed. Some of the reasons might be the ones set down here, in Chapter 2: http://codespeak.net/svn/user/antocuni/phd/thesis/thesis.pdf (Instead, I've confined my efforts to my own projects; the last interpreter I worked on did in fact get within a magnitude of C in performance, without using JIT or other fancy tricks.) The simple fact is that there are far more important things for Python developers to spend their time and effort on than optimizations like this. I don't know, there's seem to be an awful lot of effort going into addressing exactly this issue (eg. PyPy, Pscyo, Unladen Swallow, Shedskin) If such an optimization comes out of the PyPy project, I'll be cheering them on -- but it's a lot of effort for such a trivial gain. The example given by the Original Poster is contrived. Nobody sensibly writes an integer multiplication as a repeated addition like that, and any real code that would need to be put in a for loop is almost certainly going to be too complicated for the JIT compiler to benefit greatly. The loop overhead itself will almost certainly be overwhelmed by the work done in the loop: OK, you're saying there's no point in reducing the loop overhead, because the loop body is going to be even slower. All those compilers that offer loop unrolling are therefore wasting their time... -- Bartc -- http://mail.python.org/mailman/listinfo/python-list
Re: Extracting item from list of tuples?
On Sunday 05 September 2010, it occurred to Roy Smith to exclaim:
I'm using httplib, and want to get the Location header from the
response. The getheaders() method gives you back a list of (name,
value) tuples. It would be a lot more convenient if it gave you back a
dict, but it is what it is.
It may not be a dict, but you can always make it one:
L = [('a',1), ('b', 42), ('c', 0)]
dict(L)
{'a': 1, 'c': 0, 'b': 42}
--
http://mail.python.org/mailman/listinfo/python-list
Re: Extracting item from list of tuples?
On Sep 5, 2010, at 1:09 PM, Roy Smith wrote: I'm using httplib, and want to get the Location header from the response. The getheaders() method gives you back a list of (name, value) tuples. It would be a lot more convenient if it gave you back a dict, but it is what it is. Anyway, I came up with: location = [t[1] for t in headers if t[0] == 'location'][0] which works, but is getting on towards the cryptic end of things. Is there a better idiom for this? Are you concerned about efficiency? If it's not important, being less terse would make it less cryptic: location = None for name, value in headers: if name == location: location = value break If you prefer terse, IMHO this is less cryptic (but untested): location = [value for name, value in headers if name == 'location'] [0] Be careful if you decide to turn the list of tuples into a dictionary. HTTP headers can repeat and straightforward code to turn the (name, value) pairs into a dict could end up overwriting data. Cheers P -- http://mail.python.org/mailman/listinfo/python-list
Re: Extracting item from list of tuples?
Philip Semanchuk wrote: On Sep 5, 2010, at 1:09 PM, Roy Smith wrote: I'm using httplib, and want to get the Location header from the response. The getheaders() method gives you back a list of (name, value) tuples. It would be a lot more convenient if it gave you back a dict, but it is what it is. Anyway, I came up with: location = [t[1] for t in headers if t[0] == 'location'][0] which works, but is getting on towards the cryptic end of things. Is there a better idiom for this? Are you concerned about efficiency? If it's not important, being less terse would make it less cryptic: location = None for name, value in headers: if name == location: location = value break This is actually more efficient because unlike Roy's (and your other solution) it doesn't build a list, and it doesn't waste cycles in the loop once the location is found. You can write it as a generator expression like so: location = next((value for name, value in headers if name == location), None) But just converting the headers to a dict is clearer and probably even faster because it moves the loop from Python to C. If you prefer terse, IMHO this is less cryptic (but untested): location = [value for name, value in headers if name == 'location'] [0] Be careful if you decide to turn the list of tuples into a dictionary. HTTP headers can repeat and straightforward code to turn the (name, value) pairs into a dict could end up overwriting data. -- http://mail.python.org/mailman/listinfo/python-list
Re: Queue cleanup
On 9/4/2010 11:51 PM, Paul Rubin wrote: John Naglena...@animats.com writes: Unoptimized reference counting, which is what CPython does, isn't all that great either. The four big bottlenecks in Python are boxed numbers, attribute lookups, reference count updates, and the GIL. The performance hit of having to lock the refcounts before update has been the historical reason for keeping the GIL. The LOCK prefix takes something like 100 cycles on an x86. Is optimizing the refcount updates going to anywhere near make up for that? I've argued for an approach in which only synchronized or immutable objects can be shared between threads. Then, only synchronized objects have refcounts. See http://www.animats.com/papers/languages/pythonconcurrency.html; Guido doesn't like it. He doesn't like any restrictions. So we're stuck dragging around the boat anchor. I'd hoped that the Unladen Swallow people might come up with some really clever solution, but they seem to be stuck. It's been almost a year since the last quarterly release. Maybe Google is putting their effort into Go. What's so striking is that Shed Skin can deliver 20x to 60x speedups over CPython, while PyPy and Unladen Swallow have trouble getting 1.5x. The question is how much one has to restrict the language to get a serious performance improvement. Python's with statement as an approach to RAII has seemed ok to me. I can't think of a time when I've really had to use a finalizer for something with dynamic extent. They've always seemed like a code smell to me. The problem appears when you have an object that owns something, like a window or a database connection. With is single-level. John Nagle -- http://mail.python.org/mailman/listinfo/python-list
Re: Python script for repairing .mov file ?
On Sep 5, 8:54 am, ctops.legal ctops.le...@gmail.com wrote: Trying to learn Python for a specific purpose to repair a quicktime file that's corrupted, is it even possible to create a proof-of- concept python script that generates a valid 'moov' atom from a corrupt .mov video ?, file size 1.3gb (Kodak camera failed to finalize file). Mac--http://a.imageshack.us/img337/5503/cycorder.jpg Windows--http://a.imageshack.us/img375/9254/finallm.jpg Source sample Quicktime mov file 22mb; https://docs.google.com/leaf?id=0B52sCu28BET8ZjlkNTcwODAtYjFmOC00M2VlLTkxMTAtODVmZWM3NWNhNDBihl=en Source thread;http://modmyi.com/forums/windows-specific-modding-discussion/587461-e... -mov-files-corrupt-cycorder-videos-8.html -- http://mail.python.org/mailman/listinfo/python-list
Re: Extracting item from list of tuples?
On Sep 5, 2010, at 1:45 PM, Peter Otten wrote: Philip Semanchuk wrote: On Sep 5, 2010, at 1:09 PM, Roy Smith wrote: I'm using httplib, and want to get the Location header from the response. The getheaders() method gives you back a list of (name, value) tuples. It would be a lot more convenient if it gave you back a dict, but it is what it is. Anyway, I came up with: location = [t[1] for t in headers if t[0] == 'location'][0] which works, but is getting on towards the cryptic end of things. Is there a better idiom for this? Are you concerned about efficiency? If it's not important, being less terse would make it less cryptic: location = None for name, value in headers: if name == location: location = value break This is actually more efficient because unlike Roy's (and your other solution) it doesn't build a list, and it doesn't waste cycles in the loop once the location is found. Upon further reflection, though, I realize that my solution is not equivalent to the original. In the case of duplicated headers, the original would return the last value of the location header, my code would return the first. For the HTTP Location header this is only a theoretical problem because the HTTP spec doesn't provide for sending more than one. However if this code were made into a function for retrieving the value of an arbitrary header (as seems likely), one would have to decide how to handle repeating headers. Neither my code nor the original handle that possibility. Nor does a naive conversion into a dict, as I mentioned before. Cheers P You can write it as a generator expression like so: location = next((value for name, value in headers if name == location), None) But just converting the headers to a dict is clearer and probably even faster because it moves the loop from Python to C. If you prefer terse, IMHO this is less cryptic (but untested): location = [value for name, value in headers if name == 'location'] [0] Be careful if you decide to turn the list of tuples into a dictionary. HTTP headers can repeat and straightforward code to turn the (name, value) pairs into a dict could end up overwriting data. -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: Python script for repairing .mov file ?
On Sep 5, 8:54 am, ctops.legal ctops.le...@gmail.com wrote: Trying to learn Python for a specific purpose to repair a quicktime file that's corrupted, is it even possible to create a proof-of- concept python script that generates a valid 'moov' atom from a corrupt .mov video ?, file size 1.3gb (Kodak camera failed to finalize file). Mac--http://a.imageshack.us/img337/5503/cycorder.jpg Windows--http://a.imageshack.us/img375/9254/finallm.jpg Source sample Quicktime mov file 22mb; https://docs.google.com/leaf?id=0B52sCu28BET8ZjlkNTcwODAtYjFmOC00M2Vl... Source thread;http://modmyi.com/forums/windows-specific-modding-discussion/587461-e... -mov-files-corrupt-cycorder-videos-8.html GOOD Link for source mov file-- https://docs.google.com/uc?id=0B52sCu28BET8ZjlkNTcwODAtYjFmOC00M2VlLTkxMTAtODVmZWM3NWNhNDBiexport=downloadhl=en -- http://mail.python.org/mailman/listinfo/python-list
Re: Speed-up for loops
BartC, 05.09.2010 19:09: I've thought about it (writing an independent interpreter). But I don't know enough of the language, and a lot of it I don't understand (eg. OOP). Besides, I think the language itself deliberately makes it difficult to get it up to speed. Some of the reasons might be the ones set down here, in Chapter 2: http://codespeak.net/svn/user/antocuni/phd/thesis/thesis.pdf Second sentence of the abstract: However, all the existing implementations prevent programmers from developing very efficient code. For an incorrect statement, that's a pretty bold one, even in the context given by the abstract. The author is lucky not to be an English native speaker, which explains some of the offensive wording in the intro. Also, I'm impressed by an accepted Ph.D. thesis that comes along with just a bit more than four pages of references, and that fails to reference basically everything that the Python ecosystem provides for fast computation. I wouldn't mind a faster Python, but as long as Python continues to be a language that allows you to very quickly get to the point where you can benchmark and optimise your code, and as long as CPython then makes it easy to do that, I won't be the one jumping up and down for a small factor of general improvement, especially when it comes at the price of switching to a completely different runtime. After all, it's really hard to appreciate that a program can now wait 5% more often on I/O than before. Larger performance improvements are usually due to algorithmic changes (including data structure adaptations and caching), rarely due to changes in the interpreter, especially when it's as old and well optimised as CPython. I think the CPython interpreter does a really good job in providing a stable and predictable runtime environment and executing code in it, and the CPython ecosystem does a really good job in providing tools that make code run fast that needs to, be it due to efficient usage of I/O, CPU, memory, or whatever. I'm not trying to put down the achievements of the author of the cited thesis, not at all. I'm sure the results are interesting for some people and for some kinds of applications, just like the different Python implementations are interesting for some people and some applications. But an awful lot of existing apps won't benefit at all from a fast CLI based Python implementation, simply because it doesn't have access (or at least no efficient access) to many of the existing tools in the CPython ecosystem. The simple fact is that there are far more important things for Python developers to spend their time and effort on than optimizations like this. I don't know, there's seem to be an awful lot of effort going into addressing exactly this issue (eg. PyPy, Pscyo, Unladen Swallow, Shedskin) Those are very different efforts that address very different issues. All those compilers that offer loop unrolling are therefore wasting their time... Sometimes they do, yes. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 2.7 module path problems on OS X
In article
aanlktim4fk9y2mabs3h14z+5rkbxtgmp27vyvmkqe...@mail.gmail.com,
Nicholas Cole nicholas.c...@gmail.com wrote:
On Sun, Sep 5, 2010 at 10:20 AM, Ned Deily n...@acm.org wrote:
I'm not sure why you think it is broken. The Apple 2.6 and the
python.org 2.7 have different site-package directories in different
locations. That is to be expected. The Apple-supplied Python comes
with some additional packages pre-installed, like setuptools, PyObjC,
and wx. But you would need to install new versions of these for 2.7
anyway.
Sorry. I wasn't clear. Of course I understand that each version of
python needs its own additional packages, and I was expecting to
re-install them. But I don't want to install system-wide -- I'd like
to have my own, user-specific, site-packages. On previous versions of
python, I've installed into
~/Library/Python/$py_version_short/site-packages under my home
directory.
In my recollection, this has worked in the past for python installed
from python.org, but perhaps that recollection is wrong. At any rate,
it doesn't work with python 2.7.
Ah, my apologies. I overlooked the ~/Library part. So what you have
been using is the setuptools Mac OS X User Installation configuration:
(http://peak.telecommunity.com/DevCenter/EasyInstall#mac-os-x-user-instal
lation). Yes, it seems an incompatibility has been introduced in 2.7
for OS X. There has been an on-going and as yet unresolved discussion
on the python-dev list about the topic of user directory and config file
locations so things will likely change and get more consistent in the
future. But this particular problem is more immediate. I believe the
problem here is that the default user directory path was changed between
2.6 and 2.7 from:
~/Library/Python/m.n/site-packages
to
~/Library/Python/m.n/lib/python/site-packages
I will track that down further and open an issue for it, if necessary.
In the meantime, here's a little snippet that I think should work around
the problem for the moment without requiring a change to your
configuration files. (For simplicity of installation, it uses the
Distribute fork of setuptools). The main point is to create a symlink
between the old and new locations. Let me know if this works for you:
$ cat .pydistutils.cfg
[install]
install_lib = ~/Library/Python/$py_version_short/site-packages
install_scripts = ~/bin
$ rm -r ~/Library/Python/2.7 # REMOVES ANY EXISTING LOCAL PACKAGES
$ mkdir -p ~/Library/Python/2.7/lib/python/site-packages
$ ( cd ~/Library/Python/2.7 ; ln -s lib/python/site-packages
site-packages )
$ curl -s http://python-distribute.org/distribute_setup.py -o
/tmp/distr.py
$ python2.7 /tmp/distr.py
Extracting in /var/folders/pD/pD-bOKKQHluYv0YkNWJhPE+++TI/-Tmp-/tmpF5sVf7
Now working in
/var/folders/pD/pD-bOKKQHluYv0YkNWJhPE+++TI/-Tmp-/tmpF5sVf7/distribute-0.
6.14
Installing Distribute
[...]
$ ~/bin/easy_install
easy_install easy_install-2.6 easy_install-2.7
$ ~/bin/easy_install-2.7 appscript
[...]
$ python2.7 -c from appscript import *;print(app('Finder').version())
10.5.8
$ ls -l ~/Library/Python/2.7
total 8
drwxr-x--- 3 nad staff 102 Sep 5 12:20 lib/
lrwxr-x--- 1 nad staff 24 Sep 5 12:20 site-packages@ -
lib/python/site-packages
$ ls -L ~/Library/Python/2.7/site-packages
appscript-0.21.1-py2.7-macosx-10.3-fat.egg/ easy-install.pth
distribute-0.6.14-py2.7.egg/setuptools.pth
--
Ned Deily,
n...@acm.org
--
http://mail.python.org/mailman/listinfo/python-list
Class changes in circular imports when __name__ == '__main__'
Hi! I'm writing a package with several files in it, and I've found that isinstance doesn't work the way I expect under certain circumstances. Short example: here are two files. # fileone.py import filetwo class AClass( object ): pass if __name__ == '__main__': a = AClass() filetwo.is_aclass( a ) # filetwo.py import fileone def is_aclass( a ): print The argument is, ( if isinstance(a, fileone.AClass) else not), an instance of fileone.AClass If you run fileone.py, it will tell you that The argument is not an instance of fileone.AClass, which seems strange to me, given that the fileone module is the one that CREATES the object with its own AClass class. And if you replace if __name__ == '__main__' with def main(), start Python, import fileone, and call fileone.main(), it tells you that the argument IS an instance of AClass. So, the module's name change to __main__ when you run it on its own... well, it looks like it puts all of the things defined in fileone in the __main__ namespace INSTEAD of in the fileone module's namespace, and then when filetwo imports fileone, the class is created again, this time as fileone.AClass, and though it's identical in function to __main__.AClass, one is not the other. Is this kind of doubled-back 'isinstance' inherently sinful? I mean, I could solve this problem by giving all of my classes classname attributes or something, but maybe it's just a sign that I shouldn't have to do this in the first place. -- http://mail.python.org/mailman/listinfo/python-list
Re: Class changes in circular imports when __name__ == '__main__'
Spencer Pearson speeze.pear...@gmail.com writes: Hi! I'm writing a package with several files in it, and I've found that isinstance doesn't work the way I expect under certain circumstances. Short example: here are two files. # fileone.py import filetwo class AClass( object ): pass if __name__ == '__main__': a = AClass() filetwo.is_aclass( a ) # filetwo.py import fileone def is_aclass( a ): print The argument is, ( if isinstance(a, fileone.AClass) else not), an instance of fileone.AClass If you run fileone.py, it will tell you that The argument is not an instance of fileone.AClass, which seems strange to me, given that the fileone module is the one that CREATES the object with its own AClass class. And if you replace if __name__ == '__main__' with def main(), start Python, import fileone, and call fileone.main(), it tells you that the argument IS an instance of AClass. So, the module's name change to __main__ when you run it on its own... well, it looks like it puts all of the things defined in fileone in the __main__ namespace INSTEAD of in the fileone module's namespace, and then when filetwo imports fileone, the class is created again, this time as fileone.AClass, and though it's identical in function to __main__.AClass, one is not the other. Is this kind of doubled-back 'isinstance' inherently sinful? I mean, I could solve this problem by giving all of my classes classname attributes or something, but maybe it's just a sign that I shouldn't have to do this in the first place. The behaviour is normal. I suppose you could do something like this (untested): # fileone.py if __name__ == '__main__': from fileone import * a = AClass() filetwo.is_aclass( a ) import sys; sys.exit() import filetwo class AClass( object ): pass -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
accessing a text file
level: beginner how can i access the contents of a text file in Python? i would like to compare a string (word) with the content of a text file (word_list). i want to see if word is in word_list. let's assume the TXT file is stored in the same directory as the PY file. def is_valid_word(word, word_list) thanks Baba -- http://mail.python.org/mailman/listinfo/python-list
Re: accessing a text file
may be something like this f = open (file,r) data = f.read() f.close if word in data: print word, is present in file On Mon, Sep 6, 2010 at 3:17 AM, Baba raoul...@gmail.com wrote: level: beginner how can i access the contents of a text file in Python? i would like to compare a string (word) with the content of a text file (word_list). i want to see if word is in word_list. let's assume the TXT file is stored in the same directory as the PY file. def is_valid_word(word, word_list) thanks Baba -- http://mail.python.org/mailman/listinfo/python-list -- Nitin Pawar -- http://mail.python.org/mailman/listinfo/python-list
Re: accessing a text file
On 09/05/10 16:47, Baba wrote:
level: beginner
how can i access the contents of a text file in Python?
i would like to compare a string (word) with the content of a text
file (word_list). i want to see if word is in word_list. let's assume
the TXT file is stored in the same directory as the PY file.
def is_valid_word(word, word_list)
thanks
Baba
f = open('text.txt')
data = f.read()
# You may want to convert it to a list
data = data.split()
# Returns true if word is in data, false otherwise
word in data
--
http://mail.python.org/mailman/listinfo/python-list
Re: accessing a text file
On Sun, Sep 5, 2010 at 5:47 PM, Baba raoul...@gmail.com wrote: level: beginner how can i access the contents of a text file in Python? i would like to compare a string (word) with the content of a text file (word_list). i want to see if word is in word_list. let's assume the TXT file is stored in the same directory as the PY file. def is_valid_word(word, word_list) thanks Baba -- http://mail.python.org/mailman/listinfo/python-list Please do us a favor and at least try to figure things out on your own, rather than coming here every time you have a question. The very first result when you try searching python read text file is the section in the Python tutorial that explains how to do this. http://docs.python.org/tutorial/inputoutput.html#reading-and-writing-files -- http://mail.python.org/mailman/listinfo/python-list
Re: accessing a text file
On 05/09/2010 22:47, Baba wrote: level: beginner how can i access the contents of a text file in Python? That's a very basic question. I suggest you read a tutorial such as Dive Into Python: http://diveintopython.org/toc/index.html i would like to compare a string (word) with the content of a text file (word_list). i want to see if word is in word_list. let's assume the TXT file is stored in the same directory as the PY file. def is_valid_word(word, word_list) -- http://mail.python.org/mailman/listinfo/python-list
Re: accessing a text file
Baba wrote: level: beginner how can i access the contents of a text file in Python? i would like to compare a string (word) with the content of a text file (word_list). i want to see if word is in word_list. let's assume the TXT file is stored in the same directory as the PY file. def is_valid_word(word, word_list) thanks Baba Completely untested: def is_valid_word(word, filename): for line in open(filename): if word in line.split(): return True return False -- http://mail.python.org/mailman/listinfo/python-list
include a file in a python program
i've got a python.txt that contain python and it must stay as it (python.txt) how can i include it in my program ? import python.txt doesn't work is there a way : a) to make an include(python.txt) b) tell him to treat .txt as .py file that i can make an import python ? i'am using python3 Regards Bussiere fan of torchwood Google Fan boy -- http://mail.python.org/mailman/listinfo/python-list
Re: accessing a text file
Em 05-09-2010 19:06, Alexander Kapps escreveu: Baba wrote: level: beginner how can i access the contents of a text file in Python? i would like to compare a string (word) with the content of a text file (word_list). i want to see if word is in word_list. let's assume the TXT file is stored in the same directory as the PY file. def is_valid_word(word, word_list) thanks Baba Completely untested: def is_valid_word(word, filename): for line in open(filename): if word in line.split(): return True return False Damn, ironiceasy like C/C++/Java/ironic Thanks a lot! signature.asc Description: OpenPGP digital signature -- http://mail.python.org/mailman/listinfo/python-list
Re: doctest annoyance/puzzle
On Sun, 05 Sep 2010 18:41:15 +0200, Stefan Schwarzer wrote: Hi Steven, On 2010-09-05 17:30, Steven D'Aprano wrote: I run the doctests with: python2.6 -m doctest examples.txt and the first example passes, but the second fails with NameError: make_spam not defined. I run my doctests by calling doctest.testfile(filename) for each file in a loop. This way, names and their associated objects survice from one code block to the next. That's fine if you're calling it from Python, but I want a way to run the tests from the shell, and the advertised way to do that is by calling doctest.py as a script. The module calls testfile, so it should work. And now it's working for me. I have *no idea* what happened there... it wasn't working for me before, and now it is. Sorry for the noise :( -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Class changes in circular imports when __name__ == '__main__'
On Sep 5, 1:19 pm, Spencer Pearson speeze.pear...@gmail.com wrote: Hi! I'm writing a package with several files in it, and I've found that isinstance doesn't work the way I expect under certain circumstances. Short example: here are two files. # fileone.py import filetwo class AClass( object ): pass if __name__ == '__main__': a = AClass() filetwo.is_aclass( a ) # filetwo.py import fileone def is_aclass( a ): print The argument is, ( if isinstance(a, fileone.AClass) else not), an instance of fileone.AClass If you run fileone.py, it will tell you that The argument is not an instance of fileone.AClass, which seems strange to me, given that the fileone module is the one that CREATES the object with its own AClass class. And if you replace if __name__ == '__main__' with def main(), start Python, import fileone, and call fileone.main(), it tells you that the argument IS an instance of AClass. So, the module's name change to __main__ when you run it on its own... well, it looks like it puts all of the things defined in fileone in the __main__ namespace INSTEAD of in the fileone module's namespace, and then when filetwo imports fileone, the class is created again, this time as fileone.AClass, and though it's identical in function to __main__.AClass, one is not the other. Correct. Python always treats the main script as a module called __main__. If you then try to import the main script file from another module, Python will actually import it again with whatever its usual name is. This is easily one of the most confusing and unfortunate aspects of Python. Is this kind of doubled-back 'isinstance' inherently sinful? I mean, I could solve this problem by giving all of my classes classname attributes or something, but maybe it's just a sign that I shouldn't have to do this in the first place. Even if there are better ways than isinstance, the weird behavior of __main__ shouldn't be the reason not to use it. My recommendation for most programmers is to treat Python files either as scripts (which you start Python interpreter with) or modules (which you import from within Python); never both. Store most functionality in modules and keep startup scripts small. If you do this, the weird semantics of __main__ is a moot point. If you want to be able to run a module as a script while avoiding side effects due to it being named __main__, the easiest thing to do is to put something like the following boilerplate at the top of the module (this causes the module to rename itself). import sys if __name__ == '__main__': is_main = True # since you're overwriting __name__ you'll need this later __name__ = 'foo' sys.modules['foo'] = sys.modules['__main__'] else: is_main = False All of this gets a lot more complicated when packages are involved. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: include a file in a python program
On Mon, 06 Sep 2010 00:57:30 +0200, bussiere bussiere wrote: i've got a python.txt that contain python and it must stay as it (python.txt) Why? Is it against the law to change it? *wink* how can i include it in my program ? import python.txt doesn't work You could write a custom importer to handle it, but I can't help you with that. Try Google. is there a way : a) to make an include(python.txt) b) tell him to treat .txt as .py file that i can make an import python ? fp = open(python.txt) text = fp.read() fp.close() exec(text) But keep in mind that the contents of python.txt will be executed as if you had typed it yourself. If you don't trust the source with your life (or at least with the contents of your computer), don't execute it. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: include a file in a python program
In article mailman.476.1283727475.29448.python-l...@python.org, bussiere bussiere bussi...@gmail.com wrote: i've got a python.txt that contain python and it must stay as it (python.txt) how can i include it in my program ? import python.txt doesn't work is there a way : a) to make an include(python.txt) b) tell him to treat .txt as .py file that i can make an import python ? i'am using python3 The simple solution (at least for unix-ish systems) would be to make a symlink python.py - python.txt. If you don't have the ability to do that externally, you could even have your python program create the symlink on the fly, import the module, then delete the symlink. The symlink could even be in /tmp. Another possible solution is to read the entire file into a string and then eval() the string. I'm assuming eval() still exists in python 3; does it? Yet another possibility (never tried this, but it seems reasonable) would be to compile() your text file (http://docs.python.org/py3k/library/functions.html#compile). Of course, it would be a lot easier if you just renamed the file, but I'll take it as a given that there are external forces which prevent you from doing that. -- http://mail.python.org/mailman/listinfo/python-list
Re: include a file in a python program
In article 4c8423d3$0$28657$c3e8...@news.astraweb.com, Steven D'Aprano st...@remove-this-cybersource.com.au wrote: fp = open(python.txt) text = fp.read() fp.close() exec(text) But keep in mind that the contents of python.txt will be executed as if you had typed it yourself. If you don't trust the source with your life (or at least with the contents of your computer), don't execute it. Well, this is true, but eval() or exec() isn't really exposing him to anything that import isn't. -- http://mail.python.org/mailman/listinfo/python-list
Re: include a file in a python program
On 05/09/2010 23:57, bussiere bussiere wrote: i've got a python.txt that contain python and it must stay as it (python.txt) how can i include it in my program ? import python.txt doesn't work is there a way : a) to make an include(python.txt) b) tell him to treat .txt as .py file that i can make an import python ? i'am using python3 import imp python = imp.load_source(python, python.txt) Regards Bussiere fan of torchwood Google Fan boy A fan of Torchwood and Google, but not Python? :-) -- http://mail.python.org/mailman/listinfo/python-list
Re: include a file in a python program
On 9/5/2010 6:57 PM, bussiere bussiere wrote: i've got a python.txt that contain python and it must stay as it (python.txt) If you are working for someone who is such an idiot as to impose such a condition on you, you have our condolences. how can i include it in my program ? import python.txt doesn't work A file run directly as a script can be named anything. But as far as I know, imported code must be xxx.py, either in a directory or zipfile. is there a way : a) to make an include(python.txt) b) tell him to treat .txt as .py file that i can make an import python ? 1. Write a custom importer (hard, I have no idea how). 2. Copy the .txt file to .py, either before running the main program or within a custom import function. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: include a file in a python program
On 9/5/2010 7:12 PM, Steven D'Aprano wrote: fp = open(python.txt) text = fp.read() fp.close() exec(text) But keep in mind that the contents of python.txt will be executed as if you had typed it yourself. If you don't trust the source with your life (or at least with the contents of your computer), don't execute it. Also, this executes the code within the namespace of the calling code rather than within a fresh module namespace. That could be either better or worse. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: include a file in a python program
On Mon, 06 Sep 2010 00:27:13 +0100, MRAB wrote: import imp python = imp.load_source(python, python.txt) Nice! -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Class changes in circular imports when __name__ == '__main__'
On 2:59 PM, Carl Banks wrote: On Sep 5, 1:19 pm, Spencer Pearsonspeeze.pear...@gmail.com wrote: Hi! I'm writing a package with several files in it, and I've found that isinstance doesn't work the way I expect under certain circumstances. Short example: here are two files. # fileone.py import filetwo class AClass( object ): pass if __name__ ='__main__': a =Class() filetwo.is_aclass( a ) # filetwo.py import fileone def is_aclass( a ): print The argument is, ( if isinstance(a, fileone.AClass) else not), an instance of fileone.AClass If you run fileone.py, it will tell you that The argument is not an instance of fileone.AClass, which seems strange to me, given that the fileone module is the one that CREATES the object with its own AClass class. And if you replace if __name__ ='__main__' with def main(), start Python, import fileone, and call fileone.main(), it tells you that the argument IS an instance of AClass. So, the module's name change to __main__ when you run it on its own... well, it looks like it puts all of the things defined in fileone in the __main__ namespace INSTEAD of in the fileone module's namespace, and then when filetwo imports fileone, the class is created again, this time as fileone.AClass, and though it's identical in function to __main__.AClass, one is not the other. Correct. Python always treats the main script as a module called __main__. If you then try to import the main script file from another module, Python will actually import it again with whatever its usual name is. This is easily one of the most confusing and unfortunate aspects of Python. Is this kind of doubled-back 'isinstance' inherently sinful? I mean, I could solve this problem by giving all of my classes classname attributes or something, but maybe it's just a sign that I shouldn't have to do this in the first place. Even if there are better ways than isinstance, the weird behavior of __main__ shouldn't be the reason not to use it. My recommendation for most programmers is to treat Python files either as scripts (which you start Python interpreter with) or modules (which you import from within Python); never both. Store most functionality in modules and keep startup scripts small. If you do this, the weird semantics of __main__ is a moot point. If you want to be able to run a module as a script while avoiding side effects due to it being named __main__, the easiest thing to do is to put something like the following boilerplate at the top of the module (this causes the module to rename itself). import sys if __name__ ='__main__': is_main =rue # since you're overwriting __name__ you'll need this later __name__ =foo' sys.modules['foo'] =ys.modules['__main__'] else: is_main =alse All of this gets a lot more complicated when packages are involved. Carl Banks Perhaps a better answer would be to import __main__ from the second module. But to my way of thinking, the answer should be to avoid ever having circular imports. This is just the most blatant of the problems that circular imports can cause. I don't know of any cases where circular dependencies are really necessary, but if one decides to use them, then two things should be done: 1) do almost nothing in top-level code in any module involved in such circular dependency. Top-level should have all of the imports, and none of the executable code. 2) do not ever involve the startup script in the loop. If necessary, make it two lines, importing,then calling the real mainline. DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: Subclassing by monkey-patching
On Sep 5, 6:56 pm, Peter Otten __pete...@web.de wrote: Does it have to be gio.FileMonitor? pyinotify can automatically add new subdirectories out of the box. Well, since it's for a core part of the software, I'd like it to be cross platform — not in the sense of Windows/Mac, but FreeBSD, Solaris, etc. But it's looking more and more like I should give up that particular goal. Cheers, Jason -- http://mail.python.org/mailman/listinfo/python-list
Re: Question about Reading Files
On Sep 5, 4:16 am, Peter Otten __pete...@web.de wrote: genxtech wrote: Hello. I am still really new to python and I have a project where I am trying to use the data files from another program and write a new program with new user interface and all. My first step was to open one of the files in 'rb' mode and print the contents, but I am unfamiliar with the format. Here is what was printed to the terminal: b'URES\x04\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0f \x00\x00\x00\x03\t\x00c\x01\x00\x00\x0c#\x00\x00\x02\x1b\x00\x00\x00Y \x00\x00\x00\x08\x98\x00\x00t\x00\x00\x00\x01\'\x01\x00\x00z$ \x00\x00\x04,\xa7\x00\x00\xa1%\x00\x00\x05\x0b\x00\x00\x00o$\x00\x00\n \x11\x00\x00\x00\xcd\xcc\x00\x00\x0b\xf8\x00\x00\x00\xde\xcc \x00\x00\x0c\x19\x00\x00' I am using Python 3.1 on a Fedora 13 box if that makes any difference. Any advise on how to decode the data would be greatly appreciated. What's the name of the file? What's the program that uses the file? If the source code is available, what library does that program use to read the file? Peter Unfortunately the source code isn't available, so I'm trying to figure out a way to get the format of the file on my own. -- http://mail.python.org/mailman/listinfo/python-list
Re: Subclassing by monkey-patching
On Sep 6, 8:57 am, Jason jason.hee...@gmail.com wrote: But it's looking more and more like I should give up that particular goal. ...but on the other hand I just knocked together a pyinotify threaded watch system in about 50 lines. It's tempting to tell users of other platforms to write their own and submit a patch. Maybe this is all barking up the wrong tree. Cheers, Jason -- http://mail.python.org/mailman/listinfo/python-list
Re: Class changes in circular imports when __name__ == '__main__'
On Sep 5, 5:07 pm, Dave Angel da...@ieee.org wrote: On 2:59 PM, Carl Banks wrote: All of this gets a lot more complicated when packages are involved. Perhaps a better answer would be to import __main__ from the second module. Then what if the module is imported from a different script? It'll try to import __main__ but get a different script than expected. But to my way of thinking, the answer should be to avoid ever having circular imports. This is just the most blatant of the problems that circular imports can cause. I don't know of any cases where circular dependencies are really necessary, but if one decides to use them, then two things should be done: I don't think they're ever necessary but sometimes it's convenient. This could be one of those cases. One of the less misguided reasons to invoke a module as a script is to run tests on the module. When you do that you might need to call an outside module to set up a test environment, and that module might happen to import the calling module. You could refactor the test to avoid the circular import, but that kind of defeats the convenience of stowing the test in the module. 1) do almost nothing in top-level code in any module involved in such circular dependency. Top-level should have all of the imports, and none of the executable code. 2) do not ever involve the startup script in the loop. If necessary, make it two lines, importing,then calling the real mainline. All good advice for that situation. I would add that if you define a base class in one module and subclass in another, you want to keep those modules out of cycles. The problem with circular imports is that you don't usually know what order the modules will be imported, but you need to be sure the base class is defined when you subclass. (I learned that lesson the hard way, and I had to hack up an import hook to enforce that imports occurred in the correct order.) Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
MAKE UPTO $5000 P/M $2000 IN FIRST 30 DAYS! NO INV
MAKE UPTO $5000 P/M $2000 IN FIRST 30 DAYS! NO INV Generate $50 to $100 whenever you have a couple of hours free time to spare. You could make $50 or more in the next 2 hours. Starting right Now!Today! GET PAID TO: Take online surveys and make from $5 to $75, or more Participate in focus groups and make up to $150 an hour Take phone surveys and you can earn as much as $120 an hour Try new products (and keep the free products too) Preview new movie trailers for $4 to $25 an hour http://snipurl.com/11i6lw Earn from your free website Awesome earnings get paid for your honest work Join as a free member and get paid to your bank account To join the Network follow the link http://snipurl.com/11i6lw Get paid for your real work and earn awesome -- http://mail.python.org/mailman/listinfo/python-list
[issue9778] Make hash values the same width as a pointer (or Py_ssize_t)
Martin v. Löwis mar...@v.loewis.de added the comment: This would also impact the ABI, I suppose. Correct. So it either needs to happen before 3.2, or wait until 4.0, or the introduction of wide hashes needs to be done in a compatible manner, likely requiring two parallel hashing infrastructures. -- title: Make hash values the same width as a pointer (or Py_ssize_t) - Make hash values the same width as a pointer (or Py_ssize_t) ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9778 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9767] Failures in json doc examples
Georg Brandl ge...@python.org added the comment: FWIW if something is fixed, it should be the json output. However, I can imagine (without looking at the code) that this would mean a lot of special casing. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9767 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9775] Multiprocessing, logging and atexit play not well together
Georg Brandl ge...@python.org added the comment: I'll sort this out with Armin. -- nosy: +georg.brandl ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9775 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9775] Multiprocessing, logging and atexit play not well together
Changes by Georg Brandl ge...@python.org: -- assignee: - georg.brandl priority: normal - high versions: +Python 3.2 -Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9775 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9662] ctypes not building under OS X because of ffi_closure_free not being defined early enough
Georg Brandl ge...@python.org added the comment: I don't think this should block 3.2a2. +1 for supporting LLVM in general. BTW, why doesn't it build because of warnings? -- nosy: +georg.brandl priority: release blocker - deferred blocker ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9662 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9779] argparse.ArgumentParser not support unicode in print help
New submission from gkraser gkra...@gmail.com: argparse.ArgumentParser not support unicode in print help. Example: # -*- coding: utf-8 -*- import argparse import unittest class Test1(unittest.TestCase): def test_unicode_desc(self): h = u'Rus Рус' # unicode print h # ok parser = argparse.ArgumentParser(description=h) parser.print_help() # error -- components: Library (Lib) files: test_ap.py messages: 115630 nosy: gkraser priority: normal severity: normal status: open title: argparse.ArgumentParser not support unicode in print help type: behavior versions: Python 2.7 Added file: http://bugs.python.org/file18757/test_ap.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9779 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9662] ctypes not building under OS X because of ffi_closure_free not being defined early enough
Ned Deily n...@acm.org added the comment: It's not just LLVM. Building a standard OS X installer on OS X 10.5 or 10.6 (gcc-4.0, 10.4u SDK, i386/ppc, deployment target=10.3), _ctypes fails: *** WARNING: renaming _ctypes since importing it failed: dlopen(build/lib.macosx-10.3-fat-3.2/_ctypes.so, 2): Symbol not found: _ffi_closure_alloc That is definitely a regression. -- nosy: +ned.deily ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9662 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9662] ctypes not building under OS X because of ffi_closure_free not being defined early enough
Ned Deily n...@acm.org added the comment: It's not just LLVM. Building a standard OS X installer on OS X 10.5 (gcc-4.0, 10.4u SDK, i386/ppc, deployment target=10.3) or 10.6 (gcc-4.2, 10.6 SDK, i386/x86_64, dept target 10.6), _ctypes fails: *** WARNING: renaming _ctypes since importing it failed: dlopen(build/lib.macosx-10.3-fat-3.2/_ctypes.so, 2): Symbol not found: _ffi_closure_alloc That is definitely a regression. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9662 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9662] ctypes not building under OS X because of ffi_closure_free not being defined early enough
Changes by Ned Deily n...@acm.org: -- Removed message: http://bugs.python.org/msg115631 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9662 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9662] ctypes not building under OS X because of ffi_closure_free not being defined early enough
Georg Brandl ge...@python.org added the comment: OK. Assigning to Ronald for now; he'll have to deal with it when building the binaries anyway. -- assignee: theller - ronaldoussoren nosy: +ronaldoussoren ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9662 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9437] can't build extensions with non-default ldflags (e.g. -m32)
Georg Brandl ge...@python.org added the comment: Patch looks like it at least can't make things worse to me. linker_exe should probably also get LDFLAGS (but not LDSHARED). -- nosy: +georg.brandl ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9437 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9116] test_capi.test_no_FatalError_infinite_loop crash on Windows
Georg Brandl ge...@python.org added the comment: Once again, this is not important enough block 3.2a2. -- priority: release blocker - deferred blocker ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9116 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8959] WINFUNCTYPE wrapped ctypes callbacks not functioning correctly in Python 2.7
Georg Brandl ge...@python.org added the comment: Deferring for 3.2a2. -- priority: release blocker - deferred blocker ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8959 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8611] Python3 doesn't support locale different than utf8 and an non-ASCII path (POSIX)
Georg Brandl ge...@python.org added the comment: The Unicode import system won't be put in place before 3.2a2, deferring. -- priority: release blocker - deferred blocker ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8611 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9437] can't build extensions with non-default ldflags (e.g. -m32)
Georg Brandl ge...@python.org added the comment: It is however not important enough to block 3.2a2. -- priority: release blocker - deferred blocker ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9437 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9778] Make hash values the same width as a pointer (or Py_ssize_t)
Antoine Pitrou pit...@free.fr added the comment: Correct. So it either needs to happen before 3.2, or wait until 4.0, Shouldn't there be a provision for ABI versioning? Or do you suggest bumping to the next major number (4.0, 5.0...) be done on the basis of ABI changes? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9778 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9437] can't build extensions with non-default ldflags (e.g. -m32)
Łukasz Langa luk...@langa.pl added the comment: This was broken in r83837 with a patch for issue #5504. -- nosy: +lukasz.langa ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9437 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9437] can't build extensions with non-default ldflags (e.g. -m32)
Changes by Łukasz Langa luk...@langa.pl: -- Removed message: http://bugs.python.org/msg115640 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9437 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9662] ctypes not building under OS X because of ffi_closure_free not being defined early enough
Łukasz Langa luk...@langa.pl added the comment: This was broken in r83837 with a patch for issue #5504. -- nosy: +lukasz.langa ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9662 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9437] can't build extensions with non-default ldflags (e.g. -m32)
Antoine Pitrou pit...@free.fr added the comment: Other patch, fixes all failures. -- Added file: http://bugs.python.org/file18758/m32-2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9437 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3978] ZipFileExt.read() can be incredibly slow; patch included
Antoine Pitrou pit...@free.fr added the comment: The patch has been outdated by other independent performance work on the zipfile module. In Python 3.2, the zipfile module is actually slightly faster than the unzip program: - first with the supplied zeroes.zip file: $ rm -f zeroes time -p unzip -e zeroes.zip Archive: zeroes.zip inflating: zeroes real 0.56 user 0.50 sys 0.06 $ time -p ./python -m zipfile -e zeroes.zip . real 0.45 user 0.34 sys 0.10 - Then with a 100MB random file: $ rm -f random time -p unzip -e random.zip Archive: random.zip inflating: random real 0.69 user 0.61 sys 0.07 $ rm -f random time -p ./python -m zipfile -e random.zip . real 0.33 user 0.18 sys 0.14 -- resolution: - out of date status: open - closed versions: +Python 3.2 -Python 2.7, Python 3.1 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3978 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4941] Tell GCC Py_DECREF is unlikely to call the destructor
Antoine Pitrou pit...@free.fr added the comment: The whole approach doesn't seem to bear much fruit. I tried to apply again likely_decref.diff and got a 0% performance change on 3.2 (on a Core i3 processor). -- resolution: - rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4941 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4194] default subprocess.Popen buffer size
Antoine Pitrou pit...@free.fr added the comment: The subprocess doc now has a note about buffering and performance issues, closing. -- resolution: - out of date status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4194 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1568] PATCH: Armin's attribute lookup caching for 3.0
Antoine Pitrou pit...@free.fr added the comment: Closing: Neil's algorithm is not different from what is already in 3.2, except for the additional type_attrcache_callback() which probably doesn't make a difference in normal workloads. -- resolution: accepted - out of date status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1568 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5504] ctypes should work with systems where mmap can't be PROT_WRITE and PROT_EXEC
Ronald Oussoren ronaldousso...@mac.com added the comment: Fixing the alloc_closure error is easy enough: Index: ../setup.py === --- ../setup.py (revision 84528) +++ ../setup.py (working copy) @@ -1653,7 +1653,9 @@ '_ctypes/callbacks.c', '_ctypes/callproc.c', '_ctypes/stgdict.c', - '_ctypes/cfield.c'] + '_ctypes/cfield.c', + '_ctypes/malloc_closure.c', + ] depends = ['_ctypes/ctypes.h'] if sys.platform == 'darwin': That's not enough to make it possible to build ctypes on OSX though, I now get the same error for a different symbol: *** WARNING: renaming _ctypes since importing it failed: dlopen(build/lib.macosx-10.6-fat-3.2/_ctypes.so, 2): Symbol not found: _ffi_prep_closure_loc Referenced from: /Users/ronald/Projects/python/python-3.x/build/build/lib.macosx-10.6-fat-3.2/_ctypes.so Expected in: flat namespace in /Users/ronald/Projects/python/python-3.x/build/build/lib.macosx-10.6-fat-3.2/_ctypes.so -- nosy: +ronaldoussoren ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5504 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5504] ctypes should work with systems where mmap can't be PROT_WRITE and PROT_EXEC
Åukasz Langa luk...@langa.pl added the comment:
Please find attached a fix for #9662. Some explanation is in order:
- the Windows, Linux and OS X implementations of FFI included in the SVN are
different in terms of maturity
- Thomas originally when fixing #5504 used a bit of functionality regarding
closures that wasn't there in the Windows and OS X versions
- he did some dirty hacking on the Windows version to make it work: changed the
ffi_prep_closure function header to ffi_prep_closure_loc (leaving the old
implementation) and introduced names for ffi_closure_alloc and ffi_closure_free
- he didn't touch the OS X version so it failed
I did more-less the same thing but in case of OS X we have many
ffi_prep_closure implementations for different archs so I simply did an #ifdef
in the place ffi_prep_closure is used.
I think a better solution ultimately would be to update all implementation to
FFI 3.0.9 and be done with it. Still, the current patch works for me.
--
Added file: http://bugs.python.org/file18759/issue9662.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5504
___Index: setup.py
===
--- setup.py(revision 84528)
+++ setup.py(working copy)
@@ -1657,6 +1657,7 @@
depends = ['_ctypes/ctypes.h']
if sys.platform == 'darwin':
+sources.append('_ctypes/malloc_closure.c')
sources.append('_ctypes/darwin/dlfcn_simple.c')
extra_compile_args.append('-DMACOSX')
include_dirs.append('_ctypes/darwin')
Index: Modules/_ctypes/libffi_osx/include/ffi.h
===
--- Modules/_ctypes/libffi_osx/include/ffi.h(revision 84528)
+++ Modules/_ctypes/libffi_osx/include/ffi.h(working copy)
@@ -264,6 +264,9 @@
void(*fun)(ffi_cif*,void*,void**,void*),
void* user_data);
+void ffi_closure_free(void *);
+void *ffi_closure_alloc (size_t size, void **code);
+
typedef struct ffi_raw_closure {
chartramp[FFI_TRAMPOLINE_SIZE];
ffi_cif*cif;
@@ -349,4 +352,4 @@
}
#endif
-#endif // #ifndef LIBFFI_H
\ No newline at end of file
+#endif // #ifndef LIBFFI_H
Index: Modules/_ctypes/malloc_closure.c
===
--- Modules/_ctypes/malloc_closure.c(revision 84528)
+++ Modules/_ctypes/malloc_closure.c(working copy)
@@ -106,7 +106,6 @@
return NULL;
item = free_list;
free_list = item-next;
- *codeloc = (void *)item;
+*codeloc = (void *)item;
return (void *)item;
}
-
Index: Modules/_ctypes/callbacks.c
===
--- Modules/_ctypes/callbacks.c (revision 84528)
+++ Modules/_ctypes/callbacks.c (working copy)
@@ -416,9 +416,13 @@
ffi_prep_cif failed with %d, result);
goto error;
}
+#if defined(X86_DARWIN) || defined(POWERPC_DARWIN)
+result = ffi_prep_closure(p-pcl_write, p-cif, closure_fcn, p);
+#else
result = ffi_prep_closure_loc(p-pcl_write, p-cif, closure_fcn,
p,
p-pcl_exec);
+#endif
if (result != FFI_OK) {
PyErr_Format(PyExc_RuntimeError,
ffi_prep_closure failed with %d, result);
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9662] ctypes not building under OS X because of ffi_closure_free not being defined early enough
Ronald Oussoren ronaldousso...@mac.com added the comment: I'm looking into this issue. One problem is that the version of libffi used on OSX is no longer in sync with the version that is used on other platforms. The version that is used on OSX does not have some of the symbols expected by ctypes, such as 'ffi_prep_closure_loc'. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9662 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9662] ctypes not building under OS X because of ffi_closure_free not being defined early enough
Łukasz Langa luk...@langa.pl added the comment: Ronald, please see #5504. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9662 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1677872] Efficient reverse line iterator
Antoine Pitrou pit...@free.fr added the comment: Suggestions: - do it on BufferedReader, rather than TextIOWrapper: if you want full-speed scanning of log files, you probably want to open them in binary mode - rather than implementing a full-blown iterator, you can start with simple primitives: e.g. a method called readprevline() (or even scan_back() if you want to make the stop character(s) configurable) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1677872 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5504] ctypes should work with systems where mmap can't be PROT_WRITE and PROT_EXEC
Ronald Oussoren ronaldousso...@mac.com added the comment: Łukasz' patch fixes the issue for me. As the patch only affects code-paths used on OSX (patches to the libffi version for OSX and an #ifdef that makes OSX use ffi_prep_closure instead of ffi_prep_closure_loc) I intend to commit the patch later today to be able to build a working copy of the OSX installer for the next alpha release. I'm at best -0 on moving all platforms to libffi 3.0.9. 'libffi_osx' works properly on OSX and I have no idea whether all changes in that fork of libffi have been merged back into the canonical version. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5504 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9662] ctypes not building under OS X because of ffi_closure_free not being defined early enough
Ronald Oussoren ronaldousso...@mac.com added the comment: Thanks! That patch fixes the issue for me and I will commit it later today. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9662 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1677872] Efficient reverse line iterator
Changes by Guido van Rossum gu...@python.org: -- nosy: -gvanrossum ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1677872 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
