[ANN] PyInstaller 1.4
Hello, I'm happy to announce PyInstaller 1.4, the first formal release after several years of quiet development. http://www.pyinstaller.org === What it is === PyInstaller is a program that converts (packages) Python programs into stand-alone executables, under Windows, Linux, and Mac OS X. It's similar to py2exe/py2app, but it is multiplatform and with many advanced features. The main goal of PyInstaller is to be compatible with 3rd-party packages out-of-the-box. This means that, with PyInstaller, all the required tricks to make external packages work are already integrated within PyInstaller itself so that there is no user intervention required. You'll never be required to look for tricks in wikis and apply custom modification to your files or your setup scripts. === Features === * Packaging of Python programs into standard executables, that work on computers without Python installed. * Multiplatform: works under Windows, Linux and Irix. Experimental Mac OS X support available. * Multiversion: works under any version of Python from 1.5 up to 2.6. NOTE: Support for Python 2.6+ on Windows is not included in this release yet. Please see this page for a working patch: http://www.pyinstaller.org/wiki/Python26Win * Flexible packaging mode: * Single directory: build a directory containing an executable plus all the external binary modules (.dll, .pyd, .so) used by the program. * Single file: build a single executable file, totally self-contained, which runs without any external dependency. * Custom: you can automate PyInstaller to do whatever packaging mode you want through a simple script file in Python. * Explicit intelligent support for many 3rd-packages (for hidden imports, external data files, etc.), to make them work with PyInstaller out-of-the-box. * Full single-file EGG support: .egg files are automatically packaged by PyInstaller as-is, so that all features are supported at runtime as well (entry points, etc.). * Automatic support for binary libraries used through ctypes (see http://www.pyinstaller.org/wiki/CtypesDependencySupport for details). * Support for automatic binary packing through the well-known UPX compressor. * Support for code-signing executables on Windows. * Optional console mode (see standard output and standard error at runtime). * Selectable executable icon (Windows only). * Fully configurable version resource section in executable (Windows only). * Support for building COM servers (Windows only). === ChangeLog === For those already using older versions of PyInstaller, the full changelog for this release can be found here: http://www.pyinstaller.org/browser/tags/1.4/doc/CHANGES.txt === Feedback === We're eager to listent to your feedback on using PyInstaller: Ticketing system: http://www.pyinstaller.org/newticket Mailing list: http://groups-beta.google.com/group/PyInstaller -- Giovanni Bajo :: Develer S.r.l. ra...@develer.com :: http://www.develer.com Blog: http://giovanni.bajo.it Last post: ctypes support in PyInstaller -- http://mail.python.org/mailman/listinfo/python-list
Re: Counter Class -- Bag/Multiset
On 1/23/2009 2:49 AM, Chris Rebert wrote: On Thu, Jan 22, 2009 at 5:41 PM, Giovanni Bajo wrote: On Thu, 22 Jan 2009 10:11:37 -0800, Raymond Hettinger wrote: The collections module in Python 2.7 and Python 3.1 has gotten a new Counter class that works like bags and multisets in other languages. I've adapted it for Python2.5/2.6 so people can start using it right away: http://docs.python.org/dev/library/collections.html#counter-objects Here's a link to the docs for the new class: http://code.activestate.com/recipes/576611/ Hi Raymond, * I'm not a native speaker, but why use the word "Counter"? A "counter" to my ear sounds like a number that is increased each time an event occurs; the website counter, eg, comes to mind. I can understanda its meaning probably stretches to "an object that counts", but I really can't think of it as a group of object, or a container of object. Moreover, I find it a much more useful abstraction the idea of a "multi-set" (that is, a set where elements can appear with multiple cardinality), rather than stressing the concept of "counting" how many times each element appears in the set. * I find it *very* confusing c.items() vs c.elements(). Items and elements are synonymous (again, in my understanding of English). I concur and would like to say additionally that having Counter's len() be the number of *unique* items as opposed to just the number of items seems a bit counterintuitive. In fact, I think that it makes sense when you're stressing the fact that it's just a dictionary of objects and their counters (which the name "Counter" in fact stresses). And my main objection is exactly this: a multiset is a differnt abstract data structure (which is probably worth its own ABC in Python); the fact that can be implemented through a mapping of objects and counters is just its concrete implementation. This was perfectly represented by Raymond's previous bag class, that was *using* a dictionary instead of *being* a dictionary. -- Giovanni Bajo Develer S.r.l. http://www.develer.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Counter Class -- Bag/Multiset
On Thu, 22 Jan 2009 10:11:37 -0800, Raymond Hettinger wrote: > The collections module in Python 2.7 and Python 3.1 has gotten a new > Counter class that works like bags and multisets in other languages. > > I've adapted it for Python2.5/2.6 so people can start using it right > away: > http://docs.python.org/dev/library/collections.html#counter-objects > > Here's a link to the docs for the new class: > http://code.activestate.com/recipes/576611/ Hi Raymond, * I'm not a native speaker, but why use the word "Counter"? A "counter" to my ear sounds like a number that is increased each time an event occurs; the website counter, eg, comes to mind. I can understanda its meaning probably stretches to "an object that counts", but I really can't think of it as a group of object, or a container of object. Moreover, I find it a much more useful abstraction the idea of a "multi-set" (that is, a set where elements can appear with multiple cardinality), rather than stressing the concept of "counting" how many times each element appears in the set. * I find it *very* confusing c.items() vs c.elements(). Items and elements are synonymous (again, in my understanding of English). All in all, I think I prefer your previous bag class: http://code.activestate.com/recipes/259174/ -- Giovanni Bajo Develer S.r.l. http://www.develer.com -- http://mail.python.org/mailman/listinfo/python-list
Re: PyInstaller: problem to build exe with PyQt4
On Tue, 03 Jun 2008 08:35:37 +0200, Mark Delon wrote: > Hi, > > I need to generate single EXEcutable via PyInstaller. It will be > genereated -> i get one single executable. > > AFTER CALL (exe) I get an error: "no module named _gt" > > Build command: > 1. Configure.py > 2. Makespec.py -F
min/max: "stable" invariant?
Hello, assuming that a sequence contains multiple elements with minimum/maximum value, do min/max guarantee to return the *first* element in the sequence among those? Example: >>> class A: ... def __init__(self, x): self.x = x ... >>> L = [A(0), A(1), A(2), A(0)] >>> min(L, key=lambda a:a.x) is L[0] True >>> min(L, key=lambda a:a.x) is L[3] False Is this guaranteed by the Python language formal specification? Is this guaranteed to always by true in CPython? (I guess so) I can't find any mention in the documentation. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
strftime return value encoding (mbcs, locale, etc.)
Hello,
I am trying to find a good way to portably get the output of strftime()
and put it onto a dialog (I'm using PyQt, but it doesn't really matter).
The problem is that I need to decode the byte stream returned by strftime
() into Unicode.
This old bug:
http://mail.python.org/pipermail/python-bugs-list/2003-
November/020983.html
(last comment) mentions that it is "byte string in the locale's encoding".
The comment also suggests to use "mbcs" in Windows (which, AFAIK, it's
sort of an "alias" encoding for the current Windows codepage), and to
find out the exact encoding using locale.getpreferredencoding().
Thus, I was hoping that something like:
strftime("%#c", localtime()).decode(locale.getpreferredencoding())
would work... but alas I was reported this exception:
LookupError: unknown encoding: cp932
So: what is the correct code to achieve this? Will something like this
work:
data = strftime("%#c", localtime())
if os.name == "nt":
data = data.decode("mbcs")
else:
data = dada.decode(locale.getpreferredencoding())
Is this the correct way of doing it? (Yes, it sucks).
Shouldn't Python automatically alias whatever is returned by
locale.getpreferredencoding() to "mbcs", so that my original code works
portably?
Thanks in advance!
--
Giovanni Bajo
--
http://mail.python.org/mailman/listinfo/python-list
Re: Does Boost.Python participate in cyclic gc?
On 05/06/2007 21.32, [EMAIL PROTECTED] wrote: > We encountered a situation today where it appeared that a > Boost.Python-provided class didn't participate in Python's cyclic garbage > collection. The wrapped C++ instance held a reference to a method in the > Python object which referenced the Boostified C++ instance, e.g.: Those situations get more and more common as you keep wrapping your code. To the best of my knowledge, SIP is the only one which does the right thing here. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: How to receive a FILE* from Python under MinGW?
On 22/03/2007 15.34, John Pye wrote: > I downloaded your package and installed it in c:/mingw1. It complained > that it could not detect Python, although I have Python 2.4 installed > on my system (did you check HKCU as well as HKLM, perhaps?) E you're totally right! I'll have that fixed! > I note that the gccmrt utility does not work from MSYS. You will need > to provide a shell-script equivalent version, in order for that to be > useful for MSYS users. So I opened a cmd prompt and ran the command, > then restarted my MSYS session. Yes, I don't use MSYS. I'll attempt something... any idea how to ship both versions and still having the right one picked up depending on which shell you're using? Maybe it's just a matter of extensions? > There is also a need to be able to > query the *current state* of the gccmrt option. Right, I'll add that. > Next I built my code. It all compiled OK, all the way through to my > NSIS bundle. So that was nice. It includes gfortran, flex, bison, SWIG/ > Python and Tcl/Tk linkage: a bit of a coup. > > BUT when I try to run my program, I get a windows error msgbox, > "python.exe - Entry Point Not Found: The procedure entry point _ctype > could not be located in the dynamic link library msvcr71.dll". In msvcrt71.dll, there is no _ctype (see declaration at line 111 of include\ctype.h). There is only _pctype. What if you comment the declaration at line 111? I wonder what it's bringing in _ctype. Anyway, it looks like mingw-runtime does not support this _ctype change between mscvrt.dll and msvcr71.dll. These changes should probably be represented in the forms of #ifs checking __MSVCRT_VERSION__. I'm not even sure that __MSVCRT_VERSION__ is being set correctly.. that is something gccmrt should eventually take care of, but for this specific _ctype problem you probably need to submit a patch to mingw-runtime. I can guide you through it though... I don't have much time in these days though. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: How to receive a FILE* from Python under MinGW?
On 22/03/2007 3.13, John Pye wrote: >> I suggest people to try my GCC 4.1.2 binary installer for Windows which fully >> integrates with Python and has scripts in place to solve the MSVCR71.DLL >> problem. It was announced on this very list a few days ago: >> >> http://www.develer.com/oss/GccWinBinaries > > Hi Giovanni, > > Thanks very much for that suggestion. It looks like you have done some > good work on addressing this problem. > > I wonder if you might be able to expand on what those 'serious > problems' with GCC 4.x on the Windows platform are? Have a look here: http://www.mingw.org/MinGWiki/index.php/GccStatus > Also, can you comment on how one might use your compiler bundle with > MSYS? In particular, I like to run SCons from an MSYS prompt. Assuming > I don't want to kill my old MinGW and Cygwin installations, how do you > recommend that one proceeds to set up a working GCC 4 environment on > Windows? (I haven't downloaded it just yet; I'm on Linux today!) I personally don't use MSYS so I don't know exactly. I use SCons too, and I simply run it from the normal command prompt. I *believe* it's sufficient to unpack MSYS somewhere (you can either unpack it *over* the directory where you installed my GCC package, or somewhere else, I believe it works either way); since the GCC binaries are added to the system PATH by the installer, you should able to run it anyway. You can try by simply invoking "gcc -v" at the MSYS prompt and see if it's picking up the right version of GCC. I would appreciate if you give some feedback about this. I would like to incorporate your findings on the webpage. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: [Swig-user] How to receive a FILE* from Python under MinGW?
On 22/03/2007 3.46, Carl Douglas wrote: > Hi Giovanni, > > I was wondering if gccmrt can be used with older 3.x versions of GCC? > > Thanks I think so, but I have not tried it. You just need to copy gccmrt to the bin directory, and then copy "libmsvcrt.a" into "libmsvcr60.a", and "libmsvcrtd.a" into "libmsvcr60d.a". After that, it should just works. Let me know if it works for you, so that I can report results on the webpage. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: [Swig-user] How to receive a FILE* from Python under MinGW?
On 21/03/2007 6.49, Carl Douglas wrote: > Hi John, > > In my case, all I needed was a PyFile_AsFile to get the File *. > However I found that Python errors going to stdout/stderr were not > being redirected properly when my code was linking to a different > version of the CRT than the build of Python I was using. > > In the end, I came up with one of two solutions, and I really don't > like recommending either: I suggest people to try my GCC 4.1.2 binary installer for Windows which fully integrates with Python and has scripts in place to solve the MSVCR71.DLL problem. It was announced on this very list a few days ago: http://www.develer.com/oss/GccWinBinaries Feedback is welcome! -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: GCC 4.1.2 installer for Python distutils compilation
On 18/03/2007 13.24, David Rushby wrote: > Even though I have access to MSVC 7.1, so I don't really need MinGW > myself, [...] But remember that GCC 4.1.2 is almost 4 years newer than MSVC 7.1, and I found it to produce more optimized code (especially for C++). Since it's a free alternative, it might be worth to give it a go :) -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
[ANN] GCC 4.1.2 installer for Python distutils compilation
Hello, This page: http://www.develer.com/oss/GccWinBinaries contains a friendly Windows installer for GCC 4.1.2 (MinGW binary version), with full support for integrating it with Python installations so that it is used by distutils to compile Python extensions. Direct download link: http://www.develer.com/~rasky/gcc-4.1.2-mingw-setup.exe Who needs this package? * People who wants to use FLOSS tools to develop Python extensions. * People who wants to use the recent GCC 4.1.2 to develop Python extensions, given that it easily outperforms the 4-years-old Visual Studio .NET 2003. What's special about this? * mingw.org still has GCC 3.4.2, so go figure. Also, you need to compose other packages together. This is a single installer with everything inside. * By default, MinGW GCC links with MSVCRT.DLL, and not MSVCR71.DLL (used by Python 2.4 and 2.5). Fixing this is pretty complicate, and there's much confusion (Google turns up red herrings). This package handles everything for you, and it just works. -- Giovanni Bajo Develer S.r.l. http://www.develer.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Fast constant functions for Py2.5's defaultdict()
On 13/02/2007 20.01, Raymond Hettinger wrote: > FWIW, here are three ways of writing constant functions for > collections.defaultdict(): > > d = defaultdict(int) # slowest way; works only for zero > d = defaultdict(lambda: 0) # faster way; works for any constant > d = defaultdict(itertools.repeat(0).next)# fastest way; works > for any constant > > Another approach is to use the __missing__ method for dictionary > subclasses: > > class zerodict (dict): > def __missing__ (self, key): > return 0 # fast on initial miss, but slow on > non-misses; works for any constant > > The itertools.repeat(const).next approach wins on speed and > flexibility. But it's the most unreadable too. I'm surprised that defaultdict(int) is slower than the lambda one though. What's the reason? -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Missing .dlls when installing PyQt!? :(
On 24/01/2007 14.36, [EMAIL PROTECTED] wrote: > I must be doing something wrong here because I'm getting all these > missing .dll messages after installing PyQT. This is what I've done (no > step excluded): > > 1. Fresh installation of Python25 > 2. I run the PyQt-gpl-4.1.1-Py2.5-Qt4.2.2.exe file which should include > SIP > 3. I run the qt-win-opensource-4.2.2-mingw.exe file > 4. I try to launch the C:\Program > Files\PyQt4\examples\tools\qtdemo\qtdemo.pyw file > > Result: > Error message says: Can't find mingwm10.dll > > After searching for the file I found it in the C:\Qt\4.2.2\bin > directory. I copied the dll to the windows dir just to run in to the > next missing dll error (QtCore4.dll) which is also located in the same > directory. What have I missed? I've been trying to install QT for 6 > hours now... Add C:\Qt\4.2.2\bin to your PATH. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: My python programs need a GUI, wxPython or PyQt4?
On 24/01/2007 1.45, Joshua J. Kugler wrote: >> I've downloaded both the wxPython and the PyQt4 package, and by the >> first impression I must say that the PyQt4 system had a very >> compelling presentation. From what I can understand from the feedback >> I've gotten so far is that the wxPython is a better choice when it >> comes to compability (with linux), and it's free even if I want to >> create applications and sell them. >> So, from what I understand I will have to go with PyQt4 since (from >> my understanding): >> 1. I will not sell the applications I'm working with since they will >> only be used by the internal QA at a computer game company. > > Even that is getting on shaky ground, at least according to Troll Tech. > See: http://www.trolltech.com/developer/knowledgebase/190/ So, write it > for internal use, and put up for distribution on your personal web site > (pending company approval, of course). That page is legal babble, trying to trick you into buying (or making your boss buy) a commercial license. The Qt Open Source edition *IS* GPL and thus it falls under all the normal GPL clauses and uses, irrespective of what Trolltech may or may not think. For instance, see this FAQ: http://www.gnu.org/licenses/gpl-faq.html#GPLRequireSourcePostedPublic which makes pretty clear that a "company"/"organization" is basically the same of an "individual". "Releasing a software within a company for internal usage" is by no means the same of "releasing it to the public". Basically, for what the GPL is concerned, it is *not* a "release" or a "distribution" at all. Thus, it is well possible to write internal GPL software, using the Qt Open Source library, and to release/distribute/use it *ONLY* internally. Also, it should be made clear that a company can of course relicense its own proprietary libraries as GPL for internal usage only, so that they can be linked to other GPL libraries "legally". For instance, in the case of Daniel and his game company, they can link their proprietary game code to PyQt4 (Open source edition) to produce an internal tool, like a level editor, which would then fall under the GPL. If and only if they were to release that tool to the public (either for free or for sale), they would have to either go GPL and release full source code (include the parts of their game code linked to it), or buy a Qt commercial license. [[ Another page of similar babble is the one where they try to convince you that you cannot use the Qt Open Source edition to develop a software, and then buy the commercial edition only the day before you want to release it as non open-source. I can't even understand how they can even try to support such a nonsense position. You're free to develop your software for years as GPL without distributing it, and then relicense your own code whenever you want. They're obviously just trying to scare people. ]] -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: PQueue and Python 2.5
On 20/01/2007 11.29, Martin v. Löwis wrote: >> I haven't got a clue how to investigate this, but I would be willing to >> help if someone has any ideas. > > There are a number of problems in this code; the most significant one is > the incorrect usage of memory management API. In pqueue_dealloc, the > call PyMem_DEL(pqp) should read PyObject_DEL(pqp), as the object was > allocated with PyObject_NEW. Because this was a common error once, > Python up to 2.4 detected that error and silently corrected it (at some > runtime cost); Python 2.5 now has removed this work-around. And by the way, this is also a FAQ: http://effbot.org/pyfaq/why-does-my-c-extension-suddenly-crash-under-2.5.htm and even explained in the Misc/NEWS of Python 2.5. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: [ANN] PyInstaller 1.3 released
Robin Becker wrote:
> OK I found the problem. First off I still see this message about
> msvcr71.dll could not be extracted!
>
> in the debug output.
>
> Secondly I found that the icon I specified in the Makespec.py invocation
> is actually being added to the distribution exe. I actually want to put
> the icon onto the running Tk app.
>
> We're using this code to do that
>
> self.tk.call('wm','iconbitmap', self._w, '-default', 'rptlabw.ico')
>
> ie it's assumed that the icon has been extracted into the runtime folder.
>
> I guess I either need to put the icon in as a resource or figure out
> some way to find the original exe and extract the icon from it.
If you specify it as icon (--icon to Makespec), it means you want it as icon
of the executable.
If you just want to add it as an additional data file, read the manual about
that. It will be extracted to the runtime directory and you should be able to
access it through _MEIPASS2 (manual should be clear about that). You can't
just read it from the "current directory" because the current directory is
where your exectuable is run (which could even be off a CDROM...).
--
Giovanni Bajo
--
http://mail.python.org/mailman/listinfo/python-list
Re: [ANN] PyInstaller 1.3 released
Robin Becker wrote: > I just tried this on something which we currently use py2exe+nsis to > package and it certainly seems to produce a small exe, but when run I > see a cryptic message on two message boxes > > ! msvcr71.dll > > ! could not be extracted > > the program then exits. yeah that's pretty cryptic. It's a known bug which I need to come around and fix it. Anyway, the message itself is harmless: if the program then exits, there is *another* problem. If you want to help with that, you could enable the console+debug mode and see what the traceback is. I'd be very glad to fix the problem for you. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
[ANN] PyInstaller 1.3 released
Hello, PyInstaller 1.3 is out! Grab latest version at: http://pyinstaller.python-hosting.com/ Description --- PyInstaller is a program that converts (packages) Python programs into stand-alone executables, under Windows, Linux and Irix. Its main advantages over similar tools are that PyInstaller works with any version of Python since 1.5, it builds smaller executables thanks to transparent compression, it is multi-platform (so you can build one-file binaries also under Linux), and use the OS support to load the dynamic libraries, thus ensuring full compatibility. Features * Packaging of Python programs into standard executables, that work on computers without Python installed. * Multiplatform: works under Windows, Linux and Irix. (Mac port in development. See /branches/mac on SVN) * Multiversion: works under any version of Python since 1.5. * Dual packaging mode: o Single directory: build a directory containing an executable plus all the external binary modules (.dll, .pyd, .so) used by the program. o Single file: build a single executable file, totally self-contained, which runs without any external dependency. * Support for automatic binary packing through the well-known UPX compressor. * Optional console mode (see standard output and standard error at runtime). * Selectable executable icon (Windows only). * Fully configurable version resource section in executable (Windows only). * Support for building COM servers (Windows only). ChangeLog - + Fix bug with user-provided icons disappearing from built executables when these were compressed with UPX. + Fix problems with packaging of applications using PIL (that was broken because of a bug in Python's import machinery, in recent Python versions). Also add a workaround including Tcl/Tk with PIL unless ImageTk is imported. + (Windows) When used under Windows XP, packaged programs now have the correct look & feel and follow user's themes (thanks to the manifest file being linked within the generated executable). This is especially useful for applications using wxPython. + Fix a buffer overrun in the bootloader (which could lead to a crash) when the built executable is run from within a deep directory (more than 70-80 characters in the pathname). * Bootstrap modules are now compressed in the executable (so that they are not visible in plaintext by just looking at it with a hex editor). * Fixed a regression introduced in 1.1: under Linux, the bootloader does not depend on libpythonX.X.so anymore. We've moved --- PyInstaller has a new home: http://pyinstaller.python-hosting.com/ (thanks to the guys at webfaction.com for top-notch free hosting!) We also own a domain name (http://pyinstaller.org) which can be used as a permanent redirector to our home. The mailing list moved as well, it's now at http://groups.google.com/group/PyInstaller. Join us for discussion! -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Designing a cancellable function
Gabriel Genellina wrote:
>> I now would also like to add the possibility to allow the user to
>> *cancel* the execution of foo() during the processing, and I am
>> wondering what the best / most Pythonic way to design this is.
>
> I can't say if this is the "best/more Pythonic way", but a simple way
> would be to use the return value from your callback. Consider it an
> "abort" function: if it returns True, cancel execution; as long as it
> returns False, keep going.
It's even easier if the callback function simply raise an exception, which can
be caught from the outside:
class AbortOperationError(RuntimeError):
pass
def callback(N):
updateProgressBar(N)
processGUIEvents()
if exit_button_pressed:
raise AbortOperationError()
try:
longFunction(callback)
except AbortOperationError()
pass
def longFunction(callback):
for i in xrange(10):
# do something
callback(i / 10.0)
--
Giovanni Bajo
--
http://mail.python.org/mailman/listinfo/python-list
Re: Missing __length_hint__ in __getitem__ iterator
Peter Otten wrote: > It's there, just not doctored into the dir() output: > >>>> class A(object): > ... def __getitem__(self, index): return index > ... def __len__(self): return 42 > ... >>>> iter(A()).__length_hint__ > >>>> iter(A()).__length_hint__() > 42 > > Peter Ah well, actually it's there also in dir(), you just need to use Python 2.5 of course ;) -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Missing __length_hint__ in __getitem__ iterator
Hello, given the following object: >>> class A(object): ... def __getitem__(self, idx): ... if idx >= 10: raise IndexError ... return idx ... def __len__(self): ... return 10 ... I noticed that the iterator that Python constructs: >>> a = A() >>> i = iter(a) >>> dir(i) ['__class__', '__delattr__', '__doc__', '__getattribute__', '__hash__', '__init__', '__iter__', '__len__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__str__', 'next'] does not have a __length_hint__ method. Is this just a missing optimization, or there is a deep semantic reason for which a __length_hint__ could not be constructed out of the __len__ result? -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Conditional iteration
at wrote: > THE PROBLEM > > I have a lot times the following code: > > for x in [-2, -1, 0, 1, 2, 3, 4]: > if x > 0: > ... more code... > > > It is not the addional line containing 'if x > 0:' that bothers me, but the > additional indentation. for x in ...: if not x > 0: continue ... more code ... -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Subprocess with a Python Session?
Fredrik Lundh wrote:
>> No matter what I do I cant get the following code to do what I expect.
>> I hadn't used subprocess t o read and write to pipes of a
>> still-running app, and I just can't seem to get it right. What gives?
>>
>> import subprocess
>>
>> p = subprocess.Popen("python", stdout=subprocess.PIPE,
>> stdin=subprocess.PIPE)
>> p.stdin.write('print 10\n')
>
> + p.stdin.close()
>
>> assert p.stdout.readline() == '10\n'
Yeah, but WHY was the API designed like this? Why can't I read and write
freely from a pipe connected to a process as many times as I want?
--
Giovanni Bajo
--
http://mail.python.org/mailman/listinfo/python-list
Re: len() and PEP 3000
Thomas Guettler wrote: > I have read the FAQ to the len function: > http://www.python.org/doc/faq/general/#why-does-python-use-methods-for-some-functionality-e-g-list-index-but-functions-for-other-e-g-len-list Outdated. You want to read the new FAQ, here: http://effbot.org/pyfaq/why-does-python-use-methods-for-some-functionality-e-g-list-index-but-functions-for-other-e-g-len-list.htm -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing data from pyserial, an (inefficient) solution
Dennis Lee Bieber wrote:
>
>> The really sad thing is that I get a perfectly constructed
>> packet from the reading variable, and that gets butchered when I
>> try to slice it up to pick out individual elements. Since
>> pyserial doesn’t do anything to rearrange the data, then the
>> CMUcam must do the heavy lifting of extracting a perfect packet
>> from the data stream. It’s a real shame I couldn’t use it
>
> More likely it is running fast enough to keep up with the serial
> port, and synchronizes on the "\rM". Your code seems to be reading in
> bursts and somehow losing parts during your processing (which is
> strange, I'd have expected pyserial to buffer some data between reads).
>
> Lacking that, I'd probably end up creating a thread to handle the
> serial port reading, which basically looks like:
>
> while ser.read(1) != "\r": pass #synchronize
> while not Done:
> bfr = []
> while True:
> c = ser.read(1)
> bfr.append(c)
> if c == "\r": break
> queue.put("".join(bfr))
>
> This should result in complete packets (from the "M" to a "\r")
Oh well. readline(eol="\r") will do that much better.
while 1:
data = ser.readline(eol="\r")
args = data.split()
if args[0] == "M":
# command M; do something with args[1:]
elif args[0] == "KK":
# command KK; do something with args[1:]
[.. process other commands ..]
else:
# Invalid command: might be an incomplete
# packet. Just ignore it.
pass
--
http://mail.python.org/mailman/listinfo/python-list
Re: Parsing data from pyserial
Lone Wolf wrote: > reading = ser.read(40) Simply try ser.readline() here, or maybe ser.readline(eol="\r"). -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Find interface associated with default route?
Neal Becker wrote:
> A quick strace reveals that 'route' just reads /proc/net/route, so:
>
> def get_default_if():
> f = open ('/proc/net/route', 'r')
> for line in f:
> words = string.split (line)
> dest = words[1]
> try:
> if (int (dest) == 0):
> interf = words[0]
> break
> except ValueError:
> pass
> return interf
And if you acknowledge that /proc/net/route is a CSV file, you can be more
terse and clear:
>>> import csv
>>> def get_default_if():
... f = open('/proc/net/route')
... for i in csv.DictReader(f, delimiter="\t"):
... if long(i['Destination'], 16) == 0:
... return i['Iface']
... return None
...
>>>
>>> get_default_if()
'ppp0'
--
Giovanni Bajo
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python to use a non open source bug tracker?
Tim Peters wrote: > None are /totally ignored/ -- indeed, at least I see every one as it > comes in. You might want to change your claim to that no work > obviously visible to you is done on them. That would be better. Please notice that my mail was in the context of "user satisfaction with the bug tracker". I was claiming that it is useless to provide a blazingly fast support turnaround for technical issue, when there is "no visibile work" being done on most bugs that are submitted. And, in turn, this was in the context of hiring 6-10 people as the only acceptable minimum to maintain and admin a bug tracker. I was claiming that, if such a group was ever formed, it was better spent on bug triage rather than keeping their keys ready all day long to quick-fix any server breakage in minutes. > These are the actual stats as of a few minutes ago: > > Bugs: 938 open of 7169 total ~= 87% closed > Patches: 429 open of 3846 total ~= 89% closed > Feature Requests: 240 open of 479 total ~= 50% closed I claimed different numbers based on personal perception; I stand corrected, and apologize for this. I could just say that my perception was wrong, but I guess there's something in it that could be further analyzed. For instance, I would be really curious to see how these figures look like if you consider only bugs/patches/rfe *NOT* submitted by python committers. I don't think Sourceforge can ever compute this number for us, so I'll wait to ask Roundup about it (or, uh, Jira...). > There's an easy way to improve these percentages dramatically, > although they're not bad as-is: run thru them and close every one > that isn't entirely clear. For example, reject every feature request, > close every patch that changes visible behavior not /clearly/ fixing a > bona fide bug, and close every "bug report" that's really a feature > request or random "but Perl/Ruby/PHP doesn't do it this way" complaint > in disguise. Either close directly any nonsense, or ask for more feedback to the poster, until the bug/patch/rfe is sufficiently clear to be handled, or 3 months are passed and you close the bug for "no further feedback from the poster". If this would dramatically reduce the number of open bugs, then yes, Python really needs someone to do bug triaging. > For example, the oldest patch open today is a speculative > implementation of rational numbers for Python. This is really a > feature request in disguise, and has very little chance-- but not /no/ > chance --of ever being accepted. The oldest bug open today is from 6 > years ago, and looks like an easy-to-answer /question/ about the > semantics of regular expressions in Python 1.6. I could take time to > close that one now, but is that a /good/ use of time? Yes, but, at > the moment, even finishing this reply seems to be a /better/ use of my > time -- and after that, I'm going to get something to eat ;-) It might be not a good use of your time at all, since you are a developer. But having a database with 938 open bugs most of which are incomplete/nonsense/unconfirmed is much less useful than it could be. It also raises the bar for new developers: it's much harder to just "pick one" and fix it. I know because I tried sometimes, and after half an hour I couldn't find any bug that was interesting to me and "complete" enough to work on it. I also noticed that most bugs are totally uncommented like nobody cared at all. This is where my thought about Python missing bug triaging started. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Python to use a non open source bug tracker?
Aahz wrote: >> Are you ever going to try and make a point which is not "you are not >> entitled to have opinions because you do not act"? Your sarcasm is >> getting annoying. And since I'm not trolling nor flaming, I think I >> deserve a little bit more of respect. > > IMO, regardless of whether you are trolling or flaming, you are > certainly being disrespectful. Why should we treat you with any more > respect than you give others? Disrespectful? Because I say that I don't agree with some specific requirement, trying to discuss and understand the rationale behind it? -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
New-style classes slower than old-style classes? (Was: n-body problem at shootout.alioth.debian.org)
Peter Maas wrote: >> Did you try using an old-style class instead of a new-style class? > > The original program has an old style class, changing it to a new > style class increases run time by 25% (version is 2.4.3 btw). Ah yes. Years ago when I first saw this test it was still using new-style classes. Anyway, this is a bug on its own I believe. I don't think new-style classes are meant to be 25% slower than old-style classes. Can any guru clarify this? -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Python to use a non open source bug tracker?
Steve Holden wrote: >> I understand your concerns, but I have to remember you that most bug >> reports submitted by users go totally ignored for several years, or, >> better, forever. I do not have a correct statistic for this, but I'm >> confident that at least 80% of the RFE or patches filed every week >> is totally ignored, and probably at least 50% of the bugs too. I >> think there is a much bigger problem here wrt QOS. >> >> So, you might prefer 6-10 people to activate a new tracker account >> faster than light. I'd rather have 3-days delay in administrative >> issues because our single administrator is sleeping or whatever, and >> then have 2-3 people doing regular bug processing. > > ... and if wishes were horses then beggars would ride. Are you ever going to try and make a point which is not "you are not entitled to have opinions because you do not act"? Your sarcasm is getting annoying. And since I'm not trolling nor flaming, I think I deserve a little bit more of respect. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: n-body problem at shootout.alioth.debian.org
Peter Maas wrote: > I have noticed that in the language shootout at > shootout.alioth.debian.org the Python program for the n-body problem > is about 50% slower than the Perl program. This is an unusual big > difference. I tried to make the Python program faster but without > success. Has anybody an explanation for the difference? It's pure > math so I expected Perl and Python to have about the same speed. Did you try using an old-style class instead of a new-style class? -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Python to use a non open source bug tracker?
[EMAIL PROTECTED] wrote: > Giovanni> Are bug-tracker configuration issues so critical that > having Giovanni> to wait 48-72hrs to have them fixed is > absolutely unacceptable Giovanni> for Python development? > > Yes, I think that would put a crimp in things. The downtimes we see > for the SourceForge tracker tend to be of much shorter duration than > that (typically a few hours) and cause usually minor problems when > they occur. For the tracker to be down for 2-3 days would make the I was actually thinking of 48-72hrs to do regulard admin work like installing latest security patch or actrivate a new account. > developers temporarily blind to all outstanding bug reports and > patches during that time and prevent non-developers from submitting > new bugs, patches and comments. Those people might well forget about > their desired submission altogether and not return to submit them > once the tracker was back up. I understand your concerns, but I have to remember you that most bug reports submitted by users go totally ignored for several years, or, better, forever. I do not have a correct statistic for this, but I'm confident that at least 80% of the RFE or patches filed every week is totally ignored, and probably at least 50% of the bugs too. I think there is a much bigger problem here wrt QOS. So, you might prefer 6-10 people to activate a new tracker account faster than light. I'd rather have 3-days delay in administrative issues because our single administrator is sleeping or whatever, and then have 2-3 people doing regular bug processing. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Python to use a non open source bug tracker?
[EMAIL PROTECTED] wrote: > Martin> The regular admin tasks likely include stuff like this: > Martin> - the system is unavailable, bring it back to work > Martin> This is really the worst case, and a short response time > Martin> is the major factor in how users perceive the service > Martin> - the system is responding very slowly > > To all those people who have been moaning about needing 6-10 people to > administer the system, in my opinion these are the most important > reasons to have more than one person available to help. Python isn't > only used in the USofA. It has been very helpful to have > administrators scattered around the globe who were awake and alert to > handle problems with python.org when folks in the US were asleep. Of > course, spreading the load among several people helps with the other > tasks as well. > > As Martin pointed out in an earlier post, with only one person > actively administering Subversion (Martin), new requests for access > had to wait if he was away for an extended period of time. This is true of many open source projects. I don't dispute that having 6-10 people to administer Roundup would not be good. I dispute that it is the minimum requirement to make a Roundup installation acceptable for Python development. Are bug-tracker configuration issues so critical that having to wait 48-72hrs to have them fixed is absolutely unacceptable for Python development? It looks like an overexaggeration. People easily cope with 2-3 days of SVN freezing, when they are politically (rather than technically) stopped from committing to SVN. I guess they can wait 48 hrs to be able to close that bug, or open that other one, or run that query. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Python to use a non open source bug tracker?
Martin v. Löwis wrote: > That, in principle, could happen to any other free software as well. > What is critical here is that SF *hosted* the installation. If we > would use a tracker that is free software, yet hosted it elsewhere, > the same thing could happen: the hoster could make modifications to > it which > are non-free. Not even the GPL could protect from this case: the > hoster would be required to publish source only if he publishes > binaries, but he wouldn't publish any binaries, so he wouldn't need > to release the source changes, either. > > Also, even if it the software is open source and unmodified, there > still wouldn't be a guarantee that you can get the data out of it > if you want to. You *only* get the advantages of free software if > you also run it yourself. Unfortunately, there is a significant > cost associated with running the software yourself. You have many good points here, Martin. Let me notice, though, that people providing hosting not necessarily want to maintain the software by themselves alone: some python developers could still have admin access to the boxes so to double check if weird things are being done behind the curtain. I think the point of uncertainty araises only if you totally trust someone else to do the job for you. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Python to use a non open source bug tracker?
Martin v. Löwis wrote: >> In fact, are you absolutely positive that you need so much effort to >> maintain an existing bugtracker installation? I know for sure that >> GCC's Bugzilla installation is pretty much on its own; Daniel Berlin >> does some maintainance every once in a while (upgrading when new >> versions are out, applying or writing some patches for most >> requested features in the community, or sutff like that), but it's >> surely not his job, not even part-time. > > Daniel Berlin has put a tremendous amount of work into it. I know, > because I set up the first bug tracker for gcc (using GNATS), and > have been followed the several years of pondering fairly closely. > It was quite some work to set up GNATS, and it was even more work > to setup bugzilla. > > For Python, we don't have any person similar to Daniel Berlin > (actually, we have several who *could* have done similar work, > but none that ever volunteered to do it). Don't underestimate > the work of somebody else. Martin, I am by no means understimating Daniel's work. I am just noting that the spare-time work he did is, by definition, much much lower than the "6-10 people" that the PSF infrastructure committee is calling for. I would like this statement to be officially reduced to "2-3 people", since it is *really* not required much more than that to setup a bug tracker installation, and no more than 1 person to maintain it afterwards. *IF* there are more volunteers, that's good, they can offload the maintenance work from a single maintainer; but I think it's unfair to put such a high *requisite*. We do not have 6-10 people maintaining SVN afterall, even if you wish we had :) -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Python to use a non open source bug tracker?
David Goodger wrote: > Go back to the original announcement: > > """ > After evaluating the trackers on several points (issue creation, > querying, etc.), we reached a tie between JIRA and Roundup in terms of > pure tracker features. > """ > > JIRA gets a leg up because of the hosting and administration also > being offered. But... > > """ > If enough people step forward we will notify python-dev that Roundup > should be considered the recommendation of the committee and > graciously > turn down Atlassian's offer. > """ > > That is a perfectly reasonable offer. Put up or shut up. You're cherry picking your quotes: """ In order for Roundup to be considered equivalent in terms of an overall tracker package there needs to be a sufficient number of volunteer admins (roughly 6 - 10 people) who can help set up and maintain the Roundup installation. """ This is *NOT* a perfectly reasonable offer, because you do not see 6-10 people stepping up at the same time for almost *anything* in the open source world. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Python to use a non open source bug tracker?
A.M. Kuchling wrote: >> The surprise people are expressing is because they thought one of the >> goals of a big open source project would be to avoid reliance on >> closed tools. > > I don't think Python has ever had this as a goal. Python's license > lets it be embedded in closed-source products; Windows binaries are > built using closed-source tools (MS Visual C), and on some platforms > we use a closed-source system compiler; python.org used to be a > Solaris box, and now uses SourceForge which runs on top of DB/2... Notice that there is a different between "allowing/helping/supporting non-free software" and "avoid reliance on non-free software". The fact that Python license allows it to be used in non-free products falls in the former, while the usage of Jira is part of the latter. Distributing binaries compiled with closed-source tools is not a problem since people can still compile it with different free compilers. > IMHO, using Jira presents risks that are manageable: > [...] > > * A data export is available if we decide to switch. [...] Out of curiosity, how is this obtained? Is this any plan to take a daily export or so? -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Python to use a non open source bug tracker?
Paul Rubin wrote: >> You fail to recognize that Python is *already* using a non-free >> software for bug tracking, as do thousands of other projects. > > I don't think that reflects an explicit decision. SF started out as > free software and the software became nonfree after people were > already using it. Moreover, this looked like a very good chance to have this nuisance sorted out. Too bad some people don't value free software enough. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Python to use a non open source bug tracker?
Martin v. Löwis wrote: >> Frankly, I don't give a damn about the language the application is >> coded in > > That's probably one of the reasons why you aren't a member of the > Python Software Foundation. Its mission includes to publicize, > promote the > adoption of, and facilitate the ongoing development of Python-related > technology and educational resources. So the tracker being written in > Python is quite of importance. So we have a problem between the PSF and the "PSF infrastructure committee", since the latter did not put "being written in Python" has a requirement for the tracker. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Python to use a non open source bug tracker?
Steve Holden wrote: > No, I'm not on the infrastructure list, but I know that capable people > *are*: and you know I am quite capable of donating my time to the > cause, when I have it to spare (and sometimes even when I don't). > > Perhaps what I *should* have written was "Sadly *many* people spend > too much time bitching and moaning about those that roll their > sleeves up, and not enough rolling their own sleeves up and pitching > in". > > Sniping from the sidelines is far easier than hard work towards a > goal. > > Kindly note that none of the above remarks apply to you. The current request is: "please, readers of python-dev, setup a team of 6-10 people to handle roundup or we'll go to a non-free software for bug tracking". This is something which I cannot cope with, and I'm *speaking* up against. Were the request lowered to something more reasonable, I'd be willing to *act*. I have to speak before acting, so that my acting can produce a result. And besides the only thing I'm really sniping the PSF against is about *ever* having thought of non-FLOSS software. This is something I *really* do not accept. You have not seen a mail from me with random moaning as "Trac is better", "Bugzilla is better", "why this was chosen". I do respect the fact that the PSF committee did a thorough and correct evaluation: I just disagree with their initial requirements (and I have not raised this point before because, believe me if you can, I really thought it was obvious and implicit). So, if your remarks apply to me, I think you are misrepresenting my mails and my goals. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Python to use a non open source bug tracker?
[EMAIL PROTECTED] wrote: > Giovanni> In fact, are you absolutely positive that you need so > much Giovanni> effort to maintain an existing bugtracker > installation? > > The development group's experience with SF and I think to a lesser > extent, Roundup in its early days, and more generally with other > components of the development toolchain (source code control) and > python.org website maintenance suggests that some human needs to be > responsible for each key piece of technology. Maybe when it's mature > it needs very little manpower to maintain, but a substantial > investment is required when the technology is first installed. One thing is asking for a special help during the transition phase and the "landing" phase (the first few months). Another thing is asking for "roughly 6-10 people" to install and maintain a Roundup installation. This is simply not going to realistically happen, and I find it incredible for the PSF committee to ask for such a high request. Damn, we don't have "roughly 6-10 people" in charge of reviewing patches or fixing bugs. I followed the GNATS -> Bugzilla transition myself closely, and a single person (Daniel Berlin) was able to setup the Bugzilla server on the gcc.gnu.org computer, convince everybody that a transition was needed (and believe me, this was a hard work), patch it as much as needed to face the needs of the incredibly picky GCC developers (asking for every little almost-unused-and-obsoleted feature in GNATS to be replicated in Bugzilla), and later maintain the installation. It took him approximately one year to do this, and surely it wasn't full time. After that, he maintains and administer the Bugzilla installation on his own, by providing upgrades when needed and a few modifications. I wonder why the PSF infrastructure committee believes that a group of 6-10 people is needed to "install and maintain" Roundup. Let us also consider that Roundup's lead developer *was* part of the PSF infrastrucutre committee, and he might be willing to help in the transition (just my very wild guess), and he obviously knows his stuff. Also, given the requirement for the selection, there is already a running roundup installation somewhere (so the whole pipeline export -> import has already been estabilished and confirmed to work). My own opinion is that a couple of person can manage the transition/migration phase to *any* other bug tracking system, and provide support in the python-dev mailing list. After the whole thing has fully landed, I'd be really surprised if a single appointed maintainer would not be enough. If the PSF committee lowers its requests to a more realistical amount of effort, I'm sure we will see many more people willing to help. I think many people (including myself) would be willing to half-half-help with loose ends, but when faced with an abnormous "6-10 people" request they just shut up and sit in a corner. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: dictionary of list from a file
[EMAIL PROTECTED] wrote:
> while(){
> @info=split(/ +/,$_);
> push (@{$tmp{$info[0]}},$info[1]);
> }
>
> and then
> foreach $key (keys %tmp){
>print "$key -> @{$tmp{$key}}\n";
> }
Python 2.5 introduced a dictionary type with automatic creation of values,
ala Perl:
===
from collections import defaultdict
d = defaultdict(list)
for line in fl:
k, v = line.strip().split()
d[k].append(v)
for k,v in d.items():
print k, v
===
Notice that Python is always more strongly typed, so you have to specify a
factory function.
--
Giovanni Bajo
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python to use a non open source bug tracker?
A.M. Kuchling wrote: >> I am seriously concerned >> that the PSF infrastructure committee EVER considered non open-source >> applications for this. In fact, I thought that was an implicit >> requirement in the selection. > > Being open source wasn't a requirement; which is, indeed, shocking and amazing. > minimal requirements were > specified in the initial message requesting trackers > (http://wiki.python.org/moin/OriginalCallForTrackers). Where does it mention that only trackers which have at least an existing installation and a group of people for maintenance will be considered? It could easily be assumed that PSF had already enough bandwidth, server, manpower to handle any bugtracker installation. In fact, are you absolutely positive that you need so much effort to maintain an existing bugtracker installation? I know for sure that GCC's Bugzilla installation is pretty much on its own; Daniel Berlin does some maintainance every once in a while (upgrading when new versions are out, applying or writing some patches for most requested features in the community, or sutff like that), but it's surely not his job, not even part-time. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Python to use a non open source bug tracker?
Martin v. Löwis wrote: > It's significantly different from the Bitkeeper fiasco in two > important > ways: > 1. Bitkeeper is a source revisioning system, so it is similar to CVS >and Subversion. This project here is "just" the bug tracker, which >is of lesser importance. If we move to a different one some day, a >certain amount of data lossage might be acceptable (e.g. we now >likely lose the "history" of status changes and file attachments on >each report). An export of all data is high on the requirements >list, as Fredrik points out. I understand your point. OTOH, exactly because the tracker system is a far lesser importance, it's amazing there is *ever* a need to evaluate non-FLOSS solutions, when there are so many good free solutions around. Instead of recommending a closed source solution, you could have recommended Roundup *and* explained there is a need for funding and/or volunteers before the migration can happen. You might also be understimating how negative could be the reaction from the open-source community to such a move. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Python to use a non open source bug tracker?
A.M. Kuchling wrote: >> ... using a non open source tracker (called JIRA - never heard >> before of course) for Python itself. > > Other projects do use it; see > <http://wiki.apache.org/general/ApacheJira> for a partial list, and a > link to the Apache Software Foundation's issue trackers. which, in my humble opinion, is just a list of other examples of projects which are misguided. People seem to have no idea when a company is sold, when a CEO is changed, or something like that. Hhistory's repeating, but hackers are not learning. >> Does this smell "Bitkeeper fiasco" to anyone else than me? > > The committee did expect this recommendation to be controversial. :) I guess :) In fact, I have a deepest hope that this recommendation was just a fake just to get people setup a roundup installation... -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Python to use a non open source bug tracker?
[EMAIL PROTECTED] wrote: > Giovanni Bajo wrote: > >> Does this smell "Bitkeeper fiasco" to anyone else than me? > > I can't understand why people waste time arguing this stuff. > > Use whatever tool is best at it's job... if it's not written in Python > it doesn't mean that Python is not good for the task, only that there > hasn't been any Python programmer that applied himself to the problem > hard enough. This is not against using a tool not written in Java. This is against using a tool which is not FLOSS. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Python to use a non open source bug tracker?
Fredrik Lundh wrote: > that's just not true. lots of people have voiced concerns over using > closed-sourced stuff originally designed for enterprise-level Java > users for an application domain where Python has several widely used > agile alternatives to chose from. Frankly, I don't give a damn about the language the application is coded in, as long as it is OUR application and not an application of a private company which we do not own. Even though you are totally right. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Python to use a non open source bug tracker?
Martin v. Löwis wrote: >> I hope this >> recommendation from the "PSF infrastructure committee" is rejected. > > That is very very unlikely. Who would reject it, and why? The community, and I am impressed you do not want to understand the "why". It is an extremely bad picture for an open source flag like Python to go to a vendor for such an easy requirement as a bug database. I am seriously concerned that the PSF infrastructure committee EVER considered non open-source applications for this. In fact, I thought that was an implicit requirement in the selection. There are many open source applications around. They might not be the best, but at least they are yours, and not of some third party software vendors with its own interests. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Python to use a non open source bug tracker?
Hello, I just read this mail by Brett Cannon: http://mail.python.org/pipermail/python-dev/2006-October/069139.html where the "PSF infrastracture committee", after weeks of evaluation, recommends using a non open source tracker (called JIRA - never heard before of course) for Python itself. Does this smell "Bitkeeper fiasco" to anyone else than me? -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Is this a bug? Python intermittently stops dead for seconds
Steve Holden wrote: > I think you'll find that most programs that eat through memory in this > way will exhibit pretty much the same behaviour. If you *know* your > program isn't creating data cycles, just turn the GC off and rely on > reference counting. That won't save you from paging when you > eventually exhaust physical memory. Nothing can. Even better, if you know that you're *creating* tons of objects without creating many *discardable* cycles at the same, it's better to turn off GC collection during loading, and only do a single pass (gc.collect()) when you are done with the allocations. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Is this a bug? Python intermittently stops dead for seconds
charlie strauss wrote: > Below is a simple program that will cause python to intermittently > stop executing for a few seconds. it's 100% reproducible on my > machine. Confirmed with Python 2.4.2 on Windows. gc.disable() fixes it, so it looks like you found an inefficiency in the Python's GC. I have no idea whether this would be considered a bug by Python's developer, but you can try opening a bugreport... -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
time.clock() going backwards??
Hello, I experimented something very strange, a few days ago. I was debugging an application at a customer's site, and the problem turned out to be that time.clock() was going "backwards", that is it was sometimes (randomically) returning a floating point value which was "less than" the value returned by the previous invokation. The computer was a pretty fast one (P4 3Ghz I think, running Windows XP), and this happened only between very close invokations of time.clock(). I have triple-verified this, including printing the repr() of the floating point number and verifying it was really minor than the previous value by a few microseconds. In other words, I'm absolutely positive that it's not a mistake on my side, but that time.clock() was really apparently "jumping backward". This was confusing the hell out of my application, of course, and I just hacked it so to ignore these bogus reads, and just reading off again. Since the error was just of a few microseconds, reading time.clock() again produces a number which was higher than what I had before, and thus OK for my application. I was wondering if someone had experimented this behaviour before. I tried googling but to no effect. Is it possible this to be a bug in Python itself (maybe, shooting at the moon, in the conversion between the 64bit performance counter and the floating point representation returned by time.clock()), or could it be a bug in Windows itself or the mother board drivers (buf if so, wouldn't other application start going mad)? -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Python 2.5b2 Windows binaries
Hello, since I tested Python 2.5b2 on my applications, I have rebuilt some extension modules I needed. It wasn't a very simple or fast task, so I thought I'd share the result of the efforts: http://www.develer.com/oss/Py25Bins this page contains the Windows binaries (installers) for the following packages: - NumPy 0.98 - Numeric 24.2 - PyOpenGL 2.0.2.01 (with Numeric 24.2) - Pyrex 0.9.4.1 (with a Python 2.5 compatibility patch posted in its mailing list) I plan to update this page later as I build more installers (but don't hold your breath). Hope this helps everybody! -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: struct: type registration?
John Machin wrote:
>> Looks like you totally misread my message.
>
> Not at all.
>
> Your function:
>
> def mystring_pack(s):
> if len(s) > 20:
> raise ValueError, "a mystring can be at max 20 chars"
> s = (s + "\0"*20)[:20]
> s = struct.pack("20s", s)
> return s
>
> can be even better replaced by (after reading the manual "For packing,
> the string is truncated or padded with null bytes as appropriate to
> make it fit.") by:
>
> def mystring_pack(s):
> if len(s) > 20:
> raise ValueError, "a mystring can be at max 20 chars"
> return s
> # return s = (s + "\0"*20)[:20] # not needed, according to the
> manual # s = struct.pack("20s", s)
> # As I said, this particular instance of using struct.pack is a
> big fat no-op.
John, the point of the example was to show that one could write custom
packer/unpacker which calls struct.pack/unpack and, after that,
post-processes the results to obtain some custom data type. Now, I apologize
if my example wasn't exactly the shortest, most compact, most pythonic piece
of code. It was not meant to be. It was meant to be very easy to read and
very clear in what it is being done. You are nitpicking that part of my code
is a no-op. Fine. Sorry if this confused you. I was just trying to show a
simple pattern:
custom packer: adjust data, call struct.pack(), return
custom unpacker: call struct.unpack(), adjust data, return
I should have chosen a most complex example probably, but I did not want to
confuse readers. It seems I have confused them by choosing too simple an
example.
--
Giovanni Bajo
--
http://mail.python.org/mailman/listinfo/python-list
Re: struct: type registration?
Giovanni Bajo wrote:
> You need struct.unpack() to parse these datas, and you need custom
> packer/unpacker to avoid post-processing the output of unpack() just
> because it just knows of basic Python types. In binary structs, there
> happen to be *types* which do not map 1:1 to Python types, nor they
> are just basic C types (like the ones struct supports). Using custom
> formatter is a way to better represent these types (instead of
> mapping them to the "most similar" type, and then post-process it).
>
> In my example, "S" is a basic-type which is a "A 0-terminated 20-byte
> string", and expressing it in the struct format with the single
> letter "S" is more meaningful in my code than using "20s" and then
> post-processing the resulting string each and every time this happens.
Another compelling example is the SSH protocol:
http://www.openssh.com/txt/draft-ietf-secsh-architecture-12.txt
Go to section 4, "Data Type Representations Used in the SSH Protocols", and it
describes the data types used by the SSH protocol. In a perfect world, I would
write some custom packers/unpackers for those types which struct does not
handle already (like the "mpint" format), so that I could use struct to parse
and compose SSH messages. What I ended up doing was writing a new module
sshstruct.py from scratch, which duplicates struct's work, just because I
couldn't extend struct. Some examples:
client.py:cookie, server_algorithms, guess, reserverd =
sshstruct.unpack("16b10LBu", data[1:])
client.py:prompts = sshstruct.unpack("sssu" + "sB"*num_prompts,
pkt[1:])
connection.py:pkt = sshstruct.pack("busB", SSH_MSG_CHANNEL_REQUEST,
self.recipient_number, type, reply) + custom
kex.py:self.P, self.G = sshstruct.unpack("mm",pkt[1:])
Notice for instance how "s" is a SSH string and unpacks directly to a Python
string, and "m" is a SSH mpint (infinite precision integer) but unpacks
directly into a Python long. Using struct.unpack() this would have been
impossible and would have required much post-processing.
Actually, another thing that struct should support to cover the SSH protocol
(and many other binary protocols) is the ability to parse strings whose size is
not known at import-time (variable-length data types). For instance, type
"string" in the SSH protocol is a string prepended with its size as uint32. So
it's actual size depends on each instance. For this reason, my sshstruct did
not have the equivalent of struct.calcsize(). I guess that if there's a way to
extend struct, it would comprehend variable-size data types (and calcsize()
would return -1 or raise an exception).
--
Giovanni Bajo
--
http://mail.python.org/mailman/listinfo/python-list
Re: struct: type registration?
John Machin wrote:
>> given the ongoing work on struct (which I thought was a dead
>> module), I was wondering if it would be possible to add an API to
>> register custom parsing codes for struct. Whenever I use it for
>> non-trivial tasks, I always happen to write small wrapper functions
>> to adjust the values returned by struct.
>>
>> An example API would be the following:
>>
>>
>> def mystring_len():
>> return 20
>>
>> def mystring_pack(s):
>> if len(s) > 20:
>> raise ValueError, "a mystring can be at max 20 chars"
>> s = (s + "\0"*20)[:20]
>
> Have you considered s.ljust(20, "\0") ?
Right. This happened to be an example...
>> s = struct.pack("20s", s)
>> return s
>
> I am an idiot, so please be gentle with me: I don't understand why you
> are using struct.pack at all:
Because I want to be able to parse largest chunks of binary datas with custom
formatting. Did you miss the whole point of my message:
struct.unpack("3liiSiiShh", data)
You need struct.unpack() to parse these datas, and you need custom
packer/unpacker to avoid post-processing the output of unpack() just because it
just knows of basic Python types. In binary structs, there happen to be *types*
which do not map 1:1 to Python types, nor they are just basic C types (like the
ones struct supports). Using custom formatter is a way to better represent
these types (instead of mapping them to the "most similar" type, and then
post-process it).
In my example, "S" is a basic-type which is a "A 0-terminated 20-byte string",
and expressing it in the struct format with the single letter "S" is more
meaningful in my code than using "20s" and then post-processing the resulting
string each and every time this happens.
>>>>> import struct
>>>>> x = ("abcde" + "\0" * 20)[:20]
>>>>> x
> 'abcde\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
>>>>> len(x)
> 20
>>>>> y = struct.pack("20s", x)
>>>>> y == x
> True
>>>>>
>
> Looks like a big fat no-op to me; you've done all the heavy lifting
> yourself.
Looks like you totally misread my message. Your string "x" is what I find in
binary data, and I need to *unpack* into a regular Python string, which would
be "abcde".
>
>> idx = s.find("\0")
>> if idx >= 0:
>> s = s[:idx]
>> return s
>
> Have you considered this:
>
>>>>> z.rstrip("\0")
> 'abcde'
This would not work because, in the actual binary data I have to parse, only
the first \0 is meaningful and terminates the string (like in C). There is
absolutely no guarantees that the rest of the padding is made of \0s as well.
--
Giovanni Bajo
--
http://mail.python.org/mailman/listinfo/python-list
struct: type registration?
Hello,
given the ongoing work on struct (which I thought was a dead module), I was
wondering if it would be possible to add an API to register custom parsing
codes for struct. Whenever I use it for non-trivial tasks, I always happen to
write small wrapper functions to adjust the values returned by struct.
An example API would be the following:
def mystring_len():
return 20
def mystring_pack(s):
if len(s) > 20:
raise ValueError, "a mystring can be at max 20 chars"
s = (s + "\0"*20)[:20]
s = struct.pack("20s", s)
return s
def mystring_unpack(s):
assert len(s) == 20
s = struct.unpack("20s", s)[0]
idx = s.find("\0")
if idx >= 0:
s = s[:idx]
return s
struct.register("S", mystring_pack, mystring_unpack, mystring_len)
# then later
foo = struct.unpack("iilS", data)
This is only an example, any similar API which might fit better struct
internals would do as well.
As shown, the custom packer/unpacker can call the original pack/unpack as a
basis for their work. I guess an issue with this could be the endianess
problem: it would make sense if, when called recursively, struct.pack/unpack
used by the default the endianess specified by the external format string.
--
Giovanni Bajo
--
http://mail.python.org/mailman/listinfo/python-list
Re: genexp performance problem?
Fredrik Lundh wrote: >> I found this strange: >> >> python -mtimeit "sum(int(L) for L in xrange(3000))" >> 100 loops, best of 3: 5.04 msec per loop >> >> python -mtimeit "import itertools; sum(itertools.imap(int, >> xrange(3000)))" 100 loops, best of 3: 3.6 msec per loop >> >> I thought the two constructs could achieve the same speed. > > hint: how many times to the interpreter have to look up the names > "int" > and "L" in the two examples ? Ah right, thanks! -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
genexp performance problem?
Hello, I found this strange: python -mtimeit "sum(int(L) for L in xrange(3000))" 100 loops, best of 3: 5.04 msec per loop python -mtimeit "import itertools; sum(itertools.imap(int, xrange(3000)))" 100 loops, best of 3: 3.6 msec per loop I thought the two constructs could achieve the same speed. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: python 2.5a2, gcc 4.1 and memory problems
Michele Petrazzo wrote: > Then I execute my test. The memory usage of 2.5a2 and gcc 3.3 that I > see with "top", is the same (about VIRT: 260 MB and RES: 250MB ) that > with the py 2.3 and 2.4, but then I recompile with 4.1 and execute > the same test, my system "stop to work"... with "top" I can see that > it use VIRT: 2440 MB and RES: 640MB RAM (I think all that I have into > my pc) > > I haven't tried to recompile py 2.4 myself with gcc 4.1 because it is > already compiled with it (4.0.3), so I think (only think) that is a py > 2.5 problem. > I'm right? or I have to compile it with something other switches? Make sure -fno-strict-aliasing is specified. Python does not conform to ISO C type aliasing rules. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Using pyinstaller to create an .exe, when using win32com
kbperry wrote: > Hi all, > I am pretty confused on this? > > While pyinstaller has a "MakeCOMServer.py" thing, it seems to blow up > every time I run it. It keeps complaining that a dictionary that I > have has "TypeError: unscriptable object.", but when I run my code > normally (outside of trying to make an .exe), it runs perfectly. The > dictionary doesn't have any data in it, until I call a method to grab > some data from a file. > > Has anyone else tried to use this? What is the best way to create an > .exe when using the win32com module? I found makeExe.py that uses > py2exe.py, but I didn't have any luck with that either. I did some > searching, too, but to no avail. MakeCOMServer.py hasn't been touched (nor tested) since I started maintaing PyInstaller, so it might well be broken in some way. If you provide me with a complete bugreport (on the pyinstaller list, or as a ticket in the pyinstaller bug system) I might have a look. Alas, I have little time to spend on PyInstaller at the moment, but I'm eventually going to tackle this. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: efficiency of range() and xrange() in for loops
Steve R. Hastings wrote: >> in Python 2.X, range is defined to return a list. if you start >> returning something else, you'll break stuff. > > Perhaps I'm mistaken here, but I don't see how this optimization could > possibly break anything. Because you assume that the only use-case of range() is within a for-loop. range() is a builtin function that can be used in any Python expression. For instance: RED, GREEN, BLUE, WHITE, BLACK = range(5) -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: New Python Logo Revealed
robin wrote: > I have an entry on my blog discussing the new Python logo, which is > apparently due to replace the current one within the month. I'd be > interested in what people think of it. > > Surf: > http://diagrammes-modernes.blogspot.com Tell me tell me it's an april's joke, please? :) -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: whats your favourite object relational mapper?
Jonathan Ellis wrote: > ... which, of course, goes to show how stupid a metric this is, now > that even Ian Bicking has admitted that SqlObject in its current form > is a dead end. Got a pointer? -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: sizeof(struct timeval)
Tony Houghton wrote: > I'm writing a python program which reads input device events so it > needs to know sizeof(struct timeval). By using the struct module I > should be able to work out sizeof(long) from python, but I can't > think of a way to measure non-fundamental types without including a > little bit of C, which I'd rather avoid. > > How safe would I be assuming that > > sizeof(struct timeval) == 2 * sizeof(long) > > is always true on Linux on different architectures? AFAIK the input > device interface is Linux-specific so I don't need to worry about > other OS's. If I were you, I would write a Pyrex 4-liners which exposes this structure to Python in the way you prefer. This would immediately fix all these compatibility issues. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: trying to find repeated substrings with regular expression
Robert Dodier wrote: > Hello all, > > I'm trying to find substrings that look like 'FOO blah blah blah' > in a string. For example give 'blah FOO blah1a blah1b FOO blah2 > FOO blah3a blah3b blah3b' I want to get three substrings, > 'FOO blah1a blah1b', 'FOO blah2', and 'FOO blah3a blah3b blah3b'. > [...] > Can someone suggest a regular expression which will return > groups corresponding to the FOO substrings above? FOO.*?(?=(?:FOO|$)) -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: popen and exit code on Windows
[EMAIL PROTECTED] wrote: > On UNIX one can use popen* to get a pipe for reading, a pipe for > writing, and the exit code of the child process via a call to close() > on the last pipe. Is there any way, in principle, to simulate such > behaviour on Windows? Some googling reveals that direct use of the > popen* functions on Windows will not do the trick, but are there > indirect ways? This is well-tested: def launch(cmd, split_lines=True): """Launch a sub-process. Return its output (both stdout and stderr), optionally split by lines (if split_lines is True). Raise a LaunchError exception if the exit code of the process is non-zero (failure).""" if os.name not in ['nt', 'os2']: p = popen2.Popen4(cmd) p.tochild.close() if split_lines: out = p.fromchild.readlines() else: out = p.fromchild.read() ret = p.wait() if ret == 0: ret = None else: ret >>= 8 else: i,k = os.popen4(cmd) i.close() if split_lines: out = k.readlines() else: out = k.read() ret = k.close() if ret is None: return out raise LaunchError(ret, cmd, out) -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: PEP 354: Enumerations in Python
Ben Finney wrote: > Values within an enumeration cannot be meaningfully compared except > with values from the same enumeration. The comparison operation > functions return ``NotImplemented`` [#CMP-NOTIMPLEMENTED]_ when a > value from an enumeration is compared against any value not from the > same enumeration or of a different type:: > [...] > This allows the operation to succeed, evaluating to a boolean value:: Given that this is going to change in Python 3.0 for builtin types, I'm not sure there's much reason to keep it this way. I'd rather a comparison operation between different enum types to raise an exception. This would be a very compelling feature to use enum (and in fact, the implementation I chose in Cookbook for my programs have this feature). > Coercing a value from an enumeration to a ``str`` results in the > string that was specified for that value when constructing the > enumeration:: > > >>> gym_night = Weekdays.wed > >>> str(gym_night) > 'wed' What's the repr of an enumeration value? OTOH, it should be something like "Weekdays.wed", so that eval(repr()) holds true. Also, it'd be very useful in debug dumps, tracebacks and whatnot. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: PEP 354: Enumerations in Python
Kay Schluehr wrote: >> The enumerations specified in this PEP are instances of an ``enum`` >> type. Some alternative designs implement each enumeration as its own >> class, and a metaclass to define common properties of all >> enumerations. >> >> One motivation for having a class (rather than an instance) for each >> enumeration is to allow subclasses of enumerations, extending and >> altering an existing enumeration. A class, though, implies that >> instances of that class will be created; it is difficult to imagine >> what it means to have separate instances of a "days of the week" >> class, where each instance contains all days. This usually leads to >> having each class follow the Singleton pattern, further complicating >> the design. >> >> In contrast, this PEP specifies enumerations that are not expected to >> be extended or modified. It is, of course, possible to create a new >> enumeration from the string values of an existing one, or even >> subclass the ``enum`` type if desired. > > Maybe a metaclass implementation complicates design, but usage is > quite simple and flexible. The classics is already referenced by the > Python docs: > > http://www.python.org/doc/essays/metaclasses/Enum.py Agreed. Allowing subclassing of an existing enum is a strong plus to me. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Can optparse do dependencies?
Raymond Hettinger wrote:
>> I'd like to setup command line switches that are dependent on other
>> switches, similar to what rpm does listed below. From the grammar
>> below we see that the "query-options" are dependent on the query
>> switch, {-q|--query}. Can "optparse" do this or do I have to code my
>> own "thing"? Thanks.
>>
>> QUERYING AND VERIFYING PACKAGES:
>>rpm {-q|--query} [select-options] [query-options]
>> ...
>>query-options
>> [--changelog] [-c,--configfiles] [-d,--docfiles] [--dump]
>> [--filesbypkg] [-i,--info] [--last] [-l,--list]
>> [--provides] [--qf,--queryformat QUERYFMT]
>> [-R,--requires] [--scripts] [-s,--state]
>> [--triggers,--triggerscripts]
>
> The optparse module doesn't have native support for switch
> dependencies; however, it could likely be done with multiple passes.
> The parse_args() takes an args list as an argument. Make first pass
> that captures your main query switches, then run another parser on a
> slice of the args list.
>
> For example, capture the main switches on a first pass over the full
> argument list (catching all possible main switches and treating
> everything else as a catchall). Given, args=['-ql', '--changelog',
> '-d', '-c'], parse out the --changelog and then call another parser
> with args=['-d', '-c'].
>
> This functionality seems useful enough to build into the tool
> directly, so do consider putting a feature request on SourceForge
> (and assign to Greg Ward).
In fact, if we were to request an addition to optparse in this direction, I
think it should add standardized support for the "subcommand pattern", that is
the same command line arrangement that CVS, SVN and other programs uses. rpm
doesn't use it and I consider this an error in UI design (it should really have
been "rpm query --changelog" and similar).
--
Giovanni Bajo
--
http://mail.python.org/mailman/listinfo/python-list
Re: Little tool - but very big size... :-(
Felipe Almeida Lessa wrote: >> compared to the small wxPython tool that I compressed recently: 2,80 >> MB (2.942.543 Bytes) > > What about PyGtk? Does anybody have any figures? I can't test here > =(... I have a custom compilation script for PyQt which can package Qt 3.3.5 Core + OpenGL + Table in a single .pyd of 1.2Mb (compressed). You can add to that the usual Python overhead (around 1Mb compressed), thanks to having those useless CJK codecs compiled within python24.dll (sigh!). -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Little tool - but very big size... :-(
Durumdara wrote: > I need to have more compressed result. Can I compress dll-s, pyd-s > with Py2Exe ? > Can I decrease the total size with something ? > > If not, how to I create an self-unpackager and self-starter program > that use an temporary directory in the disk ? With WinRar ? You can use PyInstaller (http://pyinstaller.hpcf.upr.edu). PyInstaller easily builds a single executable with everything inside it. It also optionally compresses everything with UPX (http://upx.sourceforge.net/) so to reduce the size at the maximum. There are also other choices that can be made. For instance, wxWidgets is *HUGE*. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Mutable numbers
Suresh Jeevanandam wrote: > In python all numbers are immutable. True. > This means there is one object ( > a region in the memory ) created every time we do an numeric > operation. False. Memory regions can be pooled in a way that "allocation" is a O(1) operation. > I hope there should have been some good reasons why it was > designed this way. Yes, so that you can obtain a pass-by-value semantic for numeric types. That is, if you pass a number to a function, it can't modify it. You consistenly use "a = foo(a)" to do a calculation on "a", instead of having half programs doing "foo(a)" and the other half doing "a = foo(a)". Also, it's done so that you don't need to special case integer literals: they are (fixed) names to the same immutable instances and nobody can modify the value of 13 or any other literal. > But why not have mutable numbers also in the language. A type which > would behave as follows: > > a = MutableInt(12) > b = a > > Now both a and b should refer to the same memory location. Any change > in the object should get reflected in all the references. > > a.assign(13) # Same memory location should be updated with value 13, b > is also 13 now. > > Now we can further optimize: > > a.incrementOne() # equivalent to a++ in C++ > a.decrementOne() > > and the augmented assignment operation also could be made optimized. So you just created a numeric type which does not work with normal operators. This is so unpythonic. I don't think anybody wants to write "incrementOne" when "+ 1" exists and does the same it used to do in primary school. Even if you overloaded all the operators, you still wouldn't be able to use '=' to do assignment, which pretty much makes your class useless or very unpythonic to use (= wrong). > In any application most of the operation is numerical. So, i think, we > should get a good speed advantage with the availability of mutable > numbers. What do you think ? You are doing premature optimization. It's so premature that it's even a brain thing. You *presume* Python to be slow, you *presume* that this presumed slowness comes from numbers being immutables. But you didn't *measure* it, you didn't try it on a real-world case. You didn't even look at the Python source code to check if your assumptions were true. Try to check your assumptions before designing solutions to problems which do not exist. I suggest you start writing real-world Python before doing so many assumptions. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: how to break a for loop?
[EMAIL PROTECTED] wrote: >> I need to remove zeros from >> the begining of list, but I can't :-(. > > I believe the following is almost a direct translation of the above > sentence. > > import itertools as it > a=[0,0,0,1,0] > a[:]=it.dropwhile(lambda x: x is 0, a) Usa lambda x: x==0 or simply lambda x: x Using 'is' relies on the interpreter reusing existing instances of the immutable object '0', which is an implementation detail. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
[ANN] PyInstaller 1.1 released
PyInstaller 1.1 *** PyInstaller 1.1 has been released! This release has been composed also with the help of patches provided by the community: many thanks to everybody! Find the downloads at the usual locations: http://pyinstaller.hpcf.upr.edu (website) http://pyinstaller.hpcf.upr.edu/source/1.1/pyinstaller_1.1.zip http://pyinstaller.hpcf.upr.edu/source/1.1/pyinstaller_1.1.tar.gz ChangeLog for this release: + (Windows) Make single-file packages not depend on MSVCRT71.DLL anymore, even under Python 2.4. You can eventually ship your programs really as single-file executables, even when using the newest Python version! + Fix problem with incorrect python path detection. Now using helpers from distutils. + Fix problem with rare encodings introduced in newer Python versions: now all the encodings are automatically found and included, so this problem should be gone forever. + Fix building of COM servers (was broken in 1.0 because of the new build system). + Mimic Python 2.4 behaviour with broken imports: sys.modules is cleaned up afterwise. This allows to package SQLObject applications under Windows with Python 2.4 and above. + Add import hook for the following packages: + GTK + PyOpenGL (tested 2.0.1.09) + dsnpython (tested 1.3.4) + KInterasDB (courtesy of Eugene Prigorodov) + Fix packaging of code using "time.strptime" under Python 2.3+. + (Linux) Ignore linux-gate.so while calculating dependencies (fix provided by Vikram Aggarwal). + (Windows) With Python 2.4, setup UPX properly so to be able to compress binaries generated with Visual Studio .NET 2003 (such as most of the extensions). UPX 1.92+ is needed for this. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Single-file executables
Ravi Teja wrote: > Actually, py2exe (from version 0.6.1) onwards does create single file > exes without ANY additional dependencies. They still require an external MSVCR71.DLL if made with Python 2.4. PyInstaller 1.1 (to be released today) does not have this problem. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: numeric expression from string?
Brian Blais wrote:
> I have a string input from the user, and want to parse it to a
> number, and would like to know how to do it. I would like to be able
> to accept arithmetic operations, like:
>
> '5+5'
> '(4+3)*2'
> '5e3/10**3'
>
> I thought of using eval, which will work, but could lead to bad
> security problems (not that it's a big deal in my app, but still...)
eval() is the preferred way unless you have serious security reasons:
>>> def calc(s):
... try:
... return float(eval(s, dict(__builtins__=None)))
... except Exception, e:
... raise ValueError, "error during expression evaluation: %s" % e
...
>>> calc("3**4")
81.0
>>> calc("58+34*4")
194.0
>>> calc("58+34*4+a")
Traceback (most recent call last):
File "", line 1, in ?
File "", line 5, in calc
ValueError: error during expression evaluation: name 'a' is not defined
>>> calc("object.__class__")
Traceback (most recent call last):
File "", line 1, in ?
File "", line 5, in calc
ValueError: error during expression evaluation: name 'object' is not defined
>>> calc("3.__class__")
Traceback (most recent call last):
File "", line 1, in ?
File "", line 5, in calc
ValueError: error during expression evaluation: unexpected EOF while parsing
(line 1)
>>> calc("type(3).__class__")
Traceback (most recent call last):
File "", line 1, in ?
File "", line 5, in calc
ValueError: error during expression evaluation: name 'type' is not defined
Of course, one can still bring your system to its knees when
"1000**1000"...
--
Giovanni Bajo
--
http://mail.python.org/mailman/listinfo/python-list
Re: fairly large webapp: from Java to Python. experiences?
John M. Gabriele wrote:
>> But once it is
>> there, Python is a good choice for web apps. Java is slow
>
> Slow? They're both dynamic languages, but Java is statically
> typed (with less work to do at runtime). For long-running processes,
> I'd guess that Java bytecode executes faster than Python bytecode.
It's not the raw computing performance that counts in this case. I got this
joke in my mail today:
Python:
print "%10.2f" % x
Java:
java.text.NumberFormat formatter = java.text.NumberFormat.getNumberInstance();
formatter.setMinimumFractionDigits(2);
formatter.setMaximumFractionDigits(2);
String s = formatter.format(x);
for (int i = s.length(); i < 10; i++) System.out.print(' ');
System.out.print(s);
Let alone the time it takes to write this routine, I'm hundered percent sure
that the Python's version is also faster at runtime. Python lets you write
pretty expressive code which is easy to maintain and where the computation cost
is easily at the C level. Also Python code is pretty bare-metal, so that
file.write or socket.write go to the syscall immediately. Try that in Java and
you'll find 30 layers of complex abstractions for doubtful benefits and obvious
slowness.
--
Giovanni Bajo
--
http://mail.python.org/mailman/listinfo/python-list
Re: Regular expression query
Martin Biddiscombe wrote:
> "parameter=12ab"
> "parameter=12ab foo bar"
> "parameter='12ab'"
> "parameter='12ab' biz boz"
> "parameter="12ab""
> "parameter="12ab" junk"
>>> import shlex
>>> def extract(s):
... s = s.split("=")[1]
... s = shlex.split(s)[0]
... return s
...
>>> extract("parameter=12ab")
'12ab'
>>> extract("parameter=12ab foo bar")
'12ab'
>>> extract("parameter='12ab'")
'12ab'
>>> extract("parameter='12ab' biz boz")
'12ab'
>>> extract('parameter="12ab"')
'12ab'
>>> extract('parameter="12ab" junk')
'12ab'
--
Giovanni Bajo
--
http://mail.python.org/mailman/listinfo/python-list
Re: Deterministic destruction and RAII idioms in Python
[EMAIL PROTECTED] wrote:
> Giving up deterministic destruction in Python would be a real blow for
> me, since it is one of its unique features among GC'ed languages.
>
> So what's the deal, can I rely on it in "mainstream" Python or am
> I out of luck here?
Most people rely on that. I do that *all* the time, and I would really dislike
if CPython lose this feature. Anyway, if you want to make your code portable to
other Python implementations, you'll have to implement your RAII idioms in a
slightly different way. Specifically, Python supports a try/finally construct
that it is useful to properly release resources. Eg. the following code is
"correct" even with IronPython:
f = file(name):
try:
for L in f:
print L
finally:
f.close()
Anyway, there is some work being done to add some RAII constructs to the
language that are compatibile across Python implementations. See the new "with"
statement. http://www.python.org/peps/pep-0343.html. This would allow to write
generic "functors" for RAII using generators, such as:
@contextmanager
def opened(filename):
f = open(filename)
try:
yield f
finally:
f.close()
These building blocks can then be used in the new 'with' statement such as:
with opened("foo.txt") as f:
for L in f:
print L
Of course, there's more to this (eg, 'with' statement can be used also with
normal objects as long as they support new __enter__ and __exit__ special
method), so go to the link above for more details and more juicy examples that
I'm sure you'll like.
--
Giovanni Bajo
--
http://mail.python.org/mailman/listinfo/python-list
Re: Codec Search Function
James Stroud wrote:
> How do I get the SVN version? Maybe that is not the one I have.
svn export svn://pyinstaller.hpcf.upr.edu/trunk pyinstaller
> This
> crashes on the version I have (complete program listing):
>
> "abcde".encode('hex')
>
> LookupError: no codec search functions registered: can't find encoding
I tried it and it works for me with a default spec file. Try again with latest
PyInstaller from SVN, or otherwise please enter the "buildX" directory made
by PyInstaller through the build process, and execute "ArchiveViewer.py
out1.pyz" (ArchiveViewer.py is shipped with Python). It will display a list of
all the packaged modules: in my list, 'encodings.hex_codec' appears.
--
Giovanni Bajo
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to handle two-level option processing with optparse
R. Bernstein wrote: > It seems that with all of the flexibility of optparse it should handle > this. I'm not sure right now what the best way to do so would be > though. Suggestions? If you call OptionParser.disable_interspersed_args() on your parser, it will stop parsing at the first positional argument, leaving other options unparsed. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Codec Search Function
James Stroud wrote: > I'm using pyinstaller 1.0 (stable) on win32xp and it is not able to > find the codec for several encodings (hex, base64, etc.). I resorted > to writing my own for hex, just to see if I could get my program > deployed. But I think a more permanent solution would be to get > pyinstaller working right. > > I tried the latest release (unstable version), claiming that it solved > the problem, but it had problems finding a search function: > > LookupError: no codec search functions registered: can't find encoding > > I looked at the documentation for codecs.register, and it was really > not clear exactly how I was to create the search function that needs > registering. Any help would be appreciated. Do you have a small reproducible sample which shows the problem? With PyInstaller SVN version, the problem appeared to be fixed for the testcases I had. BTW: putting "PyInstaller" in the subject of the mail for PyInstaller-related mails (and/or mailing to the PyInstaller mailing list) would be preferred to get a quicker answer. Thanks! -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Possible memory leak?
Fredrik Lundh wrote:
> since you know the length, you can preallocate the list
>
> def iters3(n):
> L = [None]*n
> for i in xrange(n):
> L[i] = chr(i%64)
> return "".join(L)
>
> or use a preallocated array
>
> def iters4(n):
> L = array.array("B", [0])*n
> for i in xrange(n):
> L[i] = i%64
> return L.tostring()
Of course. I was just trying to make a point about string accumulation being
O(n) and not O(n^2). You can get better factors with other solutions like the
ones you propose, but even the "simple" implementation (and most readable and
Pythonic, IMHO) behaves "well" under CPython 2.4.
--
Giovanni Bajo
--
http://mail.python.org/mailman/listinfo/python-list
Re: Possible memory leak?
Steven D'Aprano wrote: > No, that's not correct. You are making a false > assumption. Did you ever try to measure the code yourself? > This is from the "What's New" for Python 2.4: > > [quote] > String concatenations in statements of the form s = s + > "abc" and s += "abc" are now performed more efficiently > in certain circumstances. This optimization won't be > present in other Python implementations such as Jython, > so you shouldn't rely on it; using the join() method of > strings is still recommended when you want to > efficiently glue a large number of strings together. > [end quote] > > Note the "more efficiently in CERTAIN CIRCUMSTANCES" > [emphasis added]? That certainly does not mean that > concatenating large numbers of strings is no longer > slow. It just means that *sometimes* (almost always? > often? occasionally?) it isn't as slow as it used to be. > > We really don't know what the optimization recognises, > how it works, or how fast a difference it makes. > Writing poor code, and trusting an implementation- > specific optimization to make it run "faster" (how much > faster?) is always a dangerous strategy. The code is poor by your definition of poor. In my definition, it used to be poor and it's not anymore. Since I was interested in exactly WHICH circumstances the optimization was performed, I investigated and I know the answer. I also know that the optimization *will* apply to the OP code. But maybe you want some numbers: --- foo.py - def iters(n): s = '' for i in xrange(n): s += chr(i%64) return s def iters2(n): L = [] for i in xrange(n): L.append(chr(i%64)) return "".join(L) --- foo.py - [D:\Work]python -m timeit -n100 -s "import foo" "foo.iters(1)" 100 loops, best of 3: 10.2 msec per loop [D:\Work]python -m timeit -n100 -s "import foo" "foo.iters(2)" 100 loops, best of 3: 20.5 msec per loop [D:\Work]python -m timeit -n100 -s "import foo" "foo.iters(4)" 100 loops, best of 3: 40.9 msec per loop [D:\Work]python -m timeit -n100 -s "import foo" "foo.iters(8)" 100 loops, best of 3: 81.6 msec per loop [D:\Work]python -m timeit -n100 -s "import foo" "foo.iters2(1)" 100 loops, best of 3: 10.9 msec per loop [D:\Work]python -m timeit -n100 -s "import foo" "foo.iters2(2)" 100 loops, best of 3: 22.2 msec per loop [D:\Work]python -m timeit -n100 -s "import foo" "foo.iters2(4)" 100 loops, best of 3: 44 msec per loop [D:\Work]python -m timeit -n100 -s "import foo" "foo.iters2(8)" 100 loops, best of 3: 89.9 msec per loop So, look, it's even faster than the solution you're proposing. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Possible memory leak?
Steven D'Aprano wrote: > But the real killer is this one line: > > row=row+chr(num/64) > > Bad, bad BAD idea. Every time you add two strings together, Python > has to copy BOTH strings. As row gets huge, this takes longer and > longer to do. This is no longer true as of CPython 2.4 though. I'm not sure which version the OP was using because he didn't say. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Using non-ascii symbols
Robert Kern wrote: >> I can't find "?, ?, or ?" on my keyboard. > > Get a better keyboard? or OS? > > On OS X, > > ? is Alt-, > ? is Alt-. > ? is Alt-= > > Fewer keystrokes than <= or >= or !=. Sure, but I can't find OS X listed as a prerequisite for using Python. So, while I don't give a damn if those symbols are going to be supported by Python, I don't think the plain ASCII version should be deprecated. There are too many situations where it's still useful (coding across old terminals and whatnot). -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Backreferences in python ?
Pankaj wrote:
>>>> Perl :::
> ***
> while( )
> {
>
> line = $_;
>
> pattern = "printf\( \"$lineNo \" \),";
>
> line =~ s/"for(.*)\((*.)\;(.*)/for$1\($pattern$2\;$3/g;
> }
>
> This is used to
>
> search for :for ( i = 0; i < 10; i++)
> Replace with: for( printf( "10" ), i =0; i < 10; i++)
> Where 10 is the line no.
import re
import fileinput
for L in fileinput.input(inplace=True):
pattern = 'printf("%d"),' % input.filelineno()
L = re.sub(r"for(.*)\((*.)\;(.*)", r"for\1\(%s\2;\3" % pattern, L)
print L,
or something
--
Giovanni Bajo
--
http://mail.python.org/mailman/listinfo/python-list
Re: Some thougts on cartesian products
Christoph Zwerschke wrote: > Sometimes I was missing such a feature. > What I expect as the result is the "cartesian product" of the strings. I've been thinking of it as well. I'd like it for lists too: >> range(3)**2 [(0,0), (0,1), (0,2), (1,0), (1,1), (1,2), (2,0), (2,1), (2,2)] -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Returning a tuple-struct
[EMAIL PROTECTED] wrote:
>>>> time.localtime()
> (2006, 1, 18, 21, 15, 11, 2, 18, 0)
>>>> time.localtime()[3]
> 21
>>>> time.localtime().tm_hour
> 21
>
> Anyway, I guess there's a few of ways to do this. In the case above,
> it would seem reasonable to override __getitem__() and other things to
> get that result.
I have a generic solution for this (never submitted to the cookbook... should
I?)
import operator
def NamedTuple(*args, **kwargs):
class named_tuple_class(tuple):
pass
values = []
idx = 0
for arg in args:
for name in arg[:-1]:
setattr(named_tuple_class, name,
property(operator.itemgetter(idx)))
values.append(arg[-1])
idx += 1
for name,val in kwargs.iteritems():
setattr(named_tuple_class, name, property(operator.itemgetter(idx)))
values.append(val)
idx += 1
return named_tuple_class(values)
>>> t = NamedTuple(("x", 12), ("y", 18))
>>> t
(12, 18)
>>> t[0]
12
>>> t.x
12
>>> t[1]
18
>>> t.y
18
>>> t.z
Traceback (most recent call last):
File "", line 1, in ?
AttributeError: 'named_tuple_class' object has no attribute 'z'
>>> t = NamedTuple(("p", "pos", "position", 12.4))
>>> t
(12.4,)
>>> t.p
12.4
>>> t.pos
12.4
>>> t.position
12.4
>>> t = NamedTuple(("p", "pos", 12.4), length="foo")
>>> t
(12.4, 'foo')
>>> t.p
12.4
>>> t.pos
12.4
>>> t.length
'foo'
--
Giovanni Bajo
--
http://mail.python.org/mailman/listinfo/python-list
Re: getopt.gnu_getopt: incorrect documentation
Steve Holden wrote: >> The official documentation for "getopt.gnu_getopt" does not mention >> the version number in which it was introduced (so I assumed it was >> introduced back when getopt was added). This is wrong, though: I was >> informed that Python 2.2 does not have this function, and a quick >> google search turned up that its addition was mentioned in Python >> 2.3 release notes. >> >> This should be fixed in the documentation. >> >> Thanks! > > Giovanni: > > Sadly this post is unlikely to change anything. Can you please email > the > address shown in the documentation ([EMAIL PROTECTED]), or by using the > Python bug tracker? Sure, I'll use e-mail. My previous attempt at using the Python bug tracker was a failure (totally ignored after years) so I'm keen on trying some other way. Thanks! -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
getopt.gnu_getopt: incorrect documentation
Hello, The official documentation for "getopt.gnu_getopt" does not mention the version number in which it was introduced (so I assumed it was introduced back when getopt was added). This is wrong, though: I was informed that Python 2.2 does not have this function, and a quick google search turned up that its addition was mentioned in Python 2.3 release notes. This should be fixed in the documentation. Thanks! -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: list(...) and list comprehensions (WAS: Arithmetic sequences in Python)
Diez B. Roggisch wrote: >> due to the nested parentheses. Note that replacing list comprehensions >> with list(...) doesn't introduce any nested parentheses; it basically >> just replaces brackets with parentheses. > > But you don't need the nested parentheses - use *args instead for the > list-constructor. > > list(a,b,c) No, you can't. That's ambigous if you pass only one argument, and that argument is iterable. This is also the reason why set() doesn't work this way. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: magical expanding hash
braver wrote: > Also, what's the shortest python idiom for get_or_set in expression? dict.setdefault, as I already explained to you. Again, I'd like to point out that what you're doing is *not* the correct Pythonic way of doing things. In Python, there is simply no implicit sub-dicts creation, nor implicit type inference from operators. And there are very good reason for that. Python is a strongly typed languages: objects have a type and keep it, they don't change it when used with different operators. setdefault() is you get'n'set, everything else has to be made explicit for a good reason. Strong typing has its virtues, let me give you a link about this: http://wingware.com/python/success/astra See specifically the paragraph "Python's Error Handling Improves Robustness" I believe you're attacking the problem from a very bad point of view. Instead of trying to write a Python data structure which behaves like Perl's, convert a Perl code snippet into Python, using the *Pythonic* way of doing it, and then compare things. Don't try to write Perl in Python, just write Python and then compare the differences. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: magical expanding hash
James Stroud wrote:
>> I need a magical expanding hash with the following properties:
>>
>> * it creates all intermediate keys
>>
>> meh['foo']['bar] = 1
>>
>> -- works even if meh['foo'] didn't exist before
>>
>> * allows pushing new elements to leaves which are arrays
>>
>> meh['foo']['list] << elem1
>> meh['foo']['list] << elem2
>>
>> * allows incrementing numeric leaves
>>
>> meh['foo']['count'] += 7
>>
>> * serializable
>>
>> I have such a class in ruby. Can python do that?
>>
>
>
> Is this too magical?
>
>
> class meh(dict):
>def __getitem__(self, item):
> if self.has_key(item):
>return dict.__getitem__(self, item)
> else:
>anitem = meh()
>dict.__setitem__(self, item, anitem)
> return anitem
Actually what the OP wants is already a method of dict, it's called
setdefault(). It's not overloaded by "[]" because it's believed to be better to
be able to say "I want auto-generation" explicitally rather than implicitly: it
gives the user more power to control, and to enforce stricter rules.
>>> class meh(dict):
... def __getitem__(self, item):
... return dict.setdefault(self, item, meh())
...
>>> a = meh()
>>> a["foo"]["bar"] = 2
>>> a["foo"]["dup"] = 3
>>> print a["foo"]["bar"]
2
>>> print a
{'foo': {'dup': 3, 'bar': 2}}
So I advise using this class, and suggest the OP to try using setdefault()
explicitally to better understand Python's philosophy.
BTW: remember that setdefault() is written "setdefault()" but it's read
"getorset()".
--
Giovanni Bajo
--
http://mail.python.org/mailman/listinfo/python-list
Re: Shrinky-dink Python (also, non-Unicode Python build is broken)
Neil Hodgson wrote: >> - There will be some opposition to the obvious policy of "keeping >> the bare minimum inside the DLL" because of inefficiencies in the >> Python build system. > > It is also non-optimal for those that do want the full set of > modules as separate files can add overhead for block sizing (both on > disk and in memory, executables pad out each section to some block > size), by requiring more load-time inter-module fixups I would be surprised if this showed up in any profile. Importing modules can already be slow no matter external stats (see programs like "mercurial" that, to win benchmarks with C-compiled counterparts, do lazy imports). As for the overhead at the border of blocks, you should be more worried with 800K of CJK codecs being loaded in your virtual memory (and not fully swapped out because of block sizing) which are totally useless for most applications. Anyway, we're picking nits here, but you have a point in being worried. If I ever write a PEP, I will produce numbers to show beyond any doubt that there is no performance difference. > , and by not > allowing the linker to perform some optimizations. It'd be worthwhile > seeing if the DLL would speed up or shrink if whole program > optimization was turned on. There is no way whole program optimization can produce any advantage as the modules are totally separated and they don't have direct calls that the compiler can exploit. -- Giovanni Bajo -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing files -- pyparsing to the rescue?
rh0dium wrote:
> I have a file which I need to parse and I need to be able to break it
> down by sections. I know it's possible but I can't seem to figure this
> out.
>
> The sections are broken by <> with one or more keywords in the <>.
> What I want to do is to be able to pars a particular section of the
> file. So for example I need to be able to look at the SYSLIB section.
> Presumably the sections are
>
>
>
> Sys Data
> Sys-Data
> asdkData
> Data
>
> Data
> Data
> Data
> Data
>
> Data
> Data
> Data
> Data
>
> Data
> Data
> Data
> Data
>
Given your description, pyparsing doesn't feel like the correct tool:
secs = {}
for L in file("foo.txt", "rU"):
L = L.rstrip("\n")
if re.match(r"<.*>", L):
name = L[1:-1]
secs[name] = []
else:
secs[name].append(L)
--
Giovanni Bajo
--
http://mail.python.org/mailman/listinfo/python-list
