Re: [Python-Dev] Sharing expat instances
Trent Mick wrote: > Okay, done. I naively added "_elementtree" to the extensions list in > msi.py. I'm guessing that that was all that was necessary. Right. Thanks! Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
[Martin v. Loewis wrote] > Trent Mick wrote: > > C:\trentm\src\python\python\PCbuild>svn st > > A _elementtree.vcproj > > M pcbuild.sln > > > > Shall I checkin the build changes? > > Go ahead. A change to Tools/msi/msi.py is also in order. Okay, done. I naively added "_elementtree" to the extensions list in msi.py. I'm guessing that that was all that was necessary. Trent -- Trent Mick [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
Trent Mick wrote:
> [Fredrik Lundh wrote]
> > can anyone with a working windows setup look at building the
> > _elementtree module for windows ?
> >
> > (see the setup.py file for build options; it wants the same defines
> > as pyexpat, plus one extra define.).
>
> C:\trentm\src\python\python\PCbuild>python
> Python 2.5a0 (#60, Dec 15 2005, 11:17:15) [MSC v.1310 32 bit (Intel)] on
> win32
> Type "help", "copyright", "credits" or "license" for more information.
> >>> from _elementtree import Element, ElementTree
> >>> root = Element("root")
> >>> root.append(Element("one"))
> >>> tree = ElementTree(root)
> >>> import sys
> >>> tree.write(sys.stdout)
> >>>
most excellent!
> C:\trentm\src\python\python\PCbuild>svn st
> A _elementtree.vcproj
> M pcbuild.sln
>
> Shall I checkin the build changes?
I sure won't stop you.
thanks! /F
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
At 09:30 PM 12/15/2005 +0100, Martin v. Löwis wrote: >Phillip J. Eby wrote: > > I don't know. I can see that the split makes sense for prefix/exec-prefix > > distinctions, but then again, the disutils will install an entire > > distribution in exec-prefix if it contains "impure" parts, so that's > > certainly an option here. > > > > On the other hand, it's not clear to me *why* the lib-dynload/DLLs > > directories exist, since it seems to me that that's what exec-prefix is > > for. > >Can you please explain? exec_prefix will point to, say, >/usr/i686; it shouldn't be that .so files are directly installed in >that location. Instead, Python searches them in >EXEC_PREFIX "/lib/python" VERSION "/lib-dynload". Right; the question is why not just EXEC_PREFIX "/lib/python" VERSION instead. What benefit does the separate directory offer? Note that the distutils, when installing a package containing C extensions, will install to site-packages under sys.exec_prefix; it does not separate the C extensions into special alternate library directories. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
Trent Mick wrote: > C:\trentm\src\python\python\PCbuild>svn st > A _elementtree.vcproj > M pcbuild.sln > > Shall I checkin the build changes? Go ahead. A change to Tools/msi/msi.py is also in order. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
[Fredrik Lundh wrote]
> can anyone with a working windows setup look at building the
> _elementtree module for windows ?
>
> (see the setup.py file for build options; it wants the same defines
> as pyexpat, plus one extra define.).
C:\trentm\src\python\python\PCbuild>python
Python 2.5a0 (#60, Dec 15 2005, 11:17:15) [MSC v.1310 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from _elementtree import Element, ElementTree
>>> root = Element("root")
>>> root.append(Element("one"))
>>> tree = ElementTree(root)
>>> import sys
>>> tree.write(sys.stdout)
>>>
C:\trentm\src\python\python\PCbuild>svn st
A _elementtree.vcproj
M pcbuild.sln
Shall I checkin the build changes?
Trent
--
Trent Mick
[EMAIL PROTECTED]
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
Phillip J. Eby wrote: > I don't know. I can see that the split makes sense for prefix/exec-prefix > distinctions, but then again, the disutils will install an entire > distribution in exec-prefix if it contains "impure" parts, so that's > certainly an option here. > > On the other hand, it's not clear to me *why* the lib-dynload/DLLs > directories exist, since it seems to me that that's what exec-prefix is > for. Can you please explain? exec_prefix will point to, say, /usr/i686; it shouldn't be that .so files are directly installed in that location. Instead, Python searches them in EXEC_PREFIX "/lib/python" VERSION "/lib-dynload". > Perhaps somebody can explain why lib-dynload/ and DLLs/ > exist? To have a directory on sys.path where native modules are found. > Perhaps some platforms have to add these directories to some > godforsaken environment variables like LD_LIBRARY_PATH or something? Not to my knowledge, no. lib-dynload was introduced in revision 8976, where it was renamed from "/sharedmodules". This, in turn, was introduced into getpath.c in revision 7775 (and 7776). It was added to Modules/Setup.in in revision 6313, and to Makefile.in in 6321. Unfortunately, the checkin message of 6321 only says More changes to install targets. The notion of a separate makefile variable for shared libraries goes back to Modules/[EMAIL PROTECTED], which first introduced dynamic loading (in 1994). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
I wrote: > my current idea is to > > 1. include it under a different name (_elementtree.so) > > 2. add a cElementTree.py under xml.etree, which simply does > > from _elementtree import * I've implemented this, for now. can anyone with a working windows setup look at building the _elementtree module for windows ? (see the setup.py file for build options; it wants the same defines as pyexpat, plus one extra define.). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
At 02:19 AM 12/15/2005 +, Steve Holden wrote: >Scott David Daniels wrote: > > Phillip J. Eby wrote: > > > >>At 05:51 PM 12/14/2005 +0100, Fredrik Lundh wrote: > >> > >>>Phillip J. Eby wrote. > >>> > >>> > >my current idea is to > > > >1. include it under a different name (_elementtree.so) > > > >2. add a cElementTree.py under xml.etree, which simply does > > > >from _elementtree import * > > > >does anyone have a better idea ? > > I was under the impression that simply installing cElementTree.so in the > relevant package directory would work; this is what the distutils do for > extensions with a package name. > >>> > >>>it would work, of course, but the core puts all the binaries in a separate > >>>directory (lib-dynload on unix, DLLs on windows, etc). > >>> > >>>do we really want to put executables in other locations ? > >> > >>I don't know. I can see that the split makes sense for prefix/exec-prefix > >>distinctions, but then again, the disutils will install an entire > >>distribution in exec-prefix if it contains "impure" parts, so that's > >>certainly an option here. > >> > >>On the other hand, it's not clear to me *why* the lib-dynload/DLLs > >>directories exist, since it seems to me that that's what exec-prefix is > >>for. Perhaps somebody can explain why lib-dynload/ and DLLs/ > >>exist? Perhaps some platforms have to add these directories to some > >>godforsaken environment variables like LD_LIBRARY_PATH or something? > > > > > > What I believe I understand about /.pyd / .so / .dll / shared libraries > > is that they are meant to allow several processes to map the same disk > > backing store to the same same virtual address for more than a single > > process. If the .egg strategy is followed, I expect that either the > > file shared is in a user(or even process)-specific location or there > > is a shared folder that is writable by many processes from which > > executable code can be run. The one solution reduces sharing, the > > other violates security principles. Actually, the discussion wasn't about eggs, but about the placement of C extensions in standard, "normal" package directories. The Python installation uses only top-level C extensions and puts them in separate directories. The question outstanding is, why? Since there already exists exec-prefix for architecture-specific files, and the distutils installs an entire package to either prefix or exec-prefix depending upon its "purity". I suspect that the reason is legacy: originally packages didn't exist, and then even when they did, the distutils didn't. Most of the stdlib is developed in-place without benefit of the distutils, so it makes sense historically that the modules would be where they are, especially since the distutils need some of those modules to run. However, this alone wouldn't be a reason not to build and install ElementTree using the distutils. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
Scott David Daniels wrote: > Phillip J. Eby wrote: > >>At 05:51 PM 12/14/2005 +0100, Fredrik Lundh wrote: >> >>>Phillip J. Eby wrote. >>> >>> >my current idea is to > >1. include it under a different name (_elementtree.so) > >2. add a cElementTree.py under xml.etree, which simply does > >from _elementtree import * > >does anyone have a better idea ? I was under the impression that simply installing cElementTree.so in the relevant package directory would work; this is what the distutils do for extensions with a package name. >>> >>>it would work, of course, but the core puts all the binaries in a separate >>>directory (lib-dynload on unix, DLLs on windows, etc). >>> >>>do we really want to put executables in other locations ? >> >>I don't know. I can see that the split makes sense for prefix/exec-prefix >>distinctions, but then again, the disutils will install an entire >>distribution in exec-prefix if it contains "impure" parts, so that's >>certainly an option here. >> >>On the other hand, it's not clear to me *why* the lib-dynload/DLLs >>directories exist, since it seems to me that that's what exec-prefix is >>for. Perhaps somebody can explain why lib-dynload/ and DLLs/ >>exist? Perhaps some platforms have to add these directories to some >>godforsaken environment variables like LD_LIBRARY_PATH or something? > > > What I believe I understand about /.pyd / .so / .dll / shared libraries > is that they are meant to allow several processes to map the same disk > backing store to the same same virtual address for more than a single > process. If the .egg strategy is followed, I expect that either the > file shared is in a user(or even process)-specific location or there > is a shared folder that is writable by many processes from which > executable code can be run. The one solution reduces sharing, the > other violates security principles. > When extension modules are to be run from zip files it's difficult to avoid a reduction in sharing anyway - without some kind of additional mechanism there's no guarantee that two modules with the same filename are actually the same library anyway, and there's every likelihood that over time several zipped applications will be released containing different versions of the same extension module. In that case each app would want the version that came with it anyway, no? There are slightly different requirements for packaged distribution than for the standard interpreter, which I assume can always rely on having an installed module available as a file (though this might not be desirable in the future). regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
On Dec 14, 2005, at 5:31 PM, Alex Martelli wrote:
> On 12/14/05, Chris Lambacher <[EMAIL PROTECTED]> wrote:
>> Py2exe manages to load .pyd files and dlls from zip. Apparently
>> they have
>> written an alternate dll loader that does not need the file to be
>> on the file
>> system. This is used for single file apps.
>>
>> I don't know if it is possible to write a portable Unix equivalent
>> for .so
>> files.
>
> If you mean, portable to all dialects of Unix and Unix-like systems, I
> am pretty confident in answering "no". But it might be possible to
> support many such systems, e.g. by exploiting the DMG format ("disk
> image" within a single file), native to MacOSX, that can be used in
> Linux (and I believe in some other Unix variants) by mounting the file
> as a "loop device" (there's all sort of niggling issues, such as the
> need to be root to do so, but perhaps there might be workarounds...).
For Mac OS X 10.4+, you don't even have to bother. The dynamic
linker finally implements the function that lets you load a code
bundle straight from RAM:
extern NSObjectFileImageReturnCode NSCreateObjectFileImageFromMemory(
const void *address,
size_t size,
NSObjectFileImage *objectFileImage);
Of course, you could always just use temporary files or a cache
somewhere.
-bob
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
On 12/14/05, Chris Lambacher <[EMAIL PROTECTED]> wrote:
> Py2exe manages to load .pyd files and dlls from zip. Apparently they have
> written an alternate dll loader that does not need the file to be on the file
> system. This is used for single file apps.
>
> I don't know if it is possible to write a portable Unix equivalent for .so
> files.
If you mean, portable to all dialects of Unix and Unix-like systems, I
am pretty confident in answering "no". But it might be possible to
support many such systems, e.g. by exploiting the DMG format ("disk
image" within a single file), native to MacOSX, that can be used in
Linux (and I believe in some other Unix variants) by mounting the file
as a "loop device" (there's all sort of niggling issues, such as the
need to be root to do so, but perhaps there might be workarounds...).
Alex
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
Py2exe manages to load .pyd files and dlls from zip. Apparently they have written an alternate dll loader that does not need the file to be on the file system. This is used for single file apps. I don't know if it is possible to write a portable Unix equivalent for .so files. -Chris P.S. I think it is standard practice to do an introduction on first post to pydev. I have been lurking on the Dev list for about a year. I recently completed my masters thesis which used Python extensively. I am helping to promote the adoption of Python for various tasks at my place of work. On Wed, Dec 14, 2005 at 01:15:49PM -0500, Barry Warsaw wrote: > On Wed, 2005-12-14 at 09:56 -0800, Scott David Daniels wrote: > > > One good reason for this is that the .pyd's or .so's cannot necessarily > > be used from zip files > > When you say "cannot necessarily", are the situations where they can be > imported from zip files? I thought the answer to that was always "no". > > -Barry > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/lambacck%40computer.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
Phillip J. Eby wrote: > At 05:51 PM 12/14/2005 +0100, Fredrik Lundh wrote: >> Phillip J. Eby wrote. >> my current idea is to 1. include it under a different name (_elementtree.so) 2. add a cElementTree.py under xml.etree, which simply does from _elementtree import * does anyone have a better idea ? >>> I was under the impression that simply installing cElementTree.so in the >>> relevant package directory would work; this is what the distutils do for >>> extensions with a package name. >> it would work, of course, but the core puts all the binaries in a separate >> directory (lib-dynload on unix, DLLs on windows, etc). >> >> do we really want to put executables in other locations ? > > I don't know. I can see that the split makes sense for prefix/exec-prefix > distinctions, but then again, the disutils will install an entire > distribution in exec-prefix if it contains "impure" parts, so that's > certainly an option here. > > On the other hand, it's not clear to me *why* the lib-dynload/DLLs > directories exist, since it seems to me that that's what exec-prefix is > for. Perhaps somebody can explain why lib-dynload/ and DLLs/ > exist? Perhaps some platforms have to add these directories to some > godforsaken environment variables like LD_LIBRARY_PATH or something? What I believe I understand about /.pyd / .so / .dll / shared libraries is that they are meant to allow several processes to map the same disk backing store to the same same virtual address for more than a single process. If the .egg strategy is followed, I expect that either the file shared is in a user(or even process)-specific location or there is a shared folder that is writable by many processes from which executable code can be run. The one solution reduces sharing, the other violates security principles. --Scott David Daniels [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
At 05:51 PM 12/14/2005 +0100, Fredrik Lundh wrote: >Phillip J. Eby wrote. > > > >my current idea is to > > > > > > 1. include it under a different name (_elementtree.so) > > > > > > 2. add a cElementTree.py under xml.etree, which simply does > > > > > > from _elementtree import * > > > > > >does anyone have a better idea ? > > > > I was under the impression that simply installing cElementTree.so in the > > relevant package directory would work; this is what the distutils do for > > extensions with a package name. > >it would work, of course, but the core puts all the binaries in a separate >directory (lib-dynload on unix, DLLs on windows, etc). > >do we really want to put executables in other locations ? I don't know. I can see that the split makes sense for prefix/exec-prefix distinctions, but then again, the disutils will install an entire distribution in exec-prefix if it contains "impure" parts, so that's certainly an option here. On the other hand, it's not clear to me *why* the lib-dynload/DLLs directories exist, since it seems to me that that's what exec-prefix is for. Perhaps somebody can explain why lib-dynload/ and DLLs/ exist? Perhaps some platforms have to add these directories to some godforsaken environment variables like LD_LIBRARY_PATH or something? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
At 09:56 AM 12/14/2005 -0800, Scott David Daniels wrote: >My (admittedly weak) understanding of how packages work is that all >parts of a package should lie off the same node of the PYTHONPATH. This isn't a requirement; packages have a __path__ attribute which can include more than one directory. The 'pkgutil' module (added in Python 2.3) even has a convenient way to merge parts of a package that are in separate directories. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
At 08:21 PM 12/14/2005 +0100, Fredrik Lundh wrote: >Scott David Daniels wrote: > > > > > One good reason for this is that the .pyd's or .so's cannot necessarily > > > > be used from zip files > > > > > > When you say "cannot necessarily", are the situations where they can be > > > imported from zip files? I thought the answer to that was always "no". > > > > I thought so too, but was not sure enough to state it that way. > >you could of course add them to the zip file, and automagically extract >them before you start importing things. The runtime system for Python Eggs does this; extraction is to a PYTHON_EGG_CACHE directory. See e.g.: http://peak.telecommunity.com/DevCenter/PkgResources#resource-extraction The egg builder writes a .py file alongside the .pyd or .so in the .egg file that contains something like: def __bootstrap__(): global __bootstrap__, __loader__, __file__ import sys, pkg_resources, imp __file__ = pkg_resources.resource_filename(__name__,'foobar.so') del __bootstrap__, __loader__ imp.load_dynamic(__name__,__file__) __bootstrap__() So, when you import from the zipfile, the .py file gets loaded (since zipimport doesn't support .pyd/.so/etc. imports directly) and then it reloads the module from the extracted file. The other magic there is just to keep any of the special names from staying behind in the reloaded module. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
On Wed, 2005-12-14 at 20:21 +0100, Fredrik Lundh wrote: > you could of course add them to the zip file, and automagically extract > them before you start importing things. Sure (although we don't). I wonder if this is useful functionality for the core. -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
Scott David Daniels wrote: > > > One good reason for this is that the .pyd's or .so's cannot necessarily > > > be used from zip files > > > > When you say "cannot necessarily", are the situations where they can be > > imported from zip files? I thought the answer to that was always "no". > > I thought so too, but was not sure enough to state it that way. you could of course add them to the zip file, and automagically extract them before you start importing things. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
Barry Warsaw wrote: > On Wed, 2005-12-14 at 09:56 -0800, Scott David Daniels wrote: >> One good reason for this is that the .pyd's or .so's cannot necessarily >> be used from zip files > When you say "cannot necessarily", are the situations where they can be > imported from zip files? I thought the answer to that was always "no". I thought so too, but was not sure enough to state it that way. --Scott David Daniels [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
On Wed, 2005-12-14 at 09:56 -0800, Scott David Daniels wrote: > One good reason for this is that the .pyd's or .so's cannot necessarily > be used from zip files When you say "cannot necessarily", are the situations where they can be imported from zip files? I thought the answer to that was always "no". -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
Fredrik Lundh wrote: > Phillip J. Eby wrote. > >>> my current idea is to >>> >>> 1. include it under a different name (_elementtree.so) >>> 2. add a cElementTree.py under xml.etree, which simply does >>> from _elementtree import * >>> >>> does anyone have a better idea ? >> I was under the impression that simply installing cElementTree.so in the >> relevant package directory would work; this is what the distutils do for >> extensions with a package name. > > it would work, of course, but the core puts all the binaries in a separate > directory (lib-dynload on unix, DLLs on windows, etc). One good reason for this is that the .pyd's or .so's cannot necessarily be used from zip files, so it is nice to keep those binaries out of the package directory (packages can then easily go in a Python25.zip). My (admittedly weak) understanding of how packages work is that all parts of a package should lie off the same node of the PYTHONPATH. --Scott David Daniels [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
Phillip J. Eby wrote. > >my current idea is to > > > > 1. include it under a different name (_elementtree.so) > > > > 2. add a cElementTree.py under xml.etree, which simply does > > > > from _elementtree import * > > > >does anyone have a better idea ? > > I was under the impression that simply installing cElementTree.so in the > relevant package directory would work; this is what the distutils do for > extensions with a package name. it would work, of course, but the core puts all the binaries in a separate directory (lib-dynload on unix, DLLs on windows, etc). do we really want to put executables in other locations ? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
On Wednesday 14 December 2005 09:51, Steve Holden wrote: > Which reminds me, what about the idea of reducing certain packages > (Carbon, dostutils and email come to mind) to just a single entry in the > global module index and adding a first-level TOC at the beginning of > that section of content? This echos a recent Doc-SIG post from Skip Montanaro that I've been meaning to reply to. I've now done that: http://mail.python.org/pipermail/doc-sig/2005-December/003453.html -Fred -- Fred L. Drake, Jr. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
Michael McLay wrote: > On Wednesday 14 December 2005 01:05, Fredrik Lundh wrote: > >>Michael McLay wrote: >> >>>Avoiding imaginaary name collisions and putting cElementTree into the xml >>>package >> >>there's nothing imaginary here -- cElementTree is an existing and quite >>popular module, and will remain available as a separate distribution. >> >>it would be nice if people could install that kit also under 2.5 without >>risking to mess up their Python installation. >> >>(another solution would of course to rule out use of cElementTree by >>modules shipped with Python...) > > > Renaming the cElementTree in the standard distribution to etree would > elmiinate collisions with the existing cElementTree. A few other names to > consider would be xmltree or xmlTree. Or, if the consensus is to keep it in > the xml package, the name could still be shorted to calling it xml.tree. > > Average name lengths in the standard distribution are getting longer. In some > cases the longer names are very helpful when browsing the module index. For > instance, the function of SimpleXMLRPCServer is immediately apparent. To me, > the purpose of a package named ElementTree isn't as apparent. While there is > value in having meaningful names, there is also an advantage in having names > short for commonly used tools. Imagine how painful it would be to type > Operatingsystem instead of os. The name xmltree would be shorter and as > descriptive as xml.cElementTree. The name etree would be shorter, but less > descriptive. Which reminds me, what about the idea of reducing certain packages (Carbon, dostutils and email come to mind) to just a single entry in the global module index and adding a first-level TOC at the beginning of that section of content? Even if we then have to call it the global package and module index! regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
On Wednesday 14 December 2005 01:05, Fredrik Lundh wrote: > Michael McLay wrote: > > Avoiding imaginaary name collisions and putting cElementTree into the xml > > package > > there's nothing imaginary here -- cElementTree is an existing and quite > popular module, and will remain available as a separate distribution. > > it would be nice if people could install that kit also under 2.5 without > risking to mess up their Python installation. > > (another solution would of course to rule out use of cElementTree by > modules shipped with Python...) Renaming the cElementTree in the standard distribution to etree would elmiinate collisions with the existing cElementTree. A few other names to consider would be xmltree or xmlTree. Or, if the consensus is to keep it in the xml package, the name could still be shorted to calling it xml.tree. Average name lengths in the standard distribution are getting longer. In some cases the longer names are very helpful when browsing the module index. For instance, the function of SimpleXMLRPCServer is immediately apparent. To me, the purpose of a package named ElementTree isn't as apparent. While there is value in having meaningful names, there is also an advantage in having names short for commonly used tools. Imagine how painful it would be to type Operatingsystem instead of os. The name xmltree would be shorter and as descriptive as xml.cElementTree. The name etree would be shorter, but less descriptive. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
Michael McLay wrote: > Avoiding imaginaary name collisions and putting cElementTree into the xml > package there's nothing imaginary here -- cElementTree is an existing and quite popular module, and will remain available as a separate distribution. it would be nice if people could install that kit also under 2.5 without risking to mess up their Python installation. (another solution would of course to rule out use of cElementTree by modules shipped with Python...) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
On Tuesday 13 December 2005 17:17, Fredrik Lundh wrote: > the remaining issue is how to include cElementTree. the current stand- > alone distribution consists of a single cElementTree module, which is in- > stalled under site-packages, as usual. > > to avoid collisions, it's probably best to install the bundled version > under xml.etree, but how do you do that for a C module ? > > my current idea is to > > 1. include it under a different name (_elementtree.so) > > 2. add a cElementTree.py under xml.etree, which simply does > > from _elementtree import * > > does anyone have a better idea ? Avoiding imaginaary name collisions and putting cElementTree into the xml package is making the task harder than need be and will perpetuate the problems caused by the earlier decision to do magic in the xml import statement. The use of magic was a violation of "explicit is better than implicit". Forgeting this rule has caused pain to many people who didn't expect a magic side effects from simply installing PyXML. Adding cElementTree to the mess that already exists doessn't make sense, and the only justification for not adding a new top level package seems to be to avoid an imaginary potential for name collisions. Why repeat the mistake of PyXML simply to have all the xml software in the same top level package? Perhaps if this were a common practice in the stdlib it might make sense. For instance, if there were a gui.Tkinter and a gui.PyGtk and a gui.wxPython it might look more consistent. Who is pushing to put all xml software inside the xml package? Collisions with names of a top level import statement are rare and Python has supports renaming in the rare event that someone did need to rename the module. import etree as stdetree. I would like the cElementTree version of the package have a shorter name. For example etree. The Python implementation could continue to be named ElementTree or shorten it to pyetree. The cElementTree version will probably be the most frequently used since it is faster, so why not make it's name short. One final addition would be to include a "See Also" reference to the new etree and pyetree modules in the introduction of the xml package. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
At 11:17 PM 12/13/2005 +0100, Fredrik Lundh wrote: >my current idea is to > > 1. include it under a different name (_elementtree.so) > > 2. add a cElementTree.py under xml.etree, which simply does > > from _elementtree import * > >does anyone have a better idea ? I was under the impression that simply installing cElementTree.so in the relevant package directory would work; this is what the distutils do for extensions with a package name. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
On 12/13/05, Fredrik Lundh <[EMAIL PROTECTED]> wrote: > I wrote: > > > 1. add an Include/pyexpat.h header file which contains a structure > > similar to the following: > > > 2. during pyexpat initialization, initialize all members of this structure, > > and > > make it available as a PyCObject: > > > 3. in cElementTree (or _elementtree, or whatever the python version will > > be named), import pyexpat, fetch the object, and verify > > > 4. in cElementTree (...), do all expat calls via the dispatch table. > > I've fixed all this, and checked in 1 and 2. > > the remaining issue is how to include cElementTree. the current stand- > alone distribution consists of a single cElementTree module, which is in- > stalled under site-packages, as usual. > > to avoid collisions, it's probably best to install the bundled version under > xml.etree, but how do you do that for a C module ? > > my current idea is to > > 1. include it under a different name (_elementtree.so) > > 2. add a cElementTree.py under xml.etree, which simply does > > from _elementtree import * > > does anyone have a better idea ? > Too bad you can't do 1 and for 2 add ``import _elementtree as cElementTree`` in etree/__init__.py . Unless I am missing something it won't work since ``import xml.etree.cElementTree`` will fail. Since the way you outlined is the standard way to do it in the stdlib I doubt anyone has thought of a better way. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
I wrote: > 1. add an Include/pyexpat.h header file which contains a structure > similar to the following: > 2. during pyexpat initialization, initialize all members of this structure, > and > make it available as a PyCObject: > 3. in cElementTree (or _elementtree, or whatever the python version will > be named), import pyexpat, fetch the object, and verify > 4. in cElementTree (...), do all expat calls via the dispatch table. I've fixed all this, and checked in 1 and 2. the remaining issue is how to include cElementTree. the current stand- alone distribution consists of a single cElementTree module, which is in- stalled under site-packages, as usual. to avoid collisions, it's probably best to install the bundled version under xml.etree, but how do you do that for a C module ? my current idea is to 1. include it under a different name (_elementtree.so) 2. add a cElementTree.py under xml.etree, which simply does from _elementtree import * does anyone have a better idea ? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
On Tuesday 13 December 2005 14:52, Fredrik Lundh wrote: > I take that as a "go ahead" ;-) Good call! :-) -Fred -- Fred L. Drake, Jr. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
Fredrik Lundh wrote: > comments ? As Fred says: go ahead. regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
Fred L. Drake, Jr. wrote: > > 1. add an Include/pyexpat.h header file which contains a structure > > similar to the following: > ... > > comments ? > > +1 I take that as a "go ahead" ;-) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sharing expat instances
On Tuesday 13 December 2005 11:22, Fredrik Lundh wrote: > here's a plan: > > 1. add an Include/pyexpat.h header file which contains a structure > similar to the following: ... > comments ? +1 -Fred -- Fred L. Drake, Jr. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Sharing expat instances
Martin v. Löwis wrote:
> > - sort out expat bundling issues, and include cElementTree as well
> > (using the same approach as above).
...
> > (one way to do this would be to add an "function pointer table" to pyexpat
> > that contains pointers to selected portions of the expat API, and then add
> > an indirection level to cElementTree)
>
> Ok, this sounds like a larger piece of work.
here's a plan:
1. add an Include/pyexpat.h header file which contains a structure
similar to the following:
#define PyExpat_DISPATCH_MAGIC "... some magic string ..."
struct PyExpat_Dispatch {
int size; /* size of this structure */
int MAJOR_VERSION;
int MINOR_VERSION;
int MICRO_VERSION;
... (*ErrorString)(...)
... (*GetErrorColumnNumber)(...)
... (*GetErrorLineNumber)(...)
... (*Parse)(...)
... (*ParserCreate_MM)(...)
... (*ParserFree)(...)
... (*SetCharacterDataHandler)(...)
... (*SetCommentHandler)(...)
... (*SetDefaultHandlerExpand)(...)
... (*SetElementHandler)(...)
... (*SetNamespaceDeclHandler)(...)
... (*SetProcessingInstructionHandler)(...)
... (*SetUserData)(...)
/* add new stuff to the end */
}
(this is the minimal stuff used by today's cElementTree; it can of course
be extended to cover a larger part of the current expat API)
2. during pyexpat initialization, initialize all members of this structure, and
make it available as a PyCObject:
static PyExpat_Dispatch dispatch;
dispatch.size = sizeof(dispatch):
dispatch.MAJOR_VERSION = XML_MAJOR_VERSION;
...
obj = PyCObject_FromVoidPtrAndDesc(
&dispatch, PyExpat_DISPATCH_MAGIC, NULL
);
... stuff object into module dictionary ...
3. in cElementTree (or _elementtree, or whatever the python version will
be named), import pyexpat, fetch the object, and verify
- that the PyExpat_DISPATCH_MAGIC matches
- that the size field is at least as large as sizeof(struct
PyExpat_Dispatch)
- that the version number matches (at least MAJOR and MINOR; I'm not
sure under what circumstances they change the MICRO number)
4. in cElementTree (...), do all expat calls via the dispatch table.
comments ?
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
