Re: [Distutils] Is it safe to embedding additional metadata in bdist_wininst installers ?

[David Cournapeau]

On Tue, Jun 15, 2010 at 3:39 PM, "Martin v. Löwis"  wrote:
> Am 15.06.2010 08:23, schrieb David Cournapeau:
>>
>> On Tue, Jun 15, 2010 at 3:13 PM, "Martin v. Löwis"
>>  wrote:

 I am interested in adding some metadata in the installers generated by
 bdist_wininst. I did it through a hack: base64-encoding the medata in
 a new section in the embedded cfgdata. From my understanding the
 installer.c sources, this seems safe, and the generated installer does
 work, but I am wondering whether there is a better way of doing it,
 and whether it could potentially breaks something,
>>>
>>> Why not use a file that gets installed in the system?
>>
>> Because I don't want the file installed. I am interested in doing this
>> for arbitrary packages (the goal is to add support for egg<->  windows
>> installer conversion in my packaging solution)
>
> Ah. I would then add a prefix META to the scheme, and modify bdist_wininst
> to skip all files with that prefix.

I did not want to go that road because of the need to recompile
bdist_wininst for all python versions, but maybe that's the best
solution.

Do you think it would make sense to do it for python itself proper (in
distutils), so that different implementations could put whatever they
want there for their purpose ?

cheers,

David
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Is it safe to embedding additional metadata in bdist_wininst installers ?

[Martin v. Löwis]

Am 15.06.2010 08:23, schrieb David Cournapeau:

On Tue, Jun 15, 2010 at 3:13 PM, "Martin v. Löwis"  wrote:

I am interested in adding some metadata in the installers generated by
bdist_wininst. I did it through a hack: base64-encoding the medata in
a new section in the embedded cfgdata. From my understanding the
installer.c sources, this seems safe, and the generated installer does
work, but I am wondering whether there is a better way of doing it,
and whether it could potentially breaks something,


Why not use a file that gets installed in the system?


Because I don't want the file installed. I am interested in doing this
for arbitrary packages (the goal is to add support for egg<->  windows
installer conversion in my packaging solution)


Ah. I would then add a prefix META to the scheme, and modify 
bdist_wininst to skip all files with that prefix.


Regards,
Martin
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Is it safe to embedding additional metadata in bdist_wininst installers ?

[David Cournapeau]

On Tue, Jun 15, 2010 at 3:13 PM, "Martin v. Löwis"  wrote:
>> I am interested in adding some metadata in the installers generated by
>> bdist_wininst. I did it through a hack: base64-encoding the medata in
>> a new section in the embedded cfgdata. From my understanding the
>> installer.c sources, this seems safe, and the generated installer does
>> work, but I am wondering whether there is a better way of doing it,
>> and whether it could potentially breaks something,
>
> Why not use a file that gets installed in the system?

Because I don't want the file installed. I am interested in doing this
for arbitrary packages (the goal is to add support for egg <-> windows
installer conversion in my packaging solution)

David
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Is it safe to embedding additional metadata in bdist_wininst installers ?

[Martin v. Löwis]

I am interested in adding some metadata in the installers generated by
bdist_wininst. I did it through a hack: base64-encoding the medata in
a new section in the embedded cfgdata. From my understanding the
installer.c sources, this seems safe, and the generated installer does
work, but I am wondering whether there is a better way of doing it,
and whether it could potentially breaks something,


Why not use a file that gets installed in the system?

Regards,
Martin
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Is it safe to embedding additional metadata in bdist_wininst installers ?

[David Cournapeau]

Hi,

I am interested in adding some metadata in the installers generated by
bdist_wininst. I did it through a hack: base64-encoding the medata in
a new section in the embedded cfgdata. From my understanding the
installer.c sources, this seems safe, and the generated installer does
work, but I am wondering whether there is a better way of doing it,
and whether it could potentially breaks something,

cheers,

David
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Design rationale for the egg format ?

[David Cournapeau]

On Tue, Jun 15, 2010 at 10:36 AM, P.J. Eby  wrote:

> As I said above, "it *also* scales better for performance" -- i.e., it's a
> secondary concern.

ok.

>  The #1 reason for separating metadata files is that it
> makes the addition of new metadata much easier than maintaining a single
> monolithic format.

Do you still think it is true today ? I am asking because AFAIK, there
aren't many packages besides setuptools which use those metadata ? I
don't mean to criticize the design, just to see if you would do things
differently today.

cheers,

David
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Design rationale for the egg format ?

[Greg Ewing]

Paul Moore wrote:

On 14 June 2010 07:59, David Cournapeau  wrote:


- why does the filename encode some metadata (which python version if
the package contains extensions, platform specifier) ?


I believe that this
is so that the basic metadata can be obtained as part of an initial
listdir()


It would also seem useful to be able to keep a collection
of versions of a package for different python configurations
in the same directory without their names clashing.

--
Greg
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Design rationale for the egg format ?

[P.J. Eby]

At 08:29 AM 6/15/2010 +0900, David Cournapeau wrote:

On Mon, Jun 14, 2010 at 11:28 PM, P.J. Eby  wrote:
> At 03:59 PM 6/14/2010 +0900, David Cournapeau wrote:
>> Â - why are the metadata split into files instead of one single metadata
>> file ?
>
> Because that's simpler than trying to define a single universal file format
> that's forward and backward-compatible with every possible feature and use
> case. Â Each use case can have an optimized file format.
>
> It also scales better for performance when you have multiple things you
> might (or might not) be reading. Â For example, since entry points are
> separate from dependencies, you you don't need to read the 
dependencies from

> an egg that doesn't have an entry point you're scanning for.

What I am interested in is the exact situations where this happens
(there is the case where eggs are used as plugins, the case where eggs
are namespace packages, etc...). For example, I don't quite understand
why reading dependencies need to be fast (it does not matter at
install time, so I guess I am missing some usecases) ?


As I said above, "it *also* scales better for performance" -- i.e., 
it's a secondary concern.  The #1 reason for separating metadata 
files is that it makes the addition of new metadata much easier than 
maintaining a single monolithic format.


That is, programs that don't understand new metadata don't have to 
read it.  Plugins that write metadata don't need to co-operate with 
others - they can just write their own files.  And so on.


That is the original reason for making separate metadata files: i.e. 
simplicity.  It just turned out to also provide a performance benefit 
in the case of cross-egg scanning for distinct types of metadata.


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Easy install problems

[P.J. Eby]

At 12:54 PM 6/14/2010 -0700, SEIGAL Nick wrote:
I am having trouble using easy install on a Windows Server 2008 
server. It seems as thought I am getting blocked from downloading 
the dependencies.


At the command line I get this:

C:\Users>easy_install camelot
Searching for camelot
Best match: camelot 09.12.07
Processing camelot-09.12.07-py2.6.egg
camelot 09.12.07 is already the active version in easy-install.pth
Installing camelot_manage-script.py script to C:\Python26\Scripts
Installing camelot_manage.exe script to C:\Python26\Scripts
Installing camelot_manage.exe.manifest script to C:\Python26\Scripts
Installing camelot_admin-script.py script to C:\Python26\Scripts
Installing camelot_admin.exe script to C:\Python26\Scripts
Installing camelot_admin.exe.manifest script to C:\Python26\Scripts

Using c:\python26\lib\site-packages\camelot-09.12.07-py2.6.egg
Processing dependencies for camelot
Searching for Babel>=0.9.4
Reading 
http://pypi.python.org/simple/Babel/
Download error: [Errno 10060] A connection attempt failed because 
the connected
party did not properly respond after a period of time, or 
established connection
failed because connected host has failed to respond -- Some packages 
may not be

found!
Couldn't find index page for 'Babel' (maybe misspelled?)

This process results in a long series of errors at this point.

Any suggestions?


Check your internet settings to see if they're going through a proxy; 
if so, you'll need to set the http_proxy environment variable, so 
Python knows how to access the 'net from there.



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Easy install problems

[SEIGAL Nick]

I am having trouble using easy install on a Windows Server 2008 server. It 
seems as thought I am getting blocked from downloading the dependencies.

At the command line I get this:

C:\Users>easy_install camelot
Searching for camelot
Best match: camelot 09.12.07
Processing camelot-09.12.07-py2.6.egg
camelot 09.12.07 is already the active version in easy-install.pth
Installing camelot_manage-script.py script to C:\Python26\Scripts
Installing camelot_manage.exe script to C:\Python26\Scripts
Installing camelot_manage.exe.manifest script to C:\Python26\Scripts
Installing camelot_admin-script.py script to C:\Python26\Scripts
Installing camelot_admin.exe script to C:\Python26\Scripts
Installing camelot_admin.exe.manifest script to C:\Python26\Scripts

Using c:\python26\lib\site-packages\camelot-09.12.07-py2.6.egg
Processing dependencies for camelot
Searching for Babel>=0.9.4
Reading http://pypi.python.org/simple/Babel/
Download error: [Errno 10060] A connection attempt failed because the connected
party did not properly respond after a period of time, or established connection
 failed because connected host has failed to respond -- Some packages may not be
 found!
Couldn't find index page for 'Babel' (maybe misspelled?)

This process results in a long series of errors at this point.

Any suggestions?

Nick Seigal
GIS Analyst/Application Developer
Lane Council of Governments  -  859 Willamette Street, Suite 500  -  Eugene, 
Oregon 97401-2910
(541) 682-6733  -  (541) 682-4099 (fax)  -  www.lcog.org


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Design rationale for the egg format ?

[P.J. Eby]

At 03:59 PM 6/14/2010 +0900, David Cournapeau wrote:

Hi,

I have a few questions about the egg format implementation, and was
hoping people who designed it could answer them:
 - why does the filename encode some metadata (which python version if
the package contains extensions, platform specifier) ?


As Paul mentioned, it's so that discovery can happen without needing 
anything more than a listdir().  (A legacy of eggs' origin as a 
binary plugin format.)



 - why are the metadata split into files instead of one single 
metadata file ?


Because that's simpler than trying to define a single universal file 
format that's forward and backward-compatible with every possible 
feature and use case.  Each use case can have an optimized file format.


It also scales better for performance when you have multiple things 
you might (or might not) be reading.  For example, since entry points 
are separate from dependencies, you you don't need to read the 
dependencies from an egg that doesn't have an entry point you're scanning for.


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Design rationale for the egg format ?

[Ben Finney]

David Cournapeau  writes:

> I have a few questions about the egg format implementation, and was
> hoping people who designed it could answer them:
[…]

> I tried to find a rationale for those, but could not find much info on
> distutils-sig or setuptools doc,

You might find it useful to read this archived thread:

I've noticed that there seems to be a lot of confusion out there
about what setuptools is and/or does, at least among Python-Dev
folks, so I thought it might be a good idea to give an overview of
its structure, so that people have a better idea of what is and
isn't "magic".
[…]

http://mail.python.org/pipermail/python-dev/2006-April/064145.html>

It goes into some amount of detail on the background and rationale for
Setuptools and its early design decisions.

-- 
 \“Sane people have an appropriate perspective on the relative |
  `\ importance of foodstuffs and human beings. Crazy people can't |
_o__) tell the difference.” —Paul Z. Myers, 2010-04-18 |
Ben Finney

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Design rationale for the egg format ?

[David Cournapeau]

On Mon, Jun 14, 2010 at 4:34 PM, Paul Moore  wrote:
> On 14 June 2010 07:59, David Cournapeau  wrote:
>> Hi,
>>
>> I have a few questions about the egg format implementation, and was
>> hoping people who designed it could answer them:
>>  - why does the filename encode some metadata (which python version if
>> the package contains extensions, platform specifier) ?
>
> I'm not one of the designers, nor an expert, but I believe that this
> is so that the basic metadata can be obtained as part of an initial
> listdir(), without needing to open and read a file at all - so
> essentially it's to reduce the number of OS calls needed in certain
> key cases.

Do you happen to know what those key cases are ?

cheers,

David
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Design rationale for the egg format ?

[Paul Moore]

On 14 June 2010 07:59, David Cournapeau  wrote:
> Hi,
>
> I have a few questions about the egg format implementation, and was
> hoping people who designed it could answer them:
>  - why does the filename encode some metadata (which python version if
> the package contains extensions, platform specifier) ?

I'm not one of the designers, nor an expert, but I believe that this
is so that the basic metadata can be obtained as part of an initial
listdir(), without needing to open and read a file at all - so
essentially it's to reduce the number of OS calls needed in certain
key cases.

Paul.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig