Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-22 Thread Oscar Benjamin
On 19 July 2013 20:48, Steve Dower steve.do...@microsoft.com wrote:
 From: Oscar Benjamin
 I don't know whether or not you intend to have wrappers also work for
 Python 2.7 (in a third-party package perhaps) but there is a slightly
 subtle point to watch out for when non-ASCII characters in sys.argv
 come into play.

 Python 2.x uses GetCommandLineA and 3.x uses GetCommandLineW. A
 wrapper to launch 2.x should use GetCommandLineA and CreateProcessA to
 ensure that the 8-bit argument strings are passed through unaltered.
 To launch 3.x it should use the W versions. If not then the MSVC
 runtime (or the OS?) will convert between the 8-bit and 16-bit
 encodings using its own lossy routines.

 The launcher should always use GetCommandLineW, because the command line is 
 already stored in a 16-bit encoding. GetCommandLineA will decode to an 8-bit 
 encoding using some code page/settings (I can probably find out exactly which 
 ones, but I don't know/care off the top of my head), and CreateProcessA will 
 convert back using (hopefully) the same code page.

 There is never any point passing data between *A APIs in Windows, because 
 they are just doing the conversion in the background. All you gain is that 
 the launcher will corrupt the command line before python.exe gets a chance to.

Okay, thanks for the correction.

The issue that made me think this was to do with calling Python 2.x as
a subprocess of 3.x and vice-versa. When I looked back at it now I saw
that the problem was to do with explicitly encoding with
sys.getfilesystemencoding() in Python and using the mbcs codec (which
previously had no error handling apart from 'replace').


Oscar
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-19 Thread Vinay Sajip
Oscar Benjamin oscar.j.benjamin at gmail.com writes:

 Python 2.x uses GetCommandLineA and 3.x uses GetCommandLineW. A
 wrapper to launch 2.x should use GetCommandLineA and CreateProcessA to
 ensure that the 8-bit argument strings are passed through unaltered.
 To launch 3.x it should use the W versions. If not then the MSVC
 runtime (or the OS?) will convert between the 8-bit and 16-bit
 encodings using its own lossy routines.

There is a standalone launcher available (referenced in PEP 397) which was the 
reference implementation, but it uses Unicode throughout. I would have assumed 
that whatever provided the decoding behind GetCommandLineW would use the same 
encoding in CreateProcessW(..., unicode_command_line, ...). Do you have any 
specific examples showing potential failure modes? 

Regards,

Vinay Sajip

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-19 Thread Steve Dower
 From: Oscar Benjamin
 I don't know whether or not you intend to have wrappers also work for
 Python 2.7 (in a third-party package perhaps) but there is a slightly
 subtle point to watch out for when non-ASCII characters in sys.argv
 come into play.
 
 Python 2.x uses GetCommandLineA and 3.x uses GetCommandLineW. A
 wrapper to launch 2.x should use GetCommandLineA and CreateProcessA to
 ensure that the 8-bit argument strings are passed through unaltered.
 To launch 3.x it should use the W versions. If not then the MSVC
 runtime (or the OS?) will convert between the 8-bit and 16-bit
 encodings using its own lossy routines.

The launcher should always use GetCommandLineW, because the command line is 
already stored in a 16-bit encoding. GetCommandLineA will decode to an 8-bit 
encoding using some code page/settings (I can probably find out exactly which 
ones, but I don't know/care off the top of my head), and CreateProcessA will 
convert back using (hopefully) the same code page.

There is never any point passing data between *A APIs in Windows, because they 
are just doing the conversion in the background. All you gain is that the 
launcher will corrupt the command line before python.exe gets a chance to.


Cheers,
Steve 



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-17 Thread Vinay Sajip
Nick Coghlan ncoghlan at gmail.com writes:

 Actually, it may be better to have a top level scripts field, distinct
from a general export mechanism.
 I'm seeing value in an exports mechanism, though.

The exports functionality is important and used enough to warrant support in
the PEP, and not only for the scripts part. There should be some way that
this data gets into .dist-info in a standardised way, so that there is an
ability to query e.g. implementations of a particular interface. Currently,
distlib supports this as an extension to the PEP by reading a file from
.dist-info, which distil puts there. At the moment the PEP is silent on the
subject, which could lead to fragmentation in the implementations - e.g.
whether JSON Or ini-style format is used for the data.

I'd like to suggest that the whole of the exports information be included in
the PEP 426 metadata, without singling out the scripts part. That way, it
ends up in .dist-info via pydist.json. It has been suggested by PJE that the
exports information should be in a separate file for speed of searching -
though that suggestion was made in a pre-JSON world, where the speed of
parsing the metadata wasn't C-assisted. Should performance still be an
issue, then the exports dict could still be written out separately in
.dist-info as e.g. exports.json by an installer.

Regards,

Vinay Sajip

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-17 Thread Donald Stufft

On Jul 17, 2013, at 3:40 AM, Vinay Sajip vinay_sa...@yahoo.co.uk wrote:

 It has been suggested by PJE that the
 exports information should be in a separate file for speed of searching -
 though that suggestion was made in a pre-JSON world, where the speed of
 parsing the metadata wasn't C-assisted.

I don't think it was speed of parsing the file that was his concern, rather that
if it's a separate file that only exists when there are entry points, then you
don't have to open a file for every installed distribution.

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-17 Thread Vinay Sajip
Donald Stufft donald at stufft.io writes:

 I don't think it was speed of parsing the file that was his concern,
rather that
 if it's a separate file that only exists when there are entry points, then you
 don't have to open a file for every installed distribution.

OK. In that case, exports.json would avoid the need to open pydist.json to
look for exports - if exports.json is missing, it would mean that dist has
no exports.

Regards,

Vinay Sajip

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Nick Coghlan
On 16 July 2013 23:08, Paul Moore p.f.mo...@gmail.com wrote:
 Nick: See the above point re PEP 426 - do you agree that this needs
 addressing in Metadata 2.0?

I believe Daniel already covered it in PEP 427 - rather than baking
the entry point wrappers into the wheel, installers can generate any
needed entry point wrappers if the wheel includes Python scripts in
{distribution}-{version}.data/scripts/ (see
http://www.python.org/dev/peps/pep-0427/#recommended-installer-features)

Now, there may be holes in that scheme, but it seemed solid enough
when I approved the PEP.

Cheers,
Nick.

--
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Paul Moore
On 16 July 2013 14:21, Nick Coghlan ncogh...@gmail.com wrote:

 On 16 July 2013 23:08, Paul Moore p.f.mo...@gmail.com wrote:
  Nick: See the above point re PEP 426 - do you agree that this needs
  addressing in Metadata 2.0?

 I believe Daniel already covered it in PEP 427 - rather than baking
 the entry point wrappers into the wheel, installers can generate any
 needed entry point wrappers if the wheel includes Python scripts in
 {distribution}-{version}.data/scripts/ (see
 http://www.python.org/dev/peps/pep-0427/#recommended-installer-features)

 Now, there may be holes in that scheme, but it seemed solid enough
 when I approved the PEP.


The big problem is that implementations don't do that :-( AFAIK, none of
distlib, pip or wheel itself do anything with script wrappers except
rewrite #! lines (which is the other, much easier, item in that section).

At the moment, bdist_wheel just puts the exe wrappers generated from the
source into the wheel itself, which again is probably wrong in the context
what the PEP suggests.

As I said in my other email, I think this is subtle enough that we need a
stdlib implementation to stop people making mistakes like this. It's
certainly not fair to expect a mostly-Unix development team to get this
sort of Windows arcana right without some help.

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Ronald Oussoren

On 16 Jul, 2013, at 15:08, Paul Moore p.f.mo...@gmail.com wrote:

 On 16 July 2013 13:42, Ronald Oussoren ronaldousso...@mac.com wrote:
  On the other hand, I'm missing something, as I don't see how the *current* 
  exe wrappers avoid meaning that there need to be separate 32-bit and 64-bit 
  versions of pip...
 
 Couldn't you just ship both variants of the exe wrappers in a single 
 distribution and then use the correct one for the current installation? 
 That's what I'm doing in py2app.
 
 That's OK for source-style installs (which is what setuptools does, and what 
 pip mostly cares about right now). But for bundling with Python it needs to 
 be considered (although it's just getting the right one in the right 
 installer). But for wheels it's a pain, because instead of having just a 
 single pip wheel, we need 32-bit and 64-bit wheels solely for the wrappers. 
 That sucks. Hard. (And it's not just for pip, nose will have the same 
 problem, as will many other projects that use exe wrappers). And the 
 bdist_wheel command currently doesn't recognise this *at all*. So wheels 
 using wrappers are potentially broken.

You could just have a wheel that contains two data files: wrapper-win32.exe and 
wrapper-win64.exe, then select the one that gets used as the wrapper for a 
specific script when you create that wrapper. 

That's assuming that the wrapper .exe gets created when a wheel is installed 
and is not included in the wheel.

 
 I think the correct solution is to explicitly have declarative support for 
 console script entry point metadata in PEP 426, as well as having tools 
 like bdist_wheel and distil do some explicit backward compatibility hacking 
 to remove legacy-style exe wrappers. The wheel install code should then 
 explicitly install appropriate wrappers for the target platform (which may be 
 exe wrappers similar to the current ones, but moving forward may be some 
 other mechanism if one is found).

Yikes, that means my assumption is wrong. The section on Recommended installer 
features in the wheel spec[1] says that the wrapper executable should be 
created on installation, does pip not do this?

 
 This is complex enough that I'm now concerned that we need reference wheel 
 install code in the stdlib, just so that people don't make up their own on 
 the basis that wheel is simple to install manually and screw it up. Also so 
 that we only have one style of command line script wrapper to deal with going 
 forward, not a multitude of mostly-compatible solutions.

I'd love to see comprehensive wheel support in the stdlib, but that may have to 
wait for 3.5 because the entire packaging systeem (wheels, metadata, ...) is 
moving forward quickly at the moment.  That said, it would be nice if distutils 
would grow support for creating wheels and modern metadata in sdists as that 
would mean I could drop usage of setuptools for most of my software (for python 
3.4). 

 
 Nick: See the above point re PEP 426 - do you agree that this needs 
 addressing in Metadata 2.0?
 
 Paul
 
 PS There is still the proviso that I haven't tested my assumption that the 
 separate 32 and 64 bit wrappers are *needed* (setuptools and distlib use 
 them, so I think it's a valid assumption, but I need to test). I will try to 
 get time to check that ASAP.

That depends on what the wrapper does, if it launches a regular python with the 
right command-line you might be able to get away with a single wrapper, if it 
loads python.dll and executes the script directory you do need separate 
wrappers for 32 and 64 bit.


[1] http://www.python.org/dev/peps/pep-0427/#recommended-installer-features

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Paul Moore
On 16 July 2013 14:08, Paul Moore p.f.mo...@gmail.com wrote:

 PS There is still the proviso that I haven't tested my assumption that the
 separate 32 and 64 bit wrappers are *needed* (setuptools and distlib use
 them, so I think it's a valid assumption, but I need to test). I will try
 to get time to check that ASAP.


Hmm. I just did a quick test, and then based on the results checked the
setuptools source code. I can see no reason why there needs to be 32 and 64
bit launcher exes. The launchers simply use CreateProcess to launch a
separate Python process using the #! line of the script. So there's no DLL
loading going on, and no reason that I can see for needing separate 32 and
64 bit builds.

Jason - can you shed any light on why there are separate builds for 32 and
64 bits?

Actually, the launcher is essentially identical to the py launcher for
Windows, except that it gets a script name to execute from the name of the
launcher. I'm wondering whether the correct approach here would be to
enhance the launcher one more time to look for a suitably named script and
auto-run it if it's present (i.e. merge the wrapper functionality into the
launcher). Then we have a standard wrapper that everyone can use and not
reinvent their own.

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Paul Moore
On 16 July 2013 14:30, Ronald Oussoren ronaldousso...@mac.com wrote:

  I think the correct solution is to explicitly have declarative support
 for console script entry point metadata in PEP 426, as well as having
 tools like bdist_wheel and distil do some explicit backward compatibility
 hacking to remove legacy-style exe wrappers. The wheel install code should
 then explicitly install appropriate wrappers for the target platform (which
 may be exe wrappers similar to the current ones, but moving forward may be
 some other mechanism if one is found).

 Yikes, that means my assumption is wrong. The section on Recommended
 installer features in the wheel spec[1] says that the wrapper executable
 should be created on installation, does pip not do this?


Yes, Nick pointed me at that part of the PEP. Nobody's doing that at the
moment, and exes are being added to the wheels at wheel build time, which
is also wrong.

That'll teach me to work from reality rather than specs :-(

Daniel, Vinay, pip developers - it looks like we need to do some work in
this area to make the code conform to the specs. The PEP only says this is
a recommended feature, but frankly I think it needs to be mandatory, or
script wrappers are going to be a mess we'll be dealing with for some time
:-(

 PS There is still the proviso that I haven't tested my assumption that
 the separate 32 and 64 bit wrappers are *needed* (setuptools and distlib
 use them, so I think it's a valid assumption, but I need to test). I will
 try to get time to check that ASAP.

 That depends on what the wrapper does, if it launches a regular python
 with the right command-line you might be able to get away with a single
 wrapper, if it loads python.dll and executes the script directory you do
 need separate wrappers for 32 and 64 bit.


As I said in another message, looks like there's no real reason for
separate wrappers. A 32-bit one should work regardless [1]. But wheels
built on 64-bit systems at the moment won't work on 32-bit ones (because
the wrappers will be 64-bit).

[1] With the possible exception that Windows' magic shadowing of 32 and 64
bit stuff - the WOW64 magic that I know nothing about - could cause odd
results in obscure cases. I propose ignoring this in the absence of actual
bug reports :-)

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Nick Coghlan
On 16 July 2013 23:29, Paul Moore p.f.mo...@gmail.com wrote:
 As I said in my other email, I think this is subtle enough that we need a
 stdlib implementation to stop people making mistakes like this. It's
 certainly not fair to expect a mostly-Unix development team to get this sort
 of Windows arcana right without some help.

Are we talking about the pip developers or python-dev, here? I think
Martin and Brian feel pretty lonely, too :)

Cheers,
Nick.

--
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Vinay Sajip
Paul Moore p.f.moore at gmail.com writes:

 Yes, Nick pointed me at that part of the PEP. Nobody's doing that at the
 moment, and exes are being added to the wheels at wheel build time, which
 is also wrong.

Not true for distlib - it doesn't add .exe wrappers to wheels at build time
:-) It adds them to the target directory when installing under Windows. (You
can also choose not to install any wrappers.)

Regards,

Vinay Sajip

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Daniel Holth
Need a script that visits all the console-script entry points to
regenerate the wrappers. Then there are also the non-console-scripts
scripts...

I consider scripts as an optional convenience, but I suppose that
isn't always the case.

On Tue, Jul 16, 2013 at 9:53 AM, Vinay Sajip vinay_sa...@yahoo.co.uk wrote:
 Paul Moore p.f.moore at gmail.com writes:

 Yes, Nick pointed me at that part of the PEP. Nobody's doing that at the
 moment, and exes are being added to the wheels at wheel build time, which
 is also wrong.

 Not true for distlib - it doesn't add .exe wrappers to wheels at build time
 :-) It adds them to the target directory when installing under Windows. (You
 can also choose not to install any wrappers.)

 Regards,

 Vinay Sajip

 ___
 Distutils-SIG maillist  -  Distutils-SIG@python.org
 http://mail.python.org/mailman/listinfo/distutils-sig
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Vinay Sajip
Vinay Sajip vinay_sajip at yahoo.co.uk writes:
 
 Not true for distlib - it doesn't add .exe wrappers to wheels at build time
  It adds them to the target directory when installing under Windows. (You
 can also choose not to install any wrappers.)

Sorry, some misinformation there - distlib does do this when invoked via
distil. However, this can be turned off at the distlib level - I will update
distil to not do this when adding wheels.

Regards,

Vinay Sajip

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Paul Moore
On 16 July 2013 15:01, Vinay Sajip vinay_sa...@yahoo.co.uk wrote:

 Vinay Sajip vinay_sajip at yahoo.co.uk writes:
 
  Not true for distlib - it doesn't add .exe wrappers to wheels at build
 time
   It adds them to the target directory when installing under Windows. (You
  can also choose not to install any wrappers.)

 Sorry, some misinformation there - distlib does do this when invoked via
 distil. However, this can be turned off at the distlib level - I will
 update
 distil to not do this when adding wheels.


OK. That sounds good.

I'm starting to become uncertain as to whether we actually have an issue
here. I think:

(1) Builders can put anything they like into the scripts directory. That's
more or less out of our control. From what I know of distil's approach,  it
doesn't actually execute setup.py so it keeps a lot more control than (say)
bdist_wheel, which effectively runs setup.py install to a temporary
directory then bundles what it finds. But in essence, as things stand right
now, the scripts directory of an arbitrary wheel could contain arbitrary
stuff.
(2) Wheel installers should make Python scripts work. All that I'm aware
of fix up shebang lines and add execute bits. Only distlib adds exe
wrappers, I believe. Others work on Windows because the builders put exe
wrappers in place (see (1)) but that has some issues. I suspect there may
be some accidental successes on Unix where setuptools scripts do or don't
get a .py extension depending on the target platform, but I have no
evidence of this myself.

So there's a lot of potentially platform dependent stuff in scripts and
wheel builders don't (can't) recognise and encode this in the tags. Wheels
mostly work at the moment because not many people use them cross platform.
But there's potential for issues down the line.

FWIW, I believe that the whole scripts directory as a concept is too
platform-specific. The only real use for it is to expose CLI applications,
and a declarative approach like setuptools console_scripts entry points
would be better. So longer term I'd argue for deprecating scripts
altogether and replacing it with some form of CLI entry point metadata
which may be exposed as part of metadata 2.0 or 3.0, or may simply be
internal metadata communicated between the builder and the installer, but
not exposed at the PyPI level. Oscar's email argues for exposing it as
project metadata, though, and I can see the benefit.

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Paul Moore
On 16 July 2013 16:09, Oscar Benjamin oscar.j.benja...@gmail.com wrote:

 If you only want the program to be invokable from cmd and PowerShell*
 then a .bat file should be fine. Depending on file extension to invoke
 .py files with py.exe is subject to input/output redirection bugs on
 some windows systems (this is solveable when using .py in PATHEXT
 instead of file associations for cmd at least).


bat files have many, many problems. The worst ones are:

* Not nestable. If pip is a bat file, saying pip install foo from within
another bat file will fail silently (control never returns to the line
after the pip command).
* If you interrupt them you get the obnoxious Do you want to terminate the
batch file? prompt.

If anyone suggests using bat files, I'll cry :-)

However, if you also want the program name to be invokable from e.g.
 subprocess with shell=False or from git-bash or Cygwin or many other
 things then neither .bat files nor PATHEXT are sufficient. Wrapper
 .exes are necessary to ensure that this works properly.


Yes. I have been convinced that ultimately, wrapper exes are the only
transparent means of writing command-line applications on Windows.

Because of this, I'd quite like it if wrapper functionality were added to
the py launcher (most of the functionality is already present, it would
probably be a pretty small change) so that we had a one obvious way of
writing wrappers. I may try to put together a patch for CPython to this
effect...

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Oscar Benjamin
On 16 July 2013 16:22, Paul Moore p.f.mo...@gmail.com wrote:
 On 16 July 2013 16:09, Oscar Benjamin oscar.j.benja...@gmail.com wrote:

 However, if you also want the program name to be invokable from e.g.
 subprocess with shell=False or from git-bash or Cygwin or many other
 things then neither .bat files nor PATHEXT are sufficient. Wrapper
 .exes are necessary to ensure that this works properly.

 Yes. I have been convinced that ultimately, wrapper exes are the only
 transparent means of writing command-line applications on Windows.

 Because of this, I'd quite like it if wrapper functionality were added to
 the py launcher (most of the functionality is already present, it would
 probably be a pretty small change) so that we had a one obvious way of
 writing wrappers. I may try to put together a patch for CPython to this
 effect...

I don't know whether or not you intend to have wrappers also work for
Python 2.7 (in a third-party package perhaps) but there is a slightly
subtle point to watch out for when non-ASCII characters in sys.argv
come into play.

Python 2.x uses GetCommandLineA and 3.x uses GetCommandLineW. A
wrapper to launch 2.x should use GetCommandLineA and CreateProcessA to
ensure that the 8-bit argument strings are passed through unaltered.
To launch 3.x it should use the W versions. If not then the MSVC
runtime (or the OS?) will convert between the 8-bit and 16-bit
encodings using its own lossy routines.


Oscar
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Vinay Sajip
Paul Moore p.f.moore at gmail.com writes:

 FWIW, I believe that the whole scripts directory as a concept is too
 platform-specific. The only real use for it is to expose CLI applications

Well, you can also expose GUI applications this way, though the applications
are fewer - Qt applications could easily be cross-platform, for example.
Also, you can put in scripts which are not entry-point related, which would
be essential if you e.g. have existing scripts to bundle as part of an
application, which could be written in other languages, say. Not common,
perhaps, but not a case you want to arbitrarily restrict given that it works
now. Of course, there the distributor of the package is responsible for
ensuring cross-platform workability of such scripts using e.g. including
.cmd files for Windows or whatever.

Regards,

Vinay Sajip

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Paul Moore
On 16 July 2013 16:51, Vinay Sajip vinay_sa...@yahoo.co.uk wrote:

 Paul Moore p.f.moore at gmail.com writes:

  FWIW, I believe that the whole scripts directory as a concept is too
  platform-specific. The only real use for it is to expose CLI applications

 Well, you can also expose GUI applications this way, though the
 applications
 are fewer - Qt applications could easily be cross-platform, for example.
 Also, you can put in scripts which are not entry-point related, which would
 be essential if you e.g. have existing scripts to bundle as part of an
 application, which could be written in other languages, say. Not common,
 perhaps, but not a case you want to arbitrarily restrict given that it
 works
 now. Of course, there the distributor of the package is responsible for
 ensuring cross-platform workability of such scripts using e.g. including
 .cmd files for Windows or whatever.


You're right, I should have included GUI apps.

I'm not aware of any cases of scripts that aren't entry-point related (and
which couldn't be converted to entry points fairly easily) although that
certainly doesn't mean there aren't any. OTOH, I do know that there used to
be a *lot* of examples of scripts doing what entry points do that were
either absolutely not cross-platform (Unix shell scripts, bat files) or
were problematic/badly implemented (Python scripts without a py extension,
etc)  Many of these have now gone, thank goodness, migrated to setuptools
entry points.

If I were writing a firm proposal, I'd go for something like entry points
as metadata, managed by installers, as the primary interface (with backward
compatibility code to automatically migrate setuptools entry points so the
impact on developers is minimal) and the scripts directory/setup argument
being solely for backward compatibility, no management by installers at
all, whatever is in there just gets dumped onto the target unchanged (and
the contents of scripts is explicitly defined as *not* affecting the
compatibility tags).

So setuptools users get new entry point metadata automatically, which is
processed at install time, and anyone still using the distutils scripts
parameter still works as before but gets no particular support from the new
tools.

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Vinay Sajip


If I were writing a firm proposal, I'd go for something like entry points as 
metadata

My extended metadata already covers this, though I use the name exports 
(suggested by PJE) because you can share not just code but data, and entry 
points generally implies code. The current version of distil creates wrappers 
for both gui and console scripts, and adds the appropriate native executable 
wrappers (32- or 64-bit, according to the running Python) on Windows.

Example of the metadata for setuptools 0.8:
exports: { distutils.commands: [ alias = setuptools.command.alias:alias,  
bdist_egg = setuptools.command.bdist_egg:bdist_egg,  bdist_rpm = 
setuptools.command.bdist_rpm:bdist_rpm,  build_ext = 
setuptools.command.build_ext:build_ext,  build_py = 
setuptools.command.build_py:build_py,  develop = 
setuptools.command.develop:develop,  easy_install = 
setuptools.command.easy_install:easy_install,  egg_info = 
setuptools.command.egg_info:egg_info,  install = 
setuptools.command.install:install,  install_lib = 
setuptools.command.install_lib:install_lib,  rotate = 
setuptools.command.rotate:rotate,  saveopts = 
setuptools.command.saveopts:saveopts,  sdist = 
setuptools.command.sdist:sdist,  setopt = setuptools.command.setopt:setopt,  
test = setuptools.command.test:test,  install_egg_info = 
setuptools.command.install_egg_info:install_egg_info,  install_scripts = 
setuptools.command.install_scripts:install_scripts,  register =
 setuptools.command.register:register,  bdist_wininst = 
setuptools.command.bdist_wininst:bdist_wininst,  upload_docs = 
setuptools.command.upload_docs:upload_docs ],  scripts: { console: [ 
easy_install = setuptools.command.easy_install:main ] },  egg_info.writers: 
[ PKG-INFO = setuptools.command.egg_info:write_pkg_info,  requires.txt = 
setuptools.command.egg_info:write_requirements,  entry_points.txt = 
setuptools.command.egg_info:write_entries,  eager_resources.txt = 
setuptools.command.egg_info:overwrite_arg,  namespace_packages.txt = 
setuptools.command.egg_info:overwrite_arg,  top_level.txt = 
setuptools.command.egg_info:write_toplevel_names,  depends.txt = 
setuptools.command.egg_info:warn_depends_obsolete,  dependency_links.txt = 
setuptools.command.egg_info:overwrite_arg ],  setuptools.file_finders: [ 
svn_cvs = setuptools.command.sdist:_default_revctrl ],  
distutils.setup_keywords: [ eager_resources=
 setuptools.dist:assert_string_list,  namespace_packages = 
setuptools.dist:check_nsp,  extras_require = 
setuptools.dist:check_extras,  install_requires   = 
setuptools.dist:check_requirements,  tests_require  = 
setuptools.dist:check_requirements,  entry_points   = 
setuptools.dist:check_entry_points,  test_suite = 
setuptools.dist:check_test_suite,  zip_safe   = 
setuptools.dist:assert_bool,  package_data   = 
setuptools.dist:check_package_data,  exclude_package_data   = 
setuptools.dist:check_package_data,  include_package_data   = 
setuptools.dist:assert_bool,  packages   = 
setuptools.dist:check_packages,  dependency_links   = 
setuptools.dist:assert_string_list,  test_loader= 
setuptools.dist:check_importable,  use_2to3   = 
setuptools.dist:assert_bool,  convert_2to3_doctests  = 
setuptools.dist:assert_string_list, 
 use_2to3_fixers= setuptools.dist:assert_string_list,  
use_2to3_exclude_fixers = setuptools.dist:assert_string_list ],  
setuptools.installation: [ eggsecutable = 
setuptools.command.easy_install:bootstrap ] }

Regards,

Vinay Sajip

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Paul Moore
On 16 July 2013 19:18, Vinay Sajip vinay_sa...@yahoo.co.uk wrote:


 If I were writing a firm proposal, I'd go for something like entry points
 as metadata

 My extended metadata already covers this, though I use the name exports
 (suggested by PJE) because you can share not just code but data, and entry
 points generally implies code. The current version of distil creates
 wrappers for both gui and console scripts, and adds the appropriate native
 executable wrappers (32- or 64-bit, according to the running Python) on
 Windows.


Nice. So we could say that wheel installers should create exe wrappers
based on the 'scripts' key of the 'exports' metadata. Then we process the
'scripts' directory unaltered for backward compatibility.

The only other part of the equation is to ensure that wheel *builders* do
not put setuptools-generated entry point wrappers into the scripts
directory. Before I embarrass myself again (:-)) is this also something you
already have implemented in distlib? :-)

When you're finished with Guido's time machine, make sure you put the keys
back :-)
Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Vinay Sajip
Paul Moore p.f.moore at gmail.com writes:

 The only other part of the equation is to ensure that wheel *builders* do
 not put setuptools-generated entry point wrappers into the scripts
 directory. Before I embarrass myself again () is this also something you
 already have implemented in distlib? 

No, because I didn't want to embarrass you ;-)

Seriously - no, because that is policy rather than mechanism. The way distil
builds wheels is to perform an installation into a working area, and then
call Wheel.build in distlib pointing to the work area. The distlib code just
copies what's in the work area to the wheel. Currently the install-scripts
part of distil installs all scripts into the work area, including ones
defined in exports; it should be a five-minute job to ensure that scripts in
exports are excluded from this, when building wheels.

A version of distil with these updates should appear soon - I'm waiting for
the dust to settle on the most recent changes to PEP 426.

Regards,

Vinay Sajip

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Jason R. Coombs
There are two versions of launchers primarily because of my naiveté when
addressing the UAC issue. 64-bit launchers were exempt from the UAC
restrictions that caused them to launch in a separate window. I believed
this to be a proper fix, when in fact those still using 32-bit launchers
were still experiencing the problem. See
https://bitbucket.org/tarek/distribute/issue/143/easy_install-opens-new-cons
ole-cant-read for more detail.

 

So I agree, it would probably be sufficient to only supply 32-bit
executables. However, my preference would be to supply
architecture-appropriate executables rather than relying on a compatibility
layer.

 

Furthermore, I don’t believe the ARM architecture has a compatibility layer
(meaning 64-bit executables are required for 64-bit ARM builds), so
architecture and word size distinction is necessary.

 

I believe you’re right about leveraging the py launcher. I’d like for
setuptools to not have to supply launchers at all but depend on py launcher
instead. The py launcher is bundled with Python 3.3 so should become
ubiquitously available soon. I believe setuptools can begin to rely on it
and not supply a launcher at all. The scripts currently installed by
setuptools are suitable for launching by py launcher, so all that will need
to happen is to stop supplying its own launcher.

 

At least, that’s how I imagine it happening.

 

From: Paul Moore [mailto:p.f.mo...@gmail.com] 
Sent: Tuesday, 16 July, 2013 09:32
To: Distutils; Jason R. Coombs
Subject: Re: Wheels and console script entry point wrappers (Was: Replacing
pip.exe with a Python script)

 

On 16 July 2013 14:08, Paul Moore p.f.mo...@gmail.com
mailto:p.f.mo...@gmail.com  wrote:

PS There is still the proviso that I haven't tested my assumption that the
separate 32 and 64 bit wrappers are *needed* (setuptools and distlib use
them, so I think it's a valid assumption, but I need to test). I will try to
get time to check that ASAP.


Hmm. I just did a quick test, and then based on the results checked the
setuptools source code. I can see no reason why there needs to be 32 and 64
bit launcher exes. The launchers simply use CreateProcess to launch a
separate Python process using the #! line of the script. So there's no DLL
loading going on, and no reason that I can see for needing separate 32 and
64 bit builds.

 

Jason - can you shed any light on why there are separate builds for 32 and
64 bits?

 

Actually, the launcher is essentially identical to the py launcher for
Windows, except that it gets a script name to execute from the name of the
launcher. I'm wondering whether the correct approach here would be to
enhance the launcher one more time to look for a suitably named script and
auto-run it if it's present (i.e. merge the wrapper functionality into the
launcher). Then we have a standard wrapper that everyone can use and not
reinvent their own.

 

Paul



smime.p7s
Description: S/MIME cryptographic signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Vinay Sajip
Vinay Sajip vinay_sajip at yahoo.co.uk writes:

 defined in exports; it should be a five-minute job to ensure that scripts in
 exports are excluded from this, when building wheels.

It was a quick job, but thinking about it, I should probably update the
Wheel.install API to take an optional process_exports=True argument, so that
the exported-script processing can be done during installation from wheels
in a standardised way.

Regards,

Vinay Sajip

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Oscar Benjamin
On 16 July 2013 14:38, Paul Moore p.f.mo...@gmail.com wrote:
 On 16 July 2013 14:30, Ronald Oussoren ronaldousso...@mac.com wrote:

  I think the correct solution is to explicitly have declarative support
  for console script entry point metadata in PEP 426, as well as having
  tools like bdist_wheel and distil do some explicit backward compatibility
  hacking to remove legacy-style exe wrappers. The wheel install code should
  then explicitly install appropriate wrappers for the target platform (which
  may be exe wrappers similar to the current ones, but moving forward may be
  some other mechanism if one is found).

There are many other uses for console script entry point metadata. For
example, it would be good to be able to query pip/pypi in order to
find out which packages supply a particular console command. One
feature of Ubuntu that I really like is the way that it automatically
tells you how to install any missing command:

oscar:~$ pypy
The program 'pypy' is currently not installed. You can install it by typing:
sudo apt-get install pypy

Obviously it's not as useful when the command and the package have
exactly the same name :)


 Yikes, that means my assumption is wrong. The section on Recommended
 installer features in the wheel spec[1] says that the wrapper executable
 should be created on installation, does pip not do this?


 Yes, Nick pointed me at that part of the PEP. Nobody's doing that at the
 moment, and exes are being added to the wheels at wheel build time, which is
 also wrong.

 That'll teach me to work from reality rather than specs :-(

 Daniel, Vinay, pip developers - it looks like we need to do some work in
 this area to make the code conform to the specs. The PEP only says this is a
 recommended feature, but frankly I think it needs to be mandatory, or
 script wrappers are going to be a mess we'll be dealing with for some time
 :-(

I think that it should be mandatory.

It should be possible for someone releasing a script via pypi to
ensure that their program can be invoked after installation under a
name of their choosing (assuming the user has the Python Scripts/bin
directory in PATH). AFAIK the only bullet-proof way to do this on
Windows is with an .exe wrapper.

If you only want the program to be invokable from cmd and PowerShell*
then a .bat file should be fine. Depending on file extension to invoke
.py files with py.exe is subject to input/output redirection bugs on
some windows systems (this is solveable when using .py in PATHEXT
instead of file associations for cmd at least).

However, if you also want the program name to be invokable from e.g.
subprocess with shell=False or from git-bash or Cygwin or many other
things then neither .bat files nor PATHEXT are sufficient. Wrapper
.exes are necessary to ensure that this works properly.


Oscar


* I don't actually use PowerShell and cannot confirm that running .bat
files works fully i.e. without screwing up sys.argv encoding or
input/output redirection or anything else.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Paul Moore
On 16 July 2013 20:21, Vinay Sajip vinay_sa...@yahoo.co.uk wrote:

  defined in exports; it should be a five-minute job to ensure that
 scripts in
  exports are excluded from this, when building wheels.

 It was a quick job, but thinking about it, I should probably update the
 Wheel.install API to take an optional process_exports=True argument, so
 that
 the exported-script processing can be done during installation from wheels
 in a standardised way.


I'm not 100% sure what your proposal is here - I'm confused about the
precise roles of setup.py/setuptools as builder vs distil as builder vs
distlib as wheel builder vs distlib as wheel installer. I'll try to get
some time in the next day or so to review the code and make sure we're not
talking at cross purposes here. Bear with me.

I understood that distlib simply built wheels from whatever is in the
directories supplied. I'm not clear where it would get the exports
metadata to store in the wheel. Assuming that's sorted, though, then
whether or not distlib processes the exports on install is not an issue to
me (I think it always should). What is more of an issue is what the thing
that puts stuff into the directories does. If that's a setuptools-based
build, it will process the exports data *itself* and put wrappers into the
scripts directory. That is what I think we should be trapping and
suppressing. But we have to be careful, as if setuptools is used to install
directly, *not* going via a wheel, it has to generate wrappers itself
somehow.

Using distil to do the build is a whole other route.

My main concern here is keeping a reasonable level of backward
compatibility/interoperability with all the assorted tools around. And in
particular with pip managing (but technically not *doing*) the builds and
installs.

I'll have a think, and do some code reviews, and try not get sucked into
sending more emails until I know the facts a bit more clearly :-)

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Vinay Sajip
Paul Moore p.f.moore at gmail.com writes:

 I'm not 100% sure what your proposal is here - I'm confused about the precise
 roles of setup.py/setuptools as builder vs distil as builder vs distlib
 as wheel builder vs distlib as wheel installer. I'll try to get some time
 in the next day or so to review the code and make sure we're not talking at
 cross purposes here. Bear with me.

I'm not sure I've got a concrete proposal yet - I'm still thinking things
out, too :-)

Currently, the wheel module in distlib doesn't do anything too clever - just
the mechanical stuff of archiving and unarchiving. Because the exports stuff
wasn't in the PEP, distil wrote the generated scripts and .exe wrappers to
the work area, and so they ended up in the wheel. (This is the case for all
released versions of distil).

 I understood that distlib simply built wheels from whatever is in the
 directories supplied. I'm not clear where it would get the exports metadata
 to store in the wheel. Assuming that's sorted, though, then whether or not

Right now it's getting the exports courtesy of distil when you use that to
build the wheel - distlib can't rely on that information being available
because it's not part of the PEP.

If the PEP is updated to include the exports, they should be in the wheel no
matter which tool builds it. Then in theory distlib could generate the
scripts during installation, but there are a lot of options to consider -
did setuptools put them in there already? Do we want native launchers? etc.
which is perfectly doable in distlib, but I'm not sure that's the best place
for it because I think wheel processing should be uncomplicated.
Wheel.install already has quite a few kwargs:

dry_run=False: Don't actually do anything
executable=None:Custom executable for shebang lines (e.g. to support path
searching)
warner=None:Used to defer warning behaviour to calling application
lib_only=False:Process site-packages contents only (you suggested this)

I'd like to not have to add any more, unless it's unavoidable :-)

Nevertheless, I will probably try implementing it in distlib as an
experiment, too see how it looks.

 distlib processes the exports on install is not an issue to me (I think it
 always should). What is more of an issue is what the thing that puts stuff
 into the directories does. If that's a setuptools-based build, it will
 process the exports data *itself* and put wrappers into the scripts
 directory. That is what I think we should be trapping and suppressing. But
 we have to be careful, as if setuptools is used to install directly, *not*
 going via a wheel, it has to generate wrappers itself somehow.

Exactly why I'm so leery of putting this logic in distlib, until we think it
through and add it to the PEP. At the moment distil does it the same way as
setuptools/pip only to remain compatible, not for any other reason.

 Using distil to do the build is a whole other route.

Right. I'm aiming for distil to be able to do just about everything pip can
(functionally, the code is pretty much there barring installs from DVCS
URLs), but backward compatibility is always a concern and a challenge :-)

Another complication for distlib is that I expect it to work in 2.6+, where
you can't always rely on the py launcher being present - hence the wrappers
in distlib, with flags to disable writing them out.

Another area to consider for scripts is which of foo, fooX and foo-X.Y to
write to the scripts folder. This is particularly important in user
site-packages, where scripts for different Python versions will coexist in
the same folder, and the possibility exists of overwriting, which sometimes
leads to unexpected behaviour (e.g. if I install foo-dist using 2.x which
installs foo and foo-2.x to ~/.local/bin, then install it using 3.x, it
would write foo and foo-3.x to ~/.local/bin. Quite apart from headaches with
native launchers, it could be that the foo script installed with 3.x doesn't
work (e.g. if it tries to process Python code which is 2.x compatible but
not 3.x compatible).

Fun and games!

Regards,

Vinay Sajip

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Paul Moore
On 16 July 2013 22:41, Vinay Sajip vinay_sa...@yahoo.co.uk wrote:

 If the PEP is updated to include the exports, they should be in the wheel
 no
 matter which tool builds it. Then in theory distlib could generate the
 scripts during installation, but there are a lot of options to consider -
 did setuptools put them in there already? Do we want native launchers? etc.
 which is perfectly doable in distlib, but I'm not sure that's the best
 place
 for it because I think wheel processing should be uncomplicated.
 Wheel.install already has quite a few kwargs:


I really don't want the wrappers to be present in the wheel, because if
they are the wheel becomes architecture-specific. Also, consider that Unix
targets should have the actual scripts written with no extension, whereas
Windows targets should have foo-script.py and foo.exe. That should be
decided at install time, bot at wheel creation time.

As regards version-specific scripts, I'd assume it's the project's job to
specify precisely what scripts they want. On that one, I'm on the side of
providing infrastructure, not setting policy. Although I could be persuaded
otherwise if there was a PEP on what commands a distribution should
provide. In that case, let the project provide the command names, and let
the installer implement the standard versioned-executable policy.

Paul.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Vinay Sajip
I really don't want the wrappers to be present in the wheel, because if they 
are the wheel becomes architecture-specific. Also, consider that Unix targets 
should have the actual scripts written with no extension, whereas Windows 
targets should have foo-script.py and foo.exe. That should be decided at 
install time, bot at wheel creation time.

I think we're agreed on that as a desirable. I've already changed the distil 
code to omit writing launchers to the wheel, and that probably won't need to 
change. But it does generate the scripts for exports - to avoid doing that, we 
need to get the exports into the wheel in a standardised way (via pydist.json, 
perhaps, or perhaps a separate file). Iterating over the exports in a 
distribution path needs to be fast - note that the exports are not just 
scripts, and it's not yet clear whether the script exports need to be separated 
out from the rest.

As regards version-specific scripts, I'd assume it's the project's job to 
specify precisely what scripts they want. On that one, I'm on the side of 
providing infrastructure, not setting policy.

This sounds like you mean that distlib needs to stay basic 
(mechanism/infrastructure), and the installer needs to do the script generation 
(policy - perhaps controlled by the user). Currently, distlib allows one to 
specify foo with optional variants fooX and foo-X.Y (before factoring native 
launchers into the mix). This is set on the ScriptMaker instance which 
generates the scripts in the target directory (including shebang rewriting, 
native launchers etc.)

Regards,

Vinay Sajip
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Nick Coghlan
On 17 Jul 2013 04:19, Vinay Sajip vinay_sa...@yahoo.co.uk wrote:



 If I were writing a firm proposal, I'd go for something like entry
points as metadata

 My extended metadata already covers this, though I use the name exports
(suggested by PJE) because you can share not just code but data, and entry
points generally implies code. The current version of distil creates
wrappers for both gui and console scripts, and adds the appropriate native
executable wrappers (32- or 64-bit, according to the running Python) on
Windows.

Yeah, originally we were going to postpone dealing with entry points to a
metadata extension (Daniel even had a proto-PEP kicking around in the
pre-JSON days).

However, I now think it makes more sense to standardise them as an
exports field in PEP 426. So run with the assumption that something like
that will be part of the standard metadata - either derived from
entry_points.txt for existing metadata, or specified directly for next
generation metadata.

Cheers,
Nick.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Donald Stufft

On Jul 16, 2013, at 8:00 PM, Nick Coghlan ncogh...@gmail.com wrote:

 
 On 17 Jul 2013 04:19, Vinay Sajip vinay_sa...@yahoo.co.uk wrote:
 
 
 
  If I were writing a firm proposal, I'd go for something like entry points 
  as metadata
 
  My extended metadata already covers this, though I use the name exports 
  (suggested by PJE) because you can share not just code but data, and entry 
  points generally implies code. The current version of distil creates 
  wrappers for both gui and console scripts, and adds the appropriate native 
  executable wrappers (32- or 64-bit, according to the running Python) on 
  Windows.
 
 Yeah, originally we were going to postpone dealing with entry points to a 
 metadata extension (Daniel even had a proto-PEP kicking around in the 
 pre-JSON days).
 
 However, I now think it makes more sense to standardise them as an exports 
 field in PEP 426. So run with the assumption that something like that will be 
 part of the standard metadata - either derived from entry_points.txt for 
 existing metadata, or specified directly for next generation metadata.
 
 Cheers,
 Nick.
 ___
 Distutils-SIG maillist  -  Distutils-SIG@python.org
 http://mail.python.org/mailman/listinfo/distutils-sig

Are these only the scripts portion of entry points, or the whole kit and 
caboodle of pluggable entry points? Because I think the first makes sense, the 
second I'm hesitant on.

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig


Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-16 Thread Nick Coghlan
On 17 Jul 2013 10:03, Donald Stufft don...@stufft.io wrote:


 On Jul 16, 2013, at 8:00 PM, Nick Coghlan ncogh...@gmail.com wrote:


 On 17 Jul 2013 04:19, Vinay Sajip vinay_sa...@yahoo.co.uk wrote:
 
 
 
  If I were writing a firm proposal, I'd go for something like entry
points as metadata
 
  My extended metadata already covers this, though I use the name
exports (suggested by PJE) because you can share not just code but data,
and entry points generally implies code. The current version of distil
creates wrappers for both gui and console scripts, and adds the appropriate
native executable wrappers (32- or 64-bit, according to the running Python)
on Windows.

 Yeah, originally we were going to postpone dealing with entry points to
a metadata extension (Daniel even had a proto-PEP kicking around in the
pre-JSON days).

 However, I now think it makes more sense to standardise them as an
exports field in PEP 426. So run with the assumption that something like
that will be part of the standard metadata - either derived from
entry_points.txt for existing metadata, or specified directly for next
generation metadata.

 Cheers,
 Nick.

 ___

 Distutils-SIG maillist  -  Distutils-SIG@python.org
 http://mail.python.org/mailman/listinfo/distutils-sig


 Are these only the scripts portion of entry points, or the whole kit and
caboodle of pluggable entry points? Because I think the first makes sense,
the second I'm hesitant on.

Actually, it may be better to have a top level scripts field, distinct
from a general export mechanism.

I'm seeing value in an exports mechanism, though. Yes, *in theory* you can
get the same effect with an extension, but extensions can do a lot of other
things, too. Python's metaprogramming is built on a model of multiple tools
with increasing levels of power, flexibility and complexity, so I'm
thinking an exports vs extensions split may be a good approach in a similar
vein.

No decision on this front yet, but I think it's at least worth my trying it
out to see how it looks in the context of the PEP. (After all, we already
know entry points are quite a popular feature of the setuptools metadata)

A couple of bonus features of standardisation are that we can tie it into
the extras system and automatic analysis tools can check the exports can
actually be imported without needing to understand arbitrary extensions.

Cheers,
Nick.


 -
 Donald Stufft
 PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
DCFA

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig