Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support

2015-02-15 Thread Paul Moore 
On 15 February 2015 at 05:33, Nick Coghlan  wrote:
> I'd definitely prefer a simple procedural API like that over offering
> a public stateful object-oriented API, as the latter significantly
> increases testing complexity without offering a sufficiently
> compelling gain in expressiveness to justify the additional upfront
> test development cost and the ongoing maintenance and support burden.

The main advantage of a class API is the ability to support building
an archive from data in memory (or by collecting parts from different
areas on the filesystem). That's something I've had a use for on a
number of occasions.

OTOH, if the pyz/pyzw extensions are already (or will be) registered
by the installer then the only remaining feature of this PEP [1] is
the archive creation app. And designing that as we go is probably the
wrong way to get something in the stdlib. Maybe it would be better to
put something on PyPI and let it develop outside the stdlib first?
There's already at least pyzaa and pyzzer available...

Paul

[1] Apart from the benefit of publicising executable zip files as a
means of deploying Python code by mentioning them in a "What's new"
section :-)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support

2015-02-15 Thread Paul Moore 
On 15 February 2015 at 08:14, Paul Moore  wrote:
> Maybe it would be better to
> put something on PyPI and let it develop outside the stdlib first?

The only place where a ".pyz" file can't easily be manipulated with
the stdlib zipfile module is in writing a shebang line to the start of
the archive (i.e. adding "prefix" bytes before the start of the
zipfile data). It would be nice if the ZipFile class supported this
(because to do it properly you need access to the file object that the
ZipFile object wraps). Would it be reasonable to add methods to the
ZipFile class to read and write the prefix data?

Paul
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support

2015-02-15 Thread Nick Coghlan 
On 15 February 2015 at 18:14, Paul Moore  wrote:
> OTOH, if the pyz/pyzw extensions are already (or will be) registered
> by the installer then the only remaining feature of this PEP [1] is
> the archive creation app. And designing that as we go is probably the
> wrong way to get something in the stdlib. Maybe it would be better to
> put something on PyPI and let it develop outside the stdlib first?
> There's already at least pyzaa and pyzzer available...

Perhaps rather than creating yet another implementation of this, we
should just add some "See Also" links to the relevant parts of the
command line usage guide and the runpy module documentation?

Particularly if the modules are pip-installable and we add the
cross-references to the 2.7 and 3.4 docs as well - we didn't install
pip by default back when Daniel first wrote PEP 441.

> [1] Apart from the benefit of publicising executable zip files as a
> means of deploying Python code by mentioning them in a "What's new"
> section :-)

That will be covered to some degree by the mention of the file
associations being registered on Windows, and we can try to persuade
Larry it's worth mentioning in the release announcement itself :)

The other option would to cut PEP 441 way back to *just* be about
standardising and registering the file associations, and recommending
the use of pip to obtain the build machinery with (whether pyzaa,
pyzzer or Twitter's more comprehensive pex). It would be a short PEP,
but potentially still worth it for the improved visibility of the
decision when folks are trying to figure out what "pyz" and "pyzw"
files are later.

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support

2015-02-15 Thread Paul Moore 
On 15 February 2015 at 08:59, Nick Coghlan  wrote:
> The other option would to cut PEP 441 way back to *just* be about
> standardising and registering the file associations, and recommending
> the use of pip to obtain the build machinery with (whether pyzaa,
> pyzzer or Twitter's more comprehensive pex). It would be a short PEP,
> but potentially still worth it for the improved visibility of the
> decision when folks are trying to figure out what "pyz" and "pyzw"
> files are later.

Ok, thinking about this a little more.

Getting the extension support is the key thing on Windows - at the
moment, people are faced with adding their own file associations or
putting binary data in a .py file, neither of which is a nice choice.
Tooling is important, though - sure, you can zip the data up and put a
header on, but it's fiddly.

Which brings us full circle. A simple module, executable as "python -m
zipapp" (see below re name) which exports a single function, pack()
that creates the archive. If we want to provide a script to wrap the
module, like pyvenv.py does for venv, I've no objection to that -
presumably it would go in Tools/Scripts? If people (like me) want to
experiment with a more programmatic API for building pyz files, they
can do so on PyPI, and if such a thing becomes sufficiently mature we
can look then at proposing it for inclusion in the stdlib, as an
extension to the zipapp module.

Regarding naming, I'm happy to go with zipapp if it's your preference.
Presumably the wrapper in Tools/Scripts would be pyzipapp.py?

So the usage would be something like

python -m zipapp [options] dir_to_zip

Options:
-p The interpreter to use on the shebang line
(defaulting to /usr/bin/env python)
-o archive_name The name of the output file (defaulting to the
source directory name with a .pyz extension)
If the argument has no extension, add '.pyz'
-m module:function  The entry point to call (written to __main__.py)
Using this is an error if there is a
__main__.py, and mandatory if there isn't

If you want anything more complex, it's easy enough to write your own
script based on zipfile, or use one of the modules on PyPI.

Does this sound reasonable? If it's OK, I'll go ahead and prepare an
update to the PEP and an implementation. (Steve, looks like I may be
learning how to maintain the wix files after all - wish me luck :-))
If I hear no objections in the next couple of days, I'll assume
everyone's OK with it and I'll prepare a PEP update and a patch.

Paul
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 486: Make the Python Launcher aware of virtual environments

2015-02-15 Thread Paul Moore 
On 13 February 2015 at 07:02, Nick Coghlan  wrote:
> Procedurally, since it's a Windows specific change, and assuming Guido
> doesn't want to pronounce on this one himself, perhaps Steve Dower
> would be an appropriate BDFL-Delegate?

The PEP is now up to date and the patch is on the tracker at
http://bugs.python.org/issue23465. There's not been any further
questions or comments, so I think I'm happy with it. As Guido's now
away (I gather) I'll wait till he gets back and then post a formal
request for pronouncement (or appointment of a BDFL-Delegate to
pronounce).

If anyone has any issues with the PEP, please speak up before then!

Paul
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support

2015-02-15 Thread Nick Coghlan 
On 15 February 2015 at 23:21, Paul Moore  wrote:
> On 15 February 2015 at 08:59, Nick Coghlan  wrote:
>> The other option would to cut PEP 441 way back to *just* be about
>> standardising and registering the file associations, and recommending
>> the use of pip to obtain the build machinery with (whether pyzaa,
>> pyzzer or Twitter's more comprehensive pex). It would be a short PEP,
>> but potentially still worth it for the improved visibility of the
>> decision when folks are trying to figure out what "pyz" and "pyzw"
>> files are later.
>
> Ok, thinking about this a little more.
>
> Getting the extension support is the key thing on Windows - at the
> moment, people are faced with adding their own file associations or
> putting binary data in a .py file, neither of which is a nice choice.
> Tooling is important, though - sure, you can zip the data up and put a
> header on, but it's fiddly.
>
> Which brings us full circle. A simple module, executable as "python -m
> zipapp" (see below re name) which exports a single function, pack()
> that creates the archive. If we want to provide a script to wrap the
> module, like pyvenv.py does for venv, I've no objection to that -
> presumably it would go in Tools/Scripts? If people (like me) want to
> experiment with a more programmatic API for building pyz files, they
> can do so on PyPI, and if such a thing becomes sufficiently mature we
> can look then at proposing it for inclusion in the stdlib, as an
> extension to the zipapp module.
>
> Regarding naming, I'm happy to go with zipapp if it's your preference.
> Presumably the wrapper in Tools/Scripts would be pyzipapp.py?

Or we just skip the wrapper and make "python -m zipapp" the
recommended invocation style. Adding a wrapper later is fairly easy,
but removing it would be difficult.

>
> So the usage would be something like
>
> python -m zipapp [options] dir_to_zip
>
> Options:
> -p The interpreter to use on the shebang line
> (defaulting to /usr/bin/env python)
> -o archive_name The name of the output file (defaulting to the
> source directory name with a .pyz extension)
> If the argument has no extension, add '.pyz'
> -m module:function  The entry point to call (written to __main__.py)
> Using this is an error if there is a
> __main__.py, and mandatory if there isn't
>
> If you want anything more complex, it's easy enough to write your own
> script based on zipfile, or use one of the modules on PyPI.
>
> Does this sound reasonable? If it's OK, I'll go ahead and prepare an
> update to the PEP and an implementation. (Steve, looks like I may be
> learning how to maintain the wix files after all - wish me luck :-))
> If I hear no objections in the next couple of days, I'll assume
> everyone's OK with it and I'll prepare a PEP update and a patch.

Sounds good to me.

Daniel, do you mind if Paul becomes a co-author on PEP 441 and updates
it as described? It seems a bit tidier than allocating a new PEP
number and rejecting PEP 441, when the revised proposal is essentially
just a simplified and renamed version of your original idea.

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support

2015-02-15 Thread Paul Moore 
On 15 February 2015 at 13:47, Nick Coghlan  wrote:
> Or we just skip the wrapper and make "python -m zipapp" the
> recommended invocation style. Adding a wrapper later is fairly easy,
> but removing it would be difficult.

Fine with me - the wrappers are basically useless on Windows where
Tools/Scripts isn't on PATH.
Paul
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support

2015-02-15 Thread Serhiy Storchaka 

On 15.02.15 10:47, Paul Moore wrote:

On 15 February 2015 at 08:14, Paul Moore  wrote:

Maybe it would be better to
put something on PyPI and let it develop outside the stdlib first?


The only place where a ".pyz" file can't easily be manipulated with
the stdlib zipfile module is in writing a shebang line to the start of
the archive (i.e. adding "prefix" bytes before the start of the
zipfile data). It would be nice if the ZipFile class supported this
(because to do it properly you need access to the file object that the
ZipFile object wraps). Would it be reasonable to add methods to the
ZipFile class to read and write the prefix data?


But the stdlib zipfile module supports this.

with open(filename, 'wb') as f:
f.write(shebang)
with zipfile.PyZipFile(f, 'a') as zf:
...


___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support

2015-02-15 Thread Thomas Wouters 
FWIW, we have a similar tool to 'pyzaa' at Google, although a lot older (it
has a pure-python implementation of zipimport, because it had to work with
Python 2.2 back in the day) that *does* support extension modules and other
shared libraries (by extracting them to disk on program startup.) We're
also working to replace that by loading the extension modules directly from
the archive (using dlopen_with_offset as described by ppluzhnikov in
https://sourceware.org/bugzilla/show_bug.cgi?id=11767, although it should
also be possible using dlopen_ehdr/dlopen_phdr), which requires that
extension modules are stored uncompressed (simple) and page-aligned
(harder, as the zipfile.ZipFile class doesn't directly support
page-aligning anything, but it turns out it's easy to hack around by
overriding _writecheck :P) And yes, we really are in a position to modify
glibc to make our life distributing Python applications easier, internally;
the old code involves shell and Python bootstrap code that tries very hard
to avoid race conditions and security problems, which is a pain to maintain.

It might be nice to consider those use-cases in pyzapp as well, especially
once the glibc feature is released. It requires some fairly big changes to
zipimport (I ended up rewriting the whole thing) but we can easily
opensource the code, if there's any interest at all. I'd be happy to
discuss this in more detail at PyCon, at which time we should be deploying
code internally using all of this.

-- 
Thomas Wouters 

Hi! I'm an email virus! Think twice before sending your email to help me
spread!
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support

2015-02-15 Thread Paul Moore 
On 15 February 2015 at 16:00, Serhiy Storchaka  wrote:
>> Would it be reasonable to add methods to the
>> ZipFile class to read and write the prefix data?
>
>
> But the stdlib zipfile module supports this.
>
> with open(filename, 'wb') as f:
> f.write(shebang)
> with zipfile.PyZipFile(f, 'a') as zf:

Good point.

It's not quite as easy to read the prefix data. I think you can do it
by reopening the file, and reading the bytes up to byte
min(i.header_offset for i in zf.infolist()). You need to reopen the
file, though, and that calculation doesn't work for an empty zipfile
(where you need to read to the start of the central directory
instead). But it's sort of possible, and it's a pretty specialised
requirement anyway (I only really needed it for testing).

Thanks,
Paul
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support

2015-02-15 Thread Paul Moore 
On 15 February 2015 at 16:15, Petr Viktorin  wrote:
> On Sun, Feb 15, 2015 at 2:21 PM, Paul Moore  wrote:
>> So the usage would be something like
>>
>> python -m zipapp [options] dir_to_zip
>>
>> Options:
>> -p The interpreter to use on the shebang line
>> (defaulting to /usr/bin/env python)
>
> On many systems this default would mean Python 2. Even if the official
> recommendation changes for 3.5, the status quo might linger for a few
> years.
> On the other hand, the number of distros that don't ship Python 3 is
> small, and the reason they're slow-moving tends to be stability and/or
> compliance, so they're not the target audience for zipapp. And the
> python3 symlink is not going away any time soon.
> So I'd suggest `/usr/bin/env python3` for the default.

But that will fail for users who only have Python 2 installed. And it
won't pick up an activated virtualenv (assuming virtualenvs on Unix
don't include a python3 command, which I believe is true - it
certainly won't on Windows).

As a Windows user, I believe that the command "python" should run the
version of Python that you choose to be your default. I know it's not
quite that simple on Unix, and the system "python" command is
typically Python 2, with "python3" for the system Python 3 version,
but I also know that tools like pyenv work like that. Equally, I've
seen lots of scripts with "#!/usr/bin/env python" as the shebang line,
and none with "#!/usr/bin/env python3". That may just be my limited
experience with Unix though.

I don't really want "#!/usr/bin/env python3" as the default shebang on
Windows (at a minimum, it would do the wrong thing for my current
environment). I'm open to advice from the Unix users as to how to set
things up better on Unix, but there's a whole new set of problems that
will arise as soon as we have different defaults on Unix and Windows,
so IMO there would need to be pretty significant benefits to justify
doing that.

And of course it *is* only a default - users can set whatever they
want. (But getting defaults right is important, so that's not to
dismiss the point).

Paul
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support

2015-02-15 Thread Paul Moore 
On 15 February 2015 at 16:21, Thomas Wouters  wrote:
> It might be nice to consider those use-cases in pyzapp as well, especially
> once the glibc feature is released. It requires some fairly big changes to
> zipimport (I ended up rewriting the whole thing) but we can easily
> opensource the code, if there's any interest at all. I'd be happy to discuss
> this in more detail at PyCon, at which time we should be deploying code
> internally using all of this.

Those cases sound more like things to consider for Python's
"executable zip" support. The zipapp module is purely concerned with
taking a bunch of stuff and making a zipfile-with-shebang
"python-executable zipfile" out of it. Of course if Python needed
certain files to be aligned a particular way in order to execute the
zipfile, that *would* be something that zipapp should take into
account. But I'd say let's wait till zipfile execution needs the
feature before adding it to zipapp.

Paul
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support

2015-02-15 Thread Petr Viktorin 
On Sun, Feb 15, 2015 at 2:21 PM, Paul Moore  wrote:
> So the usage would be something like
>
> python -m zipapp [options] dir_to_zip
>
> Options:
> -p The interpreter to use on the shebang line
> (defaulting to /usr/bin/env python)

On many systems this default would mean Python 2. Even if the official
recommendation changes for 3.5, the status quo might linger for a few
years.
On the other hand, the number of distros that don't ship Python 3 is
small, and the reason they're slow-moving tends to be stability and/or
compliance, so they're not the target audience for zipapp. And the
python3 symlink is not going away any time soon.
So I'd suggest `/usr/bin/env python3` for the default.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support

2015-02-15 Thread Daniel Holth 
Go ahead, make my pep.

I will appreciate seeing it happen.
On Feb 15, 2015 8:47 AM, "Nick Coghlan"  wrote:

> On 15 February 2015 at 23:21, Paul Moore  wrote:
> > On 15 February 2015 at 08:59, Nick Coghlan  wrote:
> >> The other option would to cut PEP 441 way back to *just* be about
> >> standardising and registering the file associations, and recommending
> >> the use of pip to obtain the build machinery with (whether pyzaa,
> >> pyzzer or Twitter's more comprehensive pex). It would be a short PEP,
> >> but potentially still worth it for the improved visibility of the
> >> decision when folks are trying to figure out what "pyz" and "pyzw"
> >> files are later.
> >
> > Ok, thinking about this a little more.
> >
> > Getting the extension support is the key thing on Windows - at the
> > moment, people are faced with adding their own file associations or
> > putting binary data in a .py file, neither of which is a nice choice.
> > Tooling is important, though - sure, you can zip the data up and put a
> > header on, but it's fiddly.
> >
> > Which brings us full circle. A simple module, executable as "python -m
> > zipapp" (see below re name) which exports a single function, pack()
> > that creates the archive. If we want to provide a script to wrap the
> > module, like pyvenv.py does for venv, I've no objection to that -
> > presumably it would go in Tools/Scripts? If people (like me) want to
> > experiment with a more programmatic API for building pyz files, they
> > can do so on PyPI, and if such a thing becomes sufficiently mature we
> > can look then at proposing it for inclusion in the stdlib, as an
> > extension to the zipapp module.
> >
> > Regarding naming, I'm happy to go with zipapp if it's your preference.
> > Presumably the wrapper in Tools/Scripts would be pyzipapp.py?
>
> Or we just skip the wrapper and make "python -m zipapp" the
> recommended invocation style. Adding a wrapper later is fairly easy,
> but removing it would be difficult.
>
> >
> > So the usage would be something like
> >
> > python -m zipapp [options] dir_to_zip
> >
> > Options:
> > -p The interpreter to use on the shebang line
> > (defaulting to /usr/bin/env python)
> > -o archive_name The name of the output file (defaulting to the
> > source directory name with a .pyz extension)
> > If the argument has no extension, add '.pyz'
> > -m module:function  The entry point to call (written to __main__.py)
> > Using this is an error if there is a
> > __main__.py, and mandatory if there isn't
> >
> > If you want anything more complex, it's easy enough to write your own
> > script based on zipfile, or use one of the modules on PyPI.
> >
> > Does this sound reasonable? If it's OK, I'll go ahead and prepare an
> > update to the PEP and an implementation. (Steve, looks like I may be
> > learning how to maintain the wix files after all - wish me luck :-))
> > If I hear no objections in the next couple of days, I'll assume
> > everyone's OK with it and I'll prepare a PEP update and a patch.
>
> Sounds good to me.
>
> Daniel, do you mind if Paul becomes a co-author on PEP 441 and updates
> it as described? It seems a bit tidier than allocating a new PEP
> number and rejecting PEP 441, when the revised proposal is essentially
> just a simplified and renamed version of your original idea.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   [email protected]   |   Brisbane, Australia
>
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support

2015-02-15 Thread Steve Dower 
"Go ahead, make my pep."

We should make a python-dev t-shirt with this on it :)

Top-posted from my Windows Phone

From: Daniel Holth
Sent: ‎2/‎15/‎2015 9:46
To: Nick Coghlan
Cc: Paul Moore; Steve 
Dower; Python 
Dev
Subject: Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support


Go ahead, make my pep.

I will appreciate seeing it happen.

On Feb 15, 2015 8:47 AM, "Nick Coghlan" 
mailto:[email protected]>> wrote:
On 15 February 2015 at 23:21, Paul Moore 
mailto:[email protected]>> wrote:
> On 15 February 2015 at 08:59, Nick Coghlan 
> mailto:[email protected]>> wrote:
>> The other option would to cut PEP 441 way back to *just* be about
>> standardising and registering the file associations, and recommending
>> the use of pip to obtain the build machinery with (whether pyzaa,
>> pyzzer or Twitter's more comprehensive pex). It would be a short PEP,
>> but potentially still worth it for the improved visibility of the
>> decision when folks are trying to figure out what "pyz" and "pyzw"
>> files are later.
>
> Ok, thinking about this a little more.
>
> Getting the extension support is the key thing on Windows - at the
> moment, people are faced with adding their own file associations or
> putting binary data in a .py file, neither of which is a nice choice.
> Tooling is important, though - sure, you can zip the data up and put a
> header on, but it's fiddly.
>
> Which brings us full circle. A simple module, executable as "python -m
> zipapp" (see below re name) which exports a single function, pack()
> that creates the archive. If we want to provide a script to wrap the
> module, like pyvenv.py does for venv, I've no objection to that -
> presumably it would go in Tools/Scripts? If people (like me) want to
> experiment with a more programmatic API for building pyz files, they
> can do so on PyPI, and if such a thing becomes sufficiently mature we
> can look then at proposing it for inclusion in the stdlib, as an
> extension to the zipapp module.
>
> Regarding naming, I'm happy to go with zipapp if it's your preference.
> Presumably the wrapper in Tools/Scripts would be pyzipapp.py?

Or we just skip the wrapper and make "python -m zipapp" the
recommended invocation style. Adding a wrapper later is fairly easy,
but removing it would be difficult.

>
> So the usage would be something like
>
> python -m zipapp [options] dir_to_zip
>
> Options:
> -p The interpreter to use on the shebang line
> (defaulting to /usr/bin/env python)
> -o archive_name The name of the output file (defaulting to the
> source directory name with a .pyz extension)
> If the argument has no extension, add '.pyz'
> -m module:function  The entry point to call (written to __main__.py)
> Using this is an error if there is a
> __main__.py, and mandatory if there isn't
>
> If you want anything more complex, it's easy enough to write your own
> script based on zipfile, or use one of the modules on PyPI.
>
> Does this sound reasonable? If it's OK, I'll go ahead and prepare an
> update to the PEP and an implementation. (Steve, looks like I may be
> learning how to maintain the wix files after all - wish me luck :-))
> If I hear no objections in the next couple of days, I'll assume
> everyone's OK with it and I'll prepare a PEP update and a patch.

Sounds good to me.

Daniel, do you mind if Paul becomes a co-author on PEP 441 and updates
it as described? It seems a bit tidier than allocating a new PEP
number and rejecting PEP 441, when the revised proposal is essentially
just a simplified and renamed version of your original idea.

Cheers,
Nick.

--
Nick Coghlan   |   [email protected]   |   
Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support

2015-02-15 Thread Petr Viktorin 
On Sun, Feb 15, 2015 at 5:43 PM, Paul Moore  wrote:
> On 15 February 2015 at 16:15, Petr Viktorin  wrote:
>> On Sun, Feb 15, 2015 at 2:21 PM, Paul Moore  wrote:
>>> So the usage would be something like
>>>
>>> python -m zipapp [options] dir_to_zip
>>>
>>> Options:
>>> -p The interpreter to use on the shebang line
>>> (defaulting to /usr/bin/env python)
>>
>> On many systems this default would mean Python 2. Even if the official
>> recommendation changes for 3.5, the status quo might linger for a few
>> years.
>> On the other hand, the number of distros that don't ship Python 3 is
>> small, and the reason they're slow-moving tends to be stability and/or
>> compliance, so they're not the target audience for zipapp. And the
>> python3 symlink is not going away any time soon.
>> So I'd suggest `/usr/bin/env python3` for the default.
>
> But that will fail for users who only have Python 2 installed. And it
> won't pick up an activated virtualenv (assuming virtualenvs on Unix
> don't include a python3 command, which I believe is true - it
> certainly won't on Windows).

Yes, it would fail with only Python 2 installed. I don't think that's a problem.

On POSIXy systems the "python3" symlink is created in all venvs. I
thought (perhaps naïvely) that Windows doesn't do shebangs natively,
so there's some Python-specific mechanism around them, which should
handle "python3".

If the app was single-source compatible with both 2 and 3, then
`/usr/bin/env python` would be a good choice. But I don't think apps
(as opposed to libraries) have much incentive to keep the backwards
compatibility.

> As a Windows user, I believe that the command "python" should run the
> version of Python that you choose to be your default. I know it's not
> quite that simple on Unix, and the system "python" command is
> typically Python 2, with "python3" for the system Python 3 version,
> but I also know that tools like pyenv work like that. Equally, I've
> seen lots of scripts with "#!/usr/bin/env python" as the shebang line,
> and none with "#!/usr/bin/env python3". That may just be my limited
> experience with Unix though.

On Linux, what version "python" is depends on the distro. Currently
the recommendation from Python upstream is for it to be Python 2,
though, and I know of just one distro that does not do this (Arch,
because it switched before the recommendation was put in place).

If you're in a venv, then of "python" is that venv's Python, of
course. But the default mode of operation for Python-unaware users is
to not be in a venv.

> I don't really want "#!/usr/bin/env python3" as the default shebang on
> Windows (at a minimum, it would do the wrong thing for my current
> environment). I'm open to advice from the Unix users as to how to set
> things up better on Unix, but there's a whole new set of problems that
> will arise as soon as we have different defaults on Unix and Windows,
> so IMO there would need to be pretty significant benefits to justify
> doing that.

I'm not familiar with how the shebang works on Windows, but if
"/usr/bin/env python3" breaks things, I find it highly unfortunate.

> And of course it *is* only a default - users can set whatever they
> want. (But getting defaults right is important, so that's not to
> dismiss the point).

I'm afraid the point is whether it's even possible to have a shebang
that does the right thing on both platforms.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support

2015-02-15 Thread Serhiy Storchaka 

On 15.02.15 18:21, Thomas Wouters wrote:

which requires that extension modules are stored uncompressed (simple)
and page-aligned (harder, as the zipfile.ZipFile class doesn't directly
support page-aligning anything


It is possible to add this feature to ZipFile. It can be useful because 
will allow to mmap uncompressed files in ZIP file.


___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 370 - per-user scripts directory on Windows

2015-02-15 Thread Vinay Sajip 
Paul Moore  gmail.com> writes:

> 2. It would be a nice, although extremely obscure, feature to be able
> to allow people who want to keep Python off their path to set
> VIRTUAL_ENV but *not* set PATH, so that py.exe does the right thing
> but there's *still* no python.exe on PATH. Limit the dependency on a
> single environment variable rather than two, in other words.

If this is the case, then your patch may not work if PATH isn't set. If a 
shebang is found in a script, it is processed in maybe_handle_shebang(), which 
means that your code to look in a venv won't be hit (though if you rely on the 
PATH being correctly set, then it will work as expected because the handling 
for e.g. /usr/bin/env python searches the PATH).

Regards,

Vinay Sajip

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support

2015-02-15 Thread Mark Lawrence 

On 15/02/2015 18:06, Steve Dower wrote:

"Go ahead, make my pep."

We should make a python-dev t-shirt with this on it :)



I'll buy one provided p&p isn't too high :)

--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 370 - per-user scripts directory on Windows

2015-02-15 Thread Paul Moore 
On 15 February 2015 at 19:23, Vinay Sajip  wrote:
> Paul Moore  gmail.com> writes:
>
>> 2. It would be a nice, although extremely obscure, feature to be able
>> to allow people who want to keep Python off their path to set
>> VIRTUAL_ENV but *not* set PATH, so that py.exe does the right thing
>> but there's *still* no python.exe on PATH. Limit the dependency on a
>> single environment variable rather than two, in other words.
>
> If this is the case, then your patch may not work if PATH isn't set. If a
> shebang is found in a script, it is processed in maybe_handle_shebang(), which
> means that your code to look in a venv won't be hit (though if you rely on the
> PATH being correctly set, then it will work as expected because the handling
> for e.g. /usr/bin/env python searches the PATH).

Yes, when I was actually coding this, I backed down on this (I updated
the PEP accordingly). Now, running py.exe with no script will pick up
VIRTUAL_ENV, but when you run a script, you need the virtualenv to be
activated for this to work.

The original PEP said nothing about a PATH search for #!/usr/bin/env
python, so I mistakenly assumed there was no way currently to write a
script that worked with an activated virtualenv. When I found that the
code did this, I decided it was unnecessary to add extra tests and
complexity just to cater for a case that nobody was ever likely to
need.

Paul
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 370 - per-user scripts directory on Windows

2015-02-15 Thread Vinay Sajip 
- Original Message -

From: Paul Moore 


> The original PEP said nothing about a PATH search for #!/usr/bin/env
> python, so I mistakenly assumed there was no way currently to write a
> script that worked with an activated virtualenv.


Actually the path search was added later, by you! See [1] and [2].

Regards,

Vinay Sajip

[1] http://bugs.python.org/issue17903
[2] 
https://bitbucket.org/pmoore/pylauncher/commits/87d41c5a8f0fef5ef22f09570903e7aa4c850e96
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 370 - per-user scripts directory on Windows

2015-02-15 Thread Paul Moore 
On 15 February 2015 at 21:00, Vinay Sajip  wrote:
>> The original PEP said nothing about a PATH search for #!/usr/bin/env
>> python, so I mistakenly assumed there was no way currently to write a
>> script that worked with an activated virtualenv.
>
>
> Actually the path search was added later, by you! See [1] and [2].

Coo, that was clever of me :-) I guess I'll have to admit to going senile now...
Paul
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 370 - per-user scripts directory on Windows

2015-02-15 Thread Chris Angelico 
On Sat, Feb 14, 2015 at 11:07 AM, Nick Coghlan  wrote:
> OTOH, it may be time to reconsider our recommendation to distros as well,
> suggesting that for Python 3.5+, we will consider it appropriate to have the
> "python" symlink refer to "python3".

If *all* distros provide a "python2" symlink, then the recommendation
can become "use python if it's 2/3 compatible, or python2/python3 to
choose", and then it won't hurt to switch python to be python3. Are
there known distros in which current versions (or, those that will be
current when 3.5 is released) aren't providing "python2"?

ChrisA
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com