Re: [Distutils] PEP: Improving Python ZIP Application Support
On Sun, Mar 31, 2013 at 9:07 AM, Brett Cannon wrote: > I would suggest allowing multiple versions for compilation (when > specified). There should also be a check that people don't specify multiple > versions that can't exist on the same directory (e.g 2.6 and 2.7). > Otherwise the compileall module's CLI handles this and people can call it > with different interpreters as necessary. > > IOW I'm advocating KISS for what the tool does. Since making the zip file > is only really tricky bit it should only handle that case. Heck you can > make it part of the zipfile module if there is resistance to adding yet > another script that python installs (obviously will need a Cheeseshop > package for older versions). > > Otherwise I like everything else. > Agreed. However, Tres is right that this is a python-dev PEP rather than a distutils-sig one :) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Help with buildout
I'm trying to use buildout to pull in 4 github repositories and combine their wsgi applications together using paste's urlmap. A few things aren't working the way I expect and so I'm looking for some more information: This is my buildout.cfg: https://github.com/sontek/notaliens.com/blob/master/buildout.cfg 1. I had to declare a [pyramid] section that pulls in Pyramid with the zc.recipe.egg recipe so that it would drop the entry_point scripts in the generated bin/ folder. Is there a way for it to do this from dependencies declared in my install_requires in setup.py rather than being declared in the buildout? 2. The 2nd problem is after I run buildout, I'm not able to import paste.deploy: I see the egg in my sys.path: bin/py >>> import sys >>> sys.path '/home/sontek/code/test_notaliens/ notaliens.com/eggs/PasteDeploy-1.5.0-py2.7.egg', but if I try to import from it: >>> from paste.deploy import loadserver Traceback (most recent call last): File "", line 1, in ImportError: No module named deploy PasteDeploy is defined as a dependency and gets pulled in properly but for some reason the interpreter isn't seeing it. any ideas? ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP: Improving Python ZIP Application Support
On Sat, Mar 30, 2013 at 7:07 PM, Brett Cannon wrote: > > On Mar 30, 2013 3:23 PM, "Daniel Holth" wrote: >> >> Python ZIP Application Support - >> >> https://docs.google.com/document/d/1MKXgPzhWD5wIUpoSQX7dxmqgTZVO6l9iZZis8dnri78/edit?usp=sharing >> >> >> PEP: 4XX >> >> Title: Improving Python ZIP Application Support >> >> Author: Daniel Holth >> >> Status: Draft >> >> Type: Standards Track >> >> Python-Version: 3.4 >> >> Created: 30 March 2013 >> >> Post-History: 30 March 2013 >> >> >> Improving Python ZIP Application Support >> >> >> Python has had the ability to execute directories or ZIP-format >> archives as scripts since version 2.6. When invoked with a zip file or >> directory as its first argument the interpreter adds that directory to >> sys.path and executes the __main__ module. These archives provide a >> great way to publish software that needs to be distributed as a single >> file script but is complex enough to need to be written as a >> collection of modules. >> >> >> This feature is not as popular as it should be for two reasons: a) >> users haven’t heard of it because it wasn’t mentioned in earlier >> versions of Python 2.6 “What’s New” document, and b) Windows users >> don’t have a file extension (other than .py) to associate with the >> launcher. >> >> >> This PEP proposes to fix these problems by re-publicising the feature, >> defining the .pyz and .pyzw extensions as “Python ZIP applications” >> and “Windowed Python Zip Applications”, and providing some simple >> tooling to manage the format. >> >> A New Python ZIP Application Extension >> >> >> The Python 3.4 installer will associate .pyz and .pyzw “Python ZIP >> Applications” with itself so they can be executed by the Windows >> launcher. A .pyz archive is a console application and a .pyzw archive >> is a windowed application. This indicates whether the console should >> appear when running the app. >> >> >> Why not use .zip or .py? Users expect a .zip file would be opened with >> an archive tool, and users expect .py to be opened with a text editor. >> Both would be confusing for this use case. >> >> >> For UNIX users, .pyz applications should be prefixed with a #! line >> pointing to the correct Python interpreter and an optional >> explanation. >> >> >> #!/usr/bin/env python >> >> # This is a Python application stored in a ZIP archive. > > ... built using pyzaa. > >> >> (binary contents of archive) >> >> >> As background, ZIP archives are defined with a footer containing >> relative offsets from the end of the file. They remain valid when >> concatenated to the end of any other file. This feature is completely >> standard and is how self-extracting ZIP archives and the bdist_wininst >> installer format work. >> >> Minimal Tooling: The pyzaa Module >> >> This PEP also proposes including a simple application for working with >> Python ZIP Archives: The Python Zip Application Archiver “pyzaa” >> (rhymes with “huzzah” or “pizza”). “pyzaa” can archive or extract >> these files, compile bytecode, and can write the __main__ module if it >> is not present. >> >> Usage >> >> python -m pyzaa (pack | unpack | compile) >> >> >> python -m pyzaa pack [-o path/name] [-m module.submodule:callable] >> [-c] [-w] [-p interpreter] directory: >> >> ZIP the contents of directory as directory.pyz or [-w] >> directory.pyzw. Adds the executable flag to the archive. >> >> -c compile .pyc files and add them to the archive >> >> -p interpreter include #!interpreter as the first line of the archive > > Would `/usr/bin/env python` (or python3 depending on interpreter used to > compile) be set otherwise? Or how about the specific python version to > prevent possible future-compatibility issues (e.g. specifying python3.3)? > >> >> -o path/name archive is written to path/name.pyz[w] instead of >> dirname. The extension is added if not specified. >> >> -m module.submodule:callable __main__.py is written as “import >> module.submodule; module.submodule.callable()” >> >> >> pyzaa pack will warn if the directory contains C extensions or if >> it doesn’t contain __main__.py. >> >> >> python -m pyzaa unpack arcname.pyz[w] >> >> The archive is unpacked into a directory [arcname] > > Is this truly necessary? If it's a zip file any archiving tool can unzip it. > Heck, we can add a CLI to the ziofile module if it doesn't already have one. It is a convenience so that the contents don't wind up in $PWD, I like the idea of dropping it though. I'll be sure to emphasize that these are completely standard ZIP archives with a new extension. I'm pretty sure zipfile has a CLI. The pack command is a convenience on top of zip & cat. Hope we don't need ignore / MANIFEST.in type features... >> >> >> python -m pyzaa compile arcname.pyz[w] >> >> The Python files in arcname.pyz[w] are compiled and appended to >> the ZIP file. > > I would suggest allowing multiple versions for compilation (when specified). > There should also be a check that people do
Re: [Distutils] PEP: Improving Python ZIP Application Support
On Mar 30, 2013 3:23 PM, "Daniel Holth" wrote: > > Python ZIP Application Support - > https://docs.google.com/document/d/1MKXgPzhWD5wIUpoSQX7dxmqgTZVO6l9iZZis8dnri78/edit?usp=sharing > > > PEP: 4XX > > Title: Improving Python ZIP Application Support > > Author: Daniel Holth > > Status: Draft > > Type: Standards Track > > Python-Version: 3.4 > > Created: 30 March 2013 > > Post-History: 30 March 2013 > > > Improving Python ZIP Application Support > > > Python has had the ability to execute directories or ZIP-format > archives as scripts since version 2.6. When invoked with a zip file or > directory as its first argument the interpreter adds that directory to > sys.path and executes the __main__ module. These archives provide a > great way to publish software that needs to be distributed as a single > file script but is complex enough to need to be written as a > collection of modules. > > > This feature is not as popular as it should be for two reasons: a) > users haven’t heard of it because it wasn’t mentioned in earlier > versions of Python 2.6 “What’s New” document, and b) Windows users > don’t have a file extension (other than .py) to associate with the > launcher. > > > This PEP proposes to fix these problems by re-publicising the feature, > defining the .pyz and .pyzw extensions as “Python ZIP applications” > and “Windowed Python Zip Applications”, and providing some simple > tooling to manage the format. > > A New Python ZIP Application Extension > > > The Python 3.4 installer will associate .pyz and .pyzw “Python ZIP > Applications” with itself so they can be executed by the Windows > launcher. A .pyz archive is a console application and a .pyzw archive > is a windowed application. This indicates whether the console should > appear when running the app. > > > Why not use .zip or .py? Users expect a .zip file would be opened with > an archive tool, and users expect .py to be opened with a text editor. > Both would be confusing for this use case. > > > For UNIX users, .pyz applications should be prefixed with a #! line > pointing to the correct Python interpreter and an optional > explanation. > > > #!/usr/bin/env python > > # This is a Python application stored in a ZIP archive. ... built using pyzaa. > > (binary contents of archive) > > > As background, ZIP archives are defined with a footer containing > relative offsets from the end of the file. They remain valid when > concatenated to the end of any other file. This feature is completely > standard and is how self-extracting ZIP archives and the bdist_wininst > installer format work. > > Minimal Tooling: The pyzaa Module > > This PEP also proposes including a simple application for working with > Python ZIP Archives: The Python Zip Application Archiver “pyzaa” > (rhymes with “huzzah” or “pizza”). “pyzaa” can archive or extract > these files, compile bytecode, and can write the __main__ module if it > is not present. > > Usage > > python -m pyzaa (pack | unpack | compile) > > > python -m pyzaa pack [-o path/name] [-m module.submodule:callable] > [-c] [-w] [-p interpreter] directory: > > ZIP the contents of directory as directory.pyz or [-w] > directory.pyzw. Adds the executable flag to the archive. > > -c compile .pyc files and add them to the archive > > -p interpreter include #!interpreter as the first line of the archive Would `/usr/bin/env python` (or python3 depending on interpreter used to compile) be set otherwise? Or how about the specific python version to prevent possible future-compatibility issues (e.g. specifying python3.3)? > > -o path/name archive is written to path/name.pyz[w] instead of > dirname. The extension is added if not specified. > > -m module.submodule:callable __main__.py is written as “import > module.submodule; module.submodule.callable()” > > > pyzaa pack will warn if the directory contains C extensions or if > it doesn’t contain __main__.py. > > > python -m pyzaa unpack arcname.pyz[w] > > The archive is unpacked into a directory [arcname] Is this truly necessary? If it's a zip file any archiving tool can unzip it. Heck, we can add a CLI to the ziofile module if it doesn't already have one. > > > python -m pyzaa compile arcname.pyz[w] > > The Python files in arcname.pyz[w] are compiled and appended to > the ZIP file. I would suggest allowing multiple versions for compilation (when specified). There should also be a check that people don't specify multiple versions that can't exist on the same directory (e.g 2.6 and 2.7). Otherwise the compileall module's CLI handles this and people can call it with different interpreters as necessary. IOW I'm advocating KISS for what the tool does. Since making the zip file is only really tricky bit it should only handle that case. Heck you can make it part of the zipfile module if there is resistance to adding yet another script that python installs (obviously will need a Cheeseshop package for older versions). Otherwise I like everything else
Re: [Distutils] PEP: Improving Python ZIP Application Support
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/30/2013 03:22 PM, Daniel Holth wrote: > Python ZIP Application Support - > https://docs.google.com/document/d/1MKXgPzhWD5wIUpoSQX7dxmqgTZVO6l9iZZis8dnri78/edit?usp=sharing +1, > but I think this actually belongs on python-dev (AFAICT it is unrelated to distutils, setuptools, etc.) Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlFXbbQACgkQ+gerLs4ltQ5+OgCgqP75xYWG8TGxiT3efvjZWe5U YT4AoKTQJMhkbkU4KBoqmhOCDUl+5/Xo =ERp8 -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [Catalog-sig] Merge catalog-sig and distutils-sig
I've set the wheels in motion. I just need a little help from the pydotorg volunteers (and some hits from the mailman cluebat). Richard On 30 March 2013 06:43, Donald Stufft wrote: > > On Mar 29, 2013, at 3:40 PM, Nick Coghlan wrote: > > > On Fri, Mar 29, 2013 at 7:47 PM, Richard Jones > wrote: > >> On 29 March 2013 14:45, Tres Seaver wrote: > >>> If we leave the main list the 'distutils-sig', and just announce that > >>> 'catalog-sig' is retired, folks who want to follow the new list just > >>> switch over. All the archives (mailman / gmane / etc.) stay valid, but > >>> the list goes into moderated mode. > >> > >> Whoever has the power to do this, do it please. > > > > +1 > > > > distutils-sig it is. We're expanding the charter to "the distutils > > standard library module, the Python Package Index and associated > > interoperabilty standards", but that's a lot easier than forcing > > everyone to rewrite their mail filters. > > > > Besides, it's gonna be a *long* time before the default build system > > in the standard library is anything other than distutils. Coupling the > > build system to the language release cycle has proven to be a *bad > > idea*, because the addition of new platform support needs to happen in > > a more timely fashion than language releases. The incorporation of pip > > bootstrapping into 3.4 will also make it a lot easier to recommend > > more readily upgraded alternatives. > > > > Cheers, > > Nick. > > > > > > -- > > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > > ___ > > Distutils-SIG maillist - Distutils-SIG@python.org > > http://mail.python.org/mailman/listinfo/distutils-sig > > Sounds good to me, whoever please to doing the needful. > > - > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 > DCFA > > ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] "packaging-user" mailing list?
>> We want less lists, not more. the basic math would be adding "packaging-user" and dropping "virtualenv", so no more lists on the whole. if you're not a virtualenv subscriber, then yes, it's one more list, but honestly, this isn't about the active *-sig people, but the users. see below. >> it's actually mostly a support forum that has periodic surges of development discussion the charter sounds like a dev list, and I think the surge is here to stay until the packaging house in order. I think distutils-sig only kinda works now as both, because most of the user traffic doesn't end up here, but rather goes to virtualenv, SO, Python-list, etc... But the problem with those lists, is the lack of certainty about answers across the whole space. If we: - properly described distutils-sig list in it's charter as a user list too - *and* announced it as such, on "Python-announce" - *and* made it prominent in the new "Python Packaging User Guide" (which is in the works; another thread for that later) Then: I think the reality of having distutils-sig serve both would sink in as not ideal. possibly not? The dev/user list distinction is pretty common, so my instinct is to follow that rut here too. my motivation is to have a better "joe user" story. currently (slightly exaggerated for fun) - joe encounters confusing pip error (due to pip being built on shaky packaging ground) - joe: "well, there's the pip user list called 'virtualenv'.". - co-worker: "that error could actually be due to Setuptools, that pip uses I think; pip's support for Setuptools is best effort I hear." - joe: "hmm, Setuptools hasn't been released for awhile. it's website says to use 'distutils-sig', but that sounds like a dev list. maybe that info is outdated." - co-worker: "oh, you're on python3, so I think you're using Distribute. there's some way to tell, but I'm not sure" - joe: "Distribute lists distutils-sig for 'Feedback and getting involved', hmm." - joe" "the distutils-sig archive sure looks like an active dev list. I might be interfering with silly user questions." - co-worker: "maybe try the 'Fellowship of the Packaging' list; that sounds friendly. but wait, it looks dead." - joe: "let me just post to Stack overflow" - co-worker: "hey, check it out, somebody just responded to your post" - joe: "I guess I'll wait a few days, and see if the answer gets votes, so I can tell if it's right or not" what I want - joe encounters error... - joe: "I'll just post to 'packaging-user'. everybody uses that now and gets answers they can usually trust." - joe: "If I want to follow dev discussions, I can join 'distutils-sig', but that's more pain than I want to see" Marcus ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] PEP: Improving Python ZIP Application Support
Python ZIP Application Support - https://docs.google.com/document/d/1MKXgPzhWD5wIUpoSQX7dxmqgTZVO6l9iZZis8dnri78/edit?usp=sharing PEP: 4XX Title: Improving Python ZIP Application Support Author: Daniel Holth Status: Draft Type: Standards Track Python-Version: 3.4 Created: 30 March 2013 Post-History: 30 March 2013 Improving Python ZIP Application Support Python has had the ability to execute directories or ZIP-format archives as scripts since version 2.6. When invoked with a zip file or directory as its first argument the interpreter adds that directory to sys.path and executes the __main__ module. These archives provide a great way to publish software that needs to be distributed as a single file script but is complex enough to need to be written as a collection of modules. This feature is not as popular as it should be for two reasons: a) users haven’t heard of it because it wasn’t mentioned in earlier versions of Python 2.6 “What’s New” document, and b) Windows users don’t have a file extension (other than .py) to associate with the launcher. This PEP proposes to fix these problems by re-publicising the feature, defining the .pyz and .pyzw extensions as “Python ZIP applications” and “Windowed Python Zip Applications”, and providing some simple tooling to manage the format. A New Python ZIP Application Extension The Python 3.4 installer will associate .pyz and .pyzw “Python ZIP Applications” with itself so they can be executed by the Windows launcher. A .pyz archive is a console application and a .pyzw archive is a windowed application. This indicates whether the console should appear when running the app. Why not use .zip or .py? Users expect a .zip file would be opened with an archive tool, and users expect .py to be opened with a text editor. Both would be confusing for this use case. For UNIX users, .pyz applications should be prefixed with a #! line pointing to the correct Python interpreter and an optional explanation. #!/usr/bin/env python # This is a Python application stored in a ZIP archive. (binary contents of archive) As background, ZIP archives are defined with a footer containing relative offsets from the end of the file. They remain valid when concatenated to the end of any other file. This feature is completely standard and is how self-extracting ZIP archives and the bdist_wininst installer format work. Minimal Tooling: The pyzaa Module This PEP also proposes including a simple application for working with Python ZIP Archives: The Python Zip Application Archiver “pyzaa” (rhymes with “huzzah” or “pizza”). “pyzaa” can archive or extract these files, compile bytecode, and can write the __main__ module if it is not present. Usage python -m pyzaa (pack | unpack | compile) python -m pyzaa pack [-o path/name] [-m module.submodule:callable] [-c] [-w] [-p interpreter] directory: ZIP the contents of directory as directory.pyz or [-w] directory.pyzw. Adds the executable flag to the archive. -c compile .pyc files and add them to the archive -p interpreter include #!interpreter as the first line of the archive -o path/name archive is written to path/name.pyz[w] instead of dirname. The extension is added if not specified. -m module.submodule:callable __main__.py is written as “import module.submodule; module.submodule.callable()” pyzaa pack will warn if the directory contains C extensions or if it doesn’t contain __main__.py. python -m pyzaa unpack arcname.pyz[w] The archive is unpacked into a directory [arcname] python -m pyzaa compile arcname.pyz[w] The Python files in arcname.pyz[w] are compiled and appended to the ZIP file. References [1] http://bugs.python.org/issue1739468 “Allow interpreter to execute a zip file” Copyright This document has been placed into the public domain. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Self-contained boostrap scripts [was: Re: A new, experimental packaging tool: distil]
On Mar 29, 2013 4:12 PM, "Nick Coghlan" wrote: > > On Fri, Mar 29, 2013 at 8:43 AM, Daniel Holth wrote: > > WinZip will ignore anything in the front of the file since the zip > > directory doesn't reference it. The #! shebang is for Unix, would > > point to the correct Python, and the +x flag would make it executable. > > The mini PEP is for the .pyz registration and for publicity. > > The two big reasons almost nobody knows about the executable zip files > and directories is we forgot to mention it in the original 2.6 What's > New (it's there now, but was added much later), and it was done in a > tracker issue [1] (with Guido participating) rather than as a PEP. > > A new PEP to: > > * register the .pyz and .pyzw extensions in the 3.4 Windows installer > * ship a tool for creating an executable pyz or pyzw file from a > directory of pure-Python files (warning if any extension files are > noticed, and with the option of bytecode precompilation) > > Would be great. And that pre-compilation could even do it for multiple versions of Python thanks to __pycache__. I've actually contemplated creating a distutils command to do this exact thing. Been thinking about this since 2010: http://sayspy.blogspot.ca/2010/03/various-ways-of-distributing-python.html > > Cheers, > Nick. > > [1] http://bugs.python.org/issue1739468 > > -- > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > http://mail.python.org/mailman/listinfo/distutils-sig ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] "packaging-user" mailing list?
On Fri, Mar 29, 2013 at 5:28 PM, Marcus Smith wrote: > Some of the pypa people have been discussing beginning a "packaging-user" > mailing list. > > - It would be open to *any* packaging or install user issues. > - It would be on python.org > - pip/virtualenv would use it instead of our "virtualenv" list > - Other projects could(would) use it too: Setuptools (old and new), > Distribute, wheel, buildout, bento? etc... > > I think python users would appreciate the simplicity of this, and the > support would be better too I think. But all the developers who actually give support are here, aren't they? Certainly I do setuptools support here, and Jim does buildout support. And all the people already being sent here from other sources aren't going to stop being sent here. > distutils-sig would maintain it's current focus as a place for development > discussions. It's actually mostly a support forum that has periodic surges of development discussion. Granted, the current development surge is bigger and longer than any that have happened in a while. ;-) ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] "packaging-user" mailing list?
On 29/03/2013 21:28, Marcus Smith wrote: distutils-sig would maintain it's current focus as a place for development discussions. thoughts? -1. We want less lists, not more. I'd say just roll it all into distutils and be done with it. cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Self-contained boostrap scripts [was: Re: A new, experimental packaging tool: distil]
Nick Coghlan gmail.com> writes: > * ship a tool for creating an executable pyz or pyzw file from a > directory of pure-Python files This could be a variant of my pyzzer.py tool: https://gist.github.com/vsajip/5276787 It doesn't print warnings for extensions or byte-compile anything, but otherwise does more or less what you mentioned. Regards, Vinay Sajip ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
