Re: [Distutils] [License-discuss] Trove Classifiers
On Wed, May 4, 2016 at 4:02 PM, Paul R. Tagliamonte wrote: > Hey all, > > For those who don't know, Trove classifiers are used by the Python > world to talk about what is contained in the Python package. Stuff > like saying "It's under the MIT/Expat license!" or "It's beta!". > > > I was looking at the tags, and I saw one that made me "wat" a bit. > >> License :: OSI Approved :: GNU Free Documentation License (FDL) > > AFAIK the GFDL is *not* OSI approved, both due to it not being a > software license, as well as I'm sure the invariant clauses being an > issue. > > Has anyone come across this yet? Anyone have objections to me trying > to clean up the Trove list? Good catch! Cleaning the list is going to be easy on the Python.org side, especially since a new Pypi site is in the making. [1] The harder or impossible part would have be to clean up the 1000+ of packages using this faulty classifier But there is really only three of these [2] and all of them look either pretty old or abandoned and none has its packages effectively hosted or distributed on Pypi. [1] https://pypi.io/ [2] https://pypi.python.org/pypi?:action=browse&c=63 -- Cordially Philippe Ombredanne ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Proposal: "Install and save"
On Sat, Jul 23, 2016, 17:04, Thomas Kluyver wrote: > As I understand it, in the Javascript world package.json is used in > both > cases. Is that something Python should try to emulate? Is it hard to > achieve given the limitations of setup.py that you pointed out? Thomas: this is not entirely correct: a node package with native code needs to have node-gyp [1] installed which in turns relies on the gyp-based Python tool ;) to actually build the natives using actual makefile-like build scripts. MO is that any kind of mildly engaged package build for NPM requires a rather more involved effort than its Python world equivalent. And you can see the JS packaging world walking somewhat of the same path that has walked here before. To get some feeling of what they are going through read this thread [2] On Sat, Jul 23, 2016 at 10:11 AM Alex Grönholm wrote: > This topic has been beaten to death. There is no way to cram the > complexities of C extension compilation setup into purely declarative > metadata. Distutils2 tried and failed. Just look at the setup.py files > of some popular projects and imagine all that logic expressed in > declarative metadata. Indeed! There is no magic and no free lunch there. [1] https://github.com/nodejs/node-gyp [2] https://github.com/npm/npm/issues/1891 -- Cordially Philippe Ombredanne ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 527 - Removing Un(der)used file types/extensions on PyPI
On Tue, Aug 23, 2016 at 10:08 PM, Glyph Lefkowitz wrote: > In particular, bdist_wininst and bdist_msi (which Twisted supported for a > long time, and still builds, so it's not like I don't understand their > benefits and history!) are incompatible with virtualenvs, and make > development under Windows harder, especially as compared to binary eggs. > The presence of these builds confuses users and creates more problems than > it solves in every interaction I've had with onboarding people onto Python > projects in the last couple of years. I can only agree. A strong +1 to deprecate bdist_wininst and bdist_msi from Pypi ... and from Python everywhere if it is be possible. I build and distribute a somewhat engaged open source app (scancode-toolkit) and earlier version used eggs and now this is all wheels in virtualenvs. This runs on linux/mac/win with vendored wheels. I cannot fathom _any_ benefit from using Windows installers in a modern Python environment like this and other where pip and virtualenv rule. Like Glyph, I think these these installers only create confusion at first and are the source of troubles down the road even if they may seem like a friendly solution for win noobs on the surface. Every time I met someone using them, this was a source of mess in their Python setup. They made sense in 1999, but are likely best forgotten today. And if this was dropped elsewhere beside Pypi, there could quite a rafter of arcane code to drop from distutils and setuptools.... a good thing IMHO. -- Cordially Philippe Ombredanne ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [proposal] module dependency specification and overriding for packages
On Fri, Sep 9, 2016 at 8:46 AM, Ronny Pfannschmidt wrote: > while trying to vendor packages that depend on six, i noticed a problematic > implication > > it is impossible to do this without patching those modules, as such adding a > fragile maintenance burden. Ronny: I grok the general issue, but would you have a concrete example? -- Cordially Philippe Ombredanne ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [buildout] windows/jenkins annoyance
On Sat, Jan 19, 2013 at 11:54 PM, Chris Withers wrote: > On 19/01/2013 21:16, Jim Fulton wrote: >> Another option would be to pin setuptools in your buildout.cfg to avoid the >> upgrade. > Just a note to say that this didn't work: > http://jenkins.simplistix.co.uk/job/testfixtures-buildout/PYTHON=2.7,label=windows/167/console > The problem is that bootstrap.py *always* dowloads setuptools. I wish it > didn't do that if the right version of setuptools was already in the egg > cache... You could try to pin the distribute or setuptools version used by bootstrap this way: python bootstrap.py --version 1.7.0 \ --distribute --setup-source=/ez_setup.py \ <-- saved locally --download-base=/dist/<-- with a saved locally version of setuptools and then have cached version of the setup and archive for distribute and setuptools, and use a buildout with 1/ pinned version of distribute or setuptools 2/ these extra buildout config info. Something along these lines: [buildout] install-from-cache = true newest = false allow-picked-versions = false download-cache = find-links = file:dist/ versions = versions [versions] setuptools = 0.6c11 Not sure it would help, but worth a try -- Philippe Ombredanne +1 650 799 0949 | pombreda...@nexb.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] buildout in offline mode - install receipts also from cache?
On Sun, Jan 20, 2013 at 7:36 PM, Jim Fulton wrote: > On Sat, Jan 12, 2013 at 7:11 PM, Philippe Ombredanne > wrote: > ... >> IMHO it is not possible today to bootstrap and buildout offline. Anyone with >> a better story there? > > To peal this a bit further. > - It's not possible to bootstrap buildout with the regular bootstrap script > without doing network access. > > You could certainly arrange a custom bootstrap script that installed > from a local cache, but you can't use offline mode, which is really > no-install mode. > > You could conceivably arrange to bootstrap in offline mode if you > pre-populated your eggs directory with buildout and setuptools/distribute > eggs, but this is splitting hairs. Actually it is perfectly possible to boostrap in a mode where no network connection is available whcih is my naive assumption for the "offline" option More details to come in another reply in the same thread -- Philippe Ombredanne +1 650 799 0949 | pombreda...@nexb.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] buildout in offline mode - install receipts also from cache?
On Sun, Jan 13, 2013 at 3:49 PM, Jim Fulton wrote: > On Sat, Jan 12, 2013 at 7:11 PM, Philippe Ombredanne > wrote: >> IMHO it is not possible today to bootstrap and buildout offline. Anyone with >> a better story there? > Try: > install-from-cache = true > without specifying offline mode. > install-from-cache was intended to support exactly this use case. I works! My bad, and I can now get something to work with for instance: python bootstrap.py --version 1.7.0 \ --distribute --setup-source=./thirdparty/dist/distribute_setup.py \ --download-base=./thirdparty/dist/ and a [buildout] with: include-site-packages = false install-from-cache = true newest = false allow-picked-versions = false # we need this in buidlout 1 too zc.buildout-version = 1.7.0 distribute-version = 0.6.34 # Weirdly enough buildout demands the files to be cached in a dist subdir # See https://github.com/buildout/buildout/issues/40 download-cache = thirdparty find-links = file:///thirdparty/ file:///thirdparty/dist/ > Again, I think this works by setting install-from-cache to true. > The original semantics of offline mode didn't fit it's name. It originally > meant "don't install anything". That's how it was implemented. The original > goal was both not to download anything and not to **install** anything. > Note that this was added before there was a cache. My naive understanding of offline is "without a network connection" > Later, offline mode was extended to avoid network access. In particular > offline mode was changed to disallow extending remote configuration > files. This change, not made by me, broke some of my companies buildouts. > I didn't argue with it at the time as the change was consistent with the > option name, if not the original intent. > Offline mode is largely a relic of buildout's early evolution. I almost > never use it and, when I do, I use it in a way that depend on it's > current semantics. In particular, I use buildout to create configurations > for installed software. In this context, buildout is running as root and it > is not OK to install new software. If anything the option name is as you said incorrect, and the "do not install" use case somewhat arcane and undocumented. But that is another story. > I believe that install-from-cache (without offline mode) would address > Philippe's use case. (It's used by zc.sourcerelease.) Yes! > That leaves the ambiguous meaning of offline mode. In particular, > buildout 2 potentially provides an opportunity to fix things in a backward- > incompatible way. > OTOH, I'd like to avoid any major changes in 2.0 at this point > because it really needs to get finished as soon as possible. > Among the options I see at the present: > - Do nothing. And just document the somewhat weird semantics > of the name. Leaving this ambiguity in offline longer would be unfortunate IMHO > - Add a new option: install, such that install=false has the same > meaning as offline=true. > Deprecate the offline option and accompanying command-line option. That would be the preferred approach IMHO Do you want a patch? Though I am not sure how to proceed to deprecate the option -- Philippe Ombredanne +1 650 799 0949 | pombreda...@nexb.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [buildout] infinite loop when specifying setuptools version on mac
On Mon, Jan 21, 2013 at 8:07 PM, Chris Withers wrote: > On 21/01/2013 09:36, Philippe Ombredanne wrote: >> On Sun, Jan 20, 2013 at 9:15 PM, Chris Withers >> wrote: >>> See the attached .tgz, here's what I did on a mac min running 10.7.4: >>> "/Library/Python/2.7/site-packages/setuptools-0.6c12dev_r88846-py2.7.egg/setuptools/package_index.py", > Can anyone else with a Mac running 10.7.4 check whether this is how they > ship? I'm fairly certain I haven't installed anything globally on that > machine, so I'd be surprised and disappointed if the above was my doing... There you go, sounds like MacOSX Lion ships with a broken setuptools : http://www.sherman.ca/archives/2011/07/25/lion-broken-python-easy_install/ The remedy is to use distribute ... I I already saw that bug elsewhere btw now that I think of it. -- Philippe Ombredanne +1 650 799 0949 | pombreda...@nexb.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Support for condition to include or exclude a buildout sections based on a Python expression
All: I prepared a pull request https://github.com/buildout/buildout/pull/51 to add support in Buildout 2 for conditional sections expressions. The gist of this is explained here: Sections headers can contain an optional arbitrary Python expression. When the expression evaluates to false the whole section is skipped. Several sections can have the same name with different expressions, enabling conditional exclusion of sections. For instance using simple expressions you can get this done: # this expression is true, so the section is included [s1: 2 + 2 == 4] a = 1 # this expression is false, so the section s2 is skipped [s2 : 2 + 2 == 5 ] long = a # Complex expressions are possible, though they should not be abused :) and they need to fit on a single line [s3:2 in map(lambda i:i*2, [i for i in range(10)])] long = c One of the typical usage is to have buildout parts that are operating system or plaform specific, such as defining eggs that may run only on one os (such as readline only for Macosx and pyreadline only for Windows.). There some convenience defaults conditionals pre-computed as well as some default modules available to an expression to support this: [install_eggs: windows] recipe = zc.recipe.egg eggs = pyreadline and [install_eggs: macosx] recipe = zc.recipe.egg eggs = readline ... become possible and only the right section will be included. The limited (but hopefully sane and sufficient) set of default modules and precomputed common expressions are: # available modules sys, re, os and platfrom should cover most of the common cases [s1: sys and re and os and platform] # major and minor python versions, yes even python 3.5 and 3.6 are there, prospectively [s2: any([python2, python3, python24 , python25 , python26 , python27 , python30 , python31 , python32 , python33 , python34 , python35 , python36]) ] # common python interpreter types [s3: cpython or pypy or jython or ironpython] # common operating systems [s4: linux or windows or cygwin or macosx or solaris or posix ] # common bitness and endianness [s5: bits32 or bits64 or little_endian or big_endian] Some other points: - expression are evaluated _during parsing_ of a cfg file, so before interpolation, hence variable substitution are not possible... we could populate the expression context with a copy of the current state of the buildout object to provide some limited access to some of the buildout options and sections that are already known at the parsing time.. - security could be a concern, but hey if you are running buildout you are already letting run arbitrary code and recipes downloaded live from unknown sources, so I do not think this is a big concern. - if you do not use expressions in your buildout, there is no impact at all. Feedback on the code and features is welcomed... ATM the code is ready to pull in and all the tests are passing -- Philippe Ombredanne +1 650 799 0949 | pombreda...@nexb.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] PEP-426 environment markers [was: Re: Support for condition to include or exclude a buildout sections based on a Python expression]
On Wed, Jan 30, 2013 at 1:39 PM, Jim Fulton wrote: > On Tue, Jan 29, 2013 at 6:25 PM, Nick Coghlan wrote: >> On Wed, Jan 30, 2013 at 12:02 AM, Philippe Ombredanne >> wrote: >>> All: >>> >>> I prepared a pull request https://github.com/buildout/buildout/pull/51 >>> to add support in Buildout 2 for conditional sections expressions. >> >> If you're going to do something like this, it would be good to base it >> on PEP 345/426 style "environment markers", which are basically the >> same idea applied to dependency definitions in the package metadata: >> http://www.python.org/dev/peps/pep-0426/#environment-markers > > I think it makes sense to support these markers. This is rather easy with a caveat: http://www.python.org/dev/peps/pep-0426/#environment-markers defines markers EXPR as one of these: python_version = '%s.%s' % (sys.version_info[0], sys.version_info[1]) python_full_version = sys.version.split()[0] os.name = os.name sys.platform = sys.platform platform.version = platform.version() platform.machine = platform.machine() platform.python_implementation = platform.python_implementation() a free string, like '2.4', or 'win32' extra = (name of requested feature) or None A python implementation of the proposed draft would have to monkey patch standard functions (platform.version, platform.machine and platform.python_implementation) with their invocation results or would require rather complex parsing. Since this is a draft, could we instead avoid a module-like syntax for all EXPR and instead use this: python_version = '%s.%s' % (sys.version_info[0], sys.version_info[1]) python_full_version = sys.version.split()[0] os_name = os.name sys_platform = sys.platform platform_version = platform.version() platform_machine = platform.machine() platform_python_implementation = platform.python_implementation() a free string, like '2.4', or 'win32' extra = (name of requested feature) or None This sounds more consistent to me to replace the dots with _ consistently the same way the original python_version has been specified Also, platform.python_implementation() is new in Python 2.6 so this could not run on 2.5 or 2.5 even though the PEP alludes to support of 2.4 and 2.5 This is not an issue for Buildout 2.0, but could be for the PEP -- Philippe Ombredanne +1 650 799 0949 | pombreda...@nexb.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Support for condition to include or exclude a buildout sections based on a Python expression
On Wed, Jan 30, 2013 at 1:39 PM, Jim Fulton wrote: > On Tue, Jan 29, 2013 at 6:25 PM, Nick Coghlan wrote: >> On Wed, Jan 30, 2013 at 12:02 AM, Philippe Ombredanne >> wrote: >>> I prepared a pull request https://github.com/buildout/buildout/pull/51 >>> to add support in Buildout 2 for conditional sections expressions. >> >> If you're going to do something like this, it would be good to base it >> on PEP 345/426 style "environment markers", which are basically the >> same idea applied to dependency definitions in the package metadata: >> http://www.python.org/dev/peps/pep-0426/#environment-markers > > I think it makes sense to support these markers. Done :) with the caveat expressed in http://mail.python.org/pipermail/distutils-sig/2013-January/019622.html as I am using underscores as separators rather than dots: See : https://github.com/pombredanne/buildout/commit/44902dcc5ac9306db1a8a26fa92ff3a761029b0e The pull request contains this, with new tests https://github.com/buildout/buildout/pull/51 and all tests are passing (or failing the same way they do with a standard buildout 2.0 for Python 3.2) https://travis-ci.org/pombredanne/buildout/builds/4475514 -- Philippe Ombredanne +1 650 799 0949 | pombreda...@nexb.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP-426 environment markers [was: Re: Support for condition to include or exclude a buildout sections based on a Python expression]
On Wed, Jan 30, 2013 at 3:36 PM, Jim Fulton wrote: > On Wed, Jan 30, 2013 at 9:00 AM, Philippe Ombredanne > wrote: >> A python implementation of the proposed draft would have to monkey >> patch standard functions (platform.version, platform.machine and >> platform.python_implementation) with their invocation results or would >> require rather complex parsing. > > I hadn't noticed that you were exposing the platform module already. > Sorry. I should have checked. No problem. > The PEP's treatment of the platform module is unfortunate. > I was going to mention this before, but I didn't think we were > exposing platform module and though "whatever". The parsing would be rather complex to implement as specified Do we have a a PEP426 reference implementation somewhere? >> Since this is a draft, could we instead avoid a module-like syntax for >> all EXPR and instead use this: > I have no interest in supporting a Python expression subset. > I'd rather stick with what you did originally and just expose the > modules. This is simpler and easier for people to remember. Agreed, that was my first take on this... but I am a trooper too :D So I commited and added to the buildout pull request a commit with PEP426 like markers and then another that undoes it ... With cherry picking you can pick and choose which one you want, the last commit in the queue being WITHOUT PEP426 markers .. -- Philippe Ombredanne +1 650 799 0949 | pombreda...@nexb.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Support for condition to include or exclude a buildout sections based on a Python expression
On Wed, Jan 30, 2013 at 4:53 PM, Leonardo Rochael Almeida wrote: > How easy it is to know which sections are actually used during a run? buildout -v (in debug) will list skipped sections > Also, it would be nice to have this info on the output of "buildout annotate" I am not sure this would be possible as the sections are skipped before annotating -- Philippe Ombredanne +1 650 799 0949 | pombreda...@nexb.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Support for condition to include or exclude a buildout sections based on a Python expression
> On Wed, Jan 30, 2013 at 4:52 PM, Philippe Ombredanne > wrote: >> On Wed, Jan 30, 2013 at 4:53 PM, Leonardo Rochael Almeida >> wrote: >>> Also, it would be nice to have this info on the output of "buildout >>> annotate" >> I am not sure this would be possible as the sections are skipped >> before annotating On Wed, Jan 30, 2013 at 8:42 PM, Leonardo Rochael Almeida wrote: > If they're skipped before annotating, but the ones that remain on > "annotate" are the ones that would be run, and they include the > conditional info, then that's Good Enough For Me (TM) :-) this is exactly the behavior ;) -- Philippe Ombredanne +1 650 799 0949 | pombreda...@nexb.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] A new script which creates Python 3.3 venvs with Distribute and pip installed in them
On Wed, Jan 30, 2013 at 9:09 PM, Vinay Sajip wrote: > Python 3.3 includes a script, pyvenv, which is used to create virtual > environments. > However, Distribute and pip are not installed in such environments - because, > though they are popular, they are third-party packages - not part of Python. > The Python 3.3 venv machinery allows customisation of virtual environments > fairly readily. To demonstrate how to do this, and to provide at the same time > a script which might be useful to people, I've created a script, pyvenvex.py, > at > https://gist.github.com/4673395 > which extends the pyvenv script to not only create virtual environments, but > to also install Distribute and pip into them. Excellent and one step closer to sane package management I wonder if you could not source instead the code that is directly in the virtualenv.py scripts? it also includes the packed distribute and pip Meaning that would allow the installation entirely offline (with the --never-download venv flag) And btw, why pip is not part of the standard Python? This is nowadays officially recommended on Pypi as the tool to use to install package Per http://pypi.python.org/pypi "Get Packages: To use a package from this index either "pip install package" (get pip) or download, unpack and "python setup.py install" it." This does not make sense to me: I know about some of the controversies but this is rather inconsistent to recommend using a tool and not supporting it directly. -- Philippe Ombredanne +1 650 799 0949 | pombreda...@nexb.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] I wonder what is the best practice to clean a buildout project ?
On Thu, Jan 31, 2013 at 3:37 PM, Stéphane Klein wrote: > Hi, > I wonder what is the best practice to clean a buildout project ? > I would like a command to remove : > * bin/ > * develop-eggs > * parts > * .installed.cfg > I can create a Makefile clean, or clean.sh script. > What do you do in your project ? I use a makefile, a clean.sh and a clean.bat to ensure support on most OS whether or not make is available, -- Philippe Ombredanne +1 650 799 0949 | pombreda...@nexb.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 426, round 733 ;)
On Mon, Feb 4, 2013 at 2:10 PM, Nick Coghlan wrote:
> As usual, PEP inline below and on the web at
> http://www.python.org/dev/peps/pep-0426/
> Version scheme
> ==
> Version numbers must comply with the following scheme::
> N.N[.N]+[{a|b|c|rc}N][.postN][.devN]
>
> Version numbers which do not comply with this scheme are an error. Projects
> which wish to use non-compliant version numbers must restrict themselves
> to metadata v1.1 (PEP 314) or earlier, as those versions do not mandate
> a particular versioning scheme.
>
> Suffixes and ordering
> -
> The following suffixes are the only ones allowed at the given level of the
> version hierarchy and they are ordered as listed.
> Within a numeric release (``1.0``, ``2.7.3``)::
>.devN, aN, bN, cN, rcN, , .postN
Are we trying to mix in too many thing in a versioning scheme?
IMHO a version (or eventually its dot-separated segments with
precedence from left to right) should increase when sorted
lexicographically so it is never ambiguous for a human reading a list
of versioned components, sorts naturally in most filesystems and tools
and is easy to implement.
With the proposed scheme, I can picture myself having to call some
code each time I need compare two package versions and scratching my
head otherwise.
Having been involved in defining version schemes in the past for
Eclipse, I came then to the conclusion that anything that does not
sort lexicographically is probably a bad idea.
Are we trying to make a version -- which is an engineering must --
into something that has also some semantics about the level of
completion of a project or some "marketing" alert on the level of
maturity of a software release? Could we stay instead in the realm of
engineering?
I think that trying to inject things like alpha, beta, post, dev,
release candidates and the likes in this is trying to bake in too many
things that are eventually just the practices of some projects and
should not be the frozen practice baked in a PEP. Instead, this
should be left to project authors to define their own scheme as long
as it sorts lexicographically (eventually by segments, with precedence
from left to right).
--
Philippe Ombredanne
+1 650 799 0949 | pombreda...@nexb.com
DejaCode Enterprise at http://www.dejacode.com
nexB Inc. at http://www.nexb.com
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 426, round 733 ;)
On Tue, Feb 5, 2013 at 3:08 PM, Daniel Holth wrote: > The "marketing pre-release" feature exists to allow publishers to put > immature versions of their software on pypi where they can be easily > downloaded. Recently SQLAlchemy did this but had to delete the beta release > from pypi because too many deployments upgraded to an unstable version > without realizing it. Once the tools are updated it will be easy to install > a beta release with pip if and only if you specifically ask for it. May be versioning scheme is trying to take on too much on itself that could possibly be solved elsewhere in a simpler way? Immature software distribution is a requirement that makes perfect sense --but even that is not in the scope of this PEP -- could it better addressed by having a something like Pypi "release channels" instead ie some separate indexes for unstable/alpha/bleeding edge packages that responsible and consenting adults could use as they please or something similar? This is FWIW a common practice on Debian. -- Philippe Ombredanne +1 650 799 0949 | pombreda...@nexb.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 426, round 733 ;)
On Tue, Feb 5, 2013 at 6:44 AM, Nick Coghlan wrote: > The version scheme is not going to change. The point of PEP 386 was, > to a very large extent, to define a scheme that *existing PyPI > projects* either already comply with, or will require only minor > cosmetic changes to comply with (such as inserting an extra period to > turn X.YdevN into X.Y.devN). [] > We already know the most likely outcome of such an effort: people will simply > stick > with v1.1 of the metadata scheme and continuing to use the existing > packaging tools, as migrating to the new ones would require a whole > lot pointless busywork to redesign their build processes and > workflows. This is in fact the killer argument here, i.e. even though I may not like all that much some of the supported proposed scheme, the practicality of adopting and supporting existing practices trumps IMHO all other arguments. So any objections I brought here about the version schemes are addressed by your answers :) I particularly like the wording you have in the PEP where you explicitly support but do not encourage more byzantine version schemes. -- Philippe Ombredanne +1 650 799 0949 | pombreda...@nexb.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 427 (wheel format) accepted
On Sat, Feb 16, 2013 at 3:47 AM, Nick Coghlan wrote: > Daniel submitted PEPs 425 and 427 to python-dev for acceptance, and I > have formally accepted 427 as BDFL delegate > (http://www.python.org/dev/peps/pep-0427/). This is awesome! Congratulations to Daniel -- Philippe Ombredanne ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Bootstrapping Buildout 2.x without a network connection
With buildout 1 and its bootstrap script, its is possible to bootstrap
a carefully crafted environment without a network connection:
python bootstrap.py --version 1.7.0 --distribute
--setup-source=distribute_setup.py --download-base=.//dist/
assuming you have a download-cache without all your packages including
buildout itself and all the dependencies, newest=false,
installed-from-cache=true and properly pinned versions.
hard to setup, but this works.
The new bootstrap script for buildout 2.0 always attempt to download
distribute_setup.py instead if distribute is not available:
See:
https://github.com/buildout/buildout/blob/ee709cfb00229fbd8fc8554db8df06a46808/bootstrap/bootstrap.py#L81
exec(urlopen('http://python-distribute.org/distribute_setup.py').read()
What is the way around this?
FWIW, the buildout 1 bootstrap script seems to work OK when pointed to
a buildout 2 version and produced the expected behavior
Or would a patch be welcomed?
--
Philippe Ombredanne
+1 650 799 0949 | pombreda...@nexb.com
DejaCode Enterprise at http://www.dejacode.com
nexB Inc. at http://www.nexb.com
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Updated pull request for conditional Buildout sections in Buildout 2.x
I posted an updated pull request with support for conditional Buildout sections in Buildout 2.x https://github.com/buildout/buildout/pull/76 This new version addresses some issues when handling arbitrary Python expressions and comments with brackets [ ] as these were sometimes ambiguous as the section names are also enclosed in comments and section expressions can also contain list comprehensions and other brackets enclosed lists. All the tests are passing and I am getting good mileage from these on rather large buildouts. You can give it a local test spin by fetching this branch: https://github.com/pombredanne/buildout/tree/buildout_21dev building a dist with setup.py and asking for a buildout with version 2.1.0dev1 in your boostrap with boostrap.py --version 2.1.0dev1 --find-links= -- Philippe Ombredanne +1 650 799 0949 | pombreda...@nexb.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 426 is now the draft spec for distribution metadata 2.0
On Sun, Feb 17, 2013 at 2:10 AM, Nick Coghlan wrote: > The latest draft of PEP 426 is up at http://www.python.org/dev/peps/pep-0426/ This is looking great! > License (optional) > -- [...] > The full text of the license would normally be included in a separate > file. I know it is late in the game to do modifications but I often find myself chasing down the actual full text of a license in a distribution. Would you be open to a new optional field that could point to the license file(s) that may be present with the distribution? May be something along these lines: License-File (multiple use) -- Optional file name or path to a file that contains the full text of the license, notice, copyright or other terms of use that may be included in a distribution such as a LICENSE or COPYING file. A path must be a posix path using ``/`` forward slash as path separator. A path or file name must be interpreted as relative to the root directory of a distribution archive. Examples:: License-File: COPYING License-File: COPYING.lib License-File: docs/legal/APACHE.txt License-File: docs/legal/NOTICE.txt -- Philippe Ombredanne +1 650 799 0949 | pombreda...@nexb.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Bootstrapping Buildout 2.x without a network connection
On Mon, Feb 18, 2013 at 1:44 AM, Reinout van Rees wrote:
> On 17-02-13 03:25, Philippe Ombredanne wrote:
>>
>> The new bootstrap script for buildout 2.0 always attempt to download
>> distribute_setup.py instead if distribute is not available:
>>
>> See:https://github.com/buildout/buildout/blob/ee709cfb00229fbd8fc8554db8df06a46808/bootstrap/bootstrap.py#L81
>> exec(urlopen('http://python-distribute.org/distribute_setup.py').read()
>
> Buildout 2 only works with distribute, not with setuptools. So it isn't
> unexpected that it downloads the distribute_setup.py.
Agreed, but the boostrap1 script can be pointed to a pre-downloaded
distribute_setup.py.
The new one cannot, I think this is a regression.
> So you'll have to use setuptools instead of distribute in your offline
> environment.
I guess you meant the to use distribute instead of setuptools?
Because using setuptools would mean I cannot use buildout2?
--
Philippe Ombredanne
+1 650 799 0949 | pombreda...@nexb.com
DejaCode Enterprise at http://www.dejacode.com
nexB Inc. at http://www.nexb.com
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [buildout] http://downloads.buildout.org/1/bootstrap.py resulting in buildout 2.0.1
On Tue, Feb 19, 2013 at 2:19 PM, Chris Withers wrote: > Am I the only one who has buildouts ending up using 2.0.1 when using: > http://downloads.buildout.org/1/bootstrap.py > ...which I thought would keep zc.recipe.egg and zc.buildout pinned pre 2.0? With this buildout.cfg: [buildout] parts= eggo [eggo] recipe=zc.recipe.egg eggs = anyreadline I get this: $ wget http://downloads.buildout.org/1/bootstrap.py $ python2.6 bootstrap.py Downloading http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c11-py2.6.egg [...] Getting distribution for 'setuptools'. Got setuptools 0.6c12dev-r88846. [...] $ bin/buildout Getting distribution for 'zc.recipe.egg<2'. Got zc.recipe.egg 1.3.2. Installing eggo. Getting distribution for 'anyreadline'. warning: install_lib: 'build/lib' does not exist -- no Python modules to install Got anyreadline 0.1.1. $ ls -l eggs total 328 drwxr-xr-x+ 1 pombredanne None 0 2013-02-19 14:32 anyreadline-0.1.1-py2.6.egg -rw-r--r--+ 1 pombredanne None 334419 2013-02-19 14:31 setuptools-0.6c12dev_r88846-py2.6.egg drwxr-xr-x+ 1 pombredanne None 0 2013-02-19 14:31 zc.buildout-1.7.0-py2.6.egg drwxr-xr-x+ 1 pombredanne None 0 2013-02-19 14:32 zc.recipe.egg-1.3.2-py2.6.egg Nothing pinned, only latest current 1.x buildout eggs coming my way. -- Philippe Ombredanne +1 650 799 0949 | pombreda...@nexb.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [buildout] http://downloads.buildout.org/1/bootstrap.py resulting in buildout 2.0.1
On Tue, Feb 19, 2013 at 2:39 PM, Chris Withers wrote: > Dunno what I'm doing wrong then, see the eggs being used here: > http://jenkins.simplistix.co.uk/job/checker-buildout/86/PYTHON=2.7,label=linux/console Weird, sounds like both the 1.3.2 and 2.0.0a3 egg recipe versions are being fetched: $ bin/buildout Develop: '/home/pombredanne/checker/checker/.' Getting distribution for 'zc.recipe.egg<2'. Got zc.recipe.egg 1.3.2. <== here Installing py. Getting distribution for 'tox'. Got tox 1.4.2. Getting distribution for 'zope.testing'. warning: no files found matching '*.test' under directory 'src' warning: no files found matching 'sampletests' under directory 'src' Got zope.testing 4.1.1. Getting distribution for 'zc.recipe.egg<2'. Got zc.recipe.egg 2.0.0a3. <== here Getting distribution for 'testfixtures'. Got testfixtures 2.3.5. You might want to run your buildout on a fresh checkout with verbose debug flags bin/buildout - to see what is cooking in there It feels as if that while processing buildout just go fetch two versions of the egg recipe? It is likely that one of your required eggs may have a relaxed requirement on zc.recipe.egg? -- Philippe Ombredanne +1 650 799 0949 | pombreda...@nexb.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] buildout and build-time dependencies
On Mar 1, 2013 5:54 AM, "Reinout van Rees" wrote: > > On 28-02-13 20:31, Ben Acland wrote: >> >> tl;dr: how to handle build time dependencies between python modules >> using buildout, without looking stupid or including .tar.gz files in my >> repo. > numpy and scipy are a hell to install. Would there be something we could do to help these important projects be easier to install? -- Philippe ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] A new, experimental packaging tool: distil
On Tue, Mar 26, 2013 at 9:54 AM, Vinay Sajip wrote: > I've created a new tool called distil which I'm using to experiment with > packaging functionality. Nice! > * Very simple deployment - just copy distil.py[1] to a location on your path, > optionally naming it to distil on POSIX platforms. There's no need to > install > distlib - it's all included. I see that you are using a pattern similar to the virtualenv.py script, embedding other code as a compressed byte array. See how virtualenv.py is turning out to be lately: https://github.com/pypa/virtualenv/blob/11ccab2698274f0e10b72da863f9efb73cf1a9aa/virtualenv.py#L1937 I am in general fine with the approach, though I feel a bit uncomfy with this approach creeping in as "the" way to bootstrap things with one single file for core distribution-related tools. Would anyone know of a better way to package things in a single python-executable bootstrapping script file without obfuscating the source contents in compressed/encoded/obfuscated byte arrays? Also, in your code calling this binary payload STUFF feels a tad scary: this is arbitrary code that I cannot see nor inspect before running. I would not want to run unknown STUFFs on my machine ... and even more so since the corresponding sources are not available publicly yet in a source repo. At the minimum, getting some comments or explicit variable names the virtualenv way on what this payload is would help IMHO: "STUFF = """ eJyEm1OMLlC3Zb+ybbtO1Snbtm3brjpl27Zt27Zt27b6Tye3b9J9k37YK9kv+2FmPMxkrC0vBQKKCgA AIAGIUeqCCqFgEh/4AACRBQCAD8AFGFs4OVtbGNLpGRoYWdnbOTrTObk7GdnZmlqY0dq7qyhDAUDqZP .." -- Cordially Philippe Ombredanne ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Self-contained boostrap scripts [was: Re: A new, experimental packaging tool: distil]
On Tue, Mar 26, 2013 at 11:08 AM, Paul Moore wrote: > On 26 March 2013 09:49, Philippe Ombredanne wrote: >> Would anyone know of a better way to package things in a single >> python-executable bootstrapping script file without obfuscating the >> source contents in compressed/encoded/obfuscated byte arrays? > > Packaging as a zip file is a good way - but on Windows the file needs > to be named xxx.py (which is surprising, to say the least :-)) for the > relevant file association to be triggered (and on Unix, a #! line > needs to be prepended). Paul: I was not talking about this type of zips, but rather the same used in virtualenv, i.e. a string in a .py file that contains an encoded zip. That string is then decoded and unzipped at runtime as in here: https://github.com/pypa/virtualenv/blob/develop/virtualenv.py#L1933 This is not a zip, not an egg, not a wheel but some egg-in-py, zip-in-py or wheel-in-py and is similar to a shar shell archive. My point was that on the one hand, I like the fact that everything is self contained in one single .py file that you can execute right away. On the other hand, I find it somewhat discomforting as an emerging best way to package and distribute self-contained bootstrap scripts. Yet I cannot think of a better way atm: for instance splitting things in non-encoded non-binary plain strings would be quite weird too. Virtualenv does it, distil is doing it now, pip tried some of it here https://github.com/pypa/pip/blob/develop/contrib/get-pip.py In contrast, buildout, distribute and setuptools bootstrap scripts do not embed their dependencies and either try to get them satisfied locally or attempt to download the requirements. Having some support to do self-contained bootstrap scripts (as in requiring no network access and embedding all their dependencies) using this shar style could be something to consider normalizing? -- Philippe Ombredanne +1 650 799 0949 | pombreda...@nexb.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Self-contained boostrap scripts [was: Re: A new, experimental packaging tool: distil]
On Thu, Mar 28, 2013 at 2:33 PM, Vinay Sajip wrote: >> From: Philippe Ombredanne >> On the other hand, I find it somewhat discomforting as an emerging >> best way to package and distribute self-contained bootstrap scripts. >> Virtualenv does it, distil is doing it now, pip tried some of it here >> https://github.com/pypa/pip/blob/develop/contrib/get-pip.py >> In contrast, buildout, distribute and setuptools bootstrap scripts do >> not embed their dependencies and either try to get them satisfied >> locally or attempt to download the requirements. > > And all this time, they would have been vulnerable to a MITM attack > on PyPI because PyPI didn't support verifiable SSL connections > until recently. It's good to be cautious, but Bruce Schneier has > plenty of stories about caution directed in the wrong directions. I am not so worried about security... I brought the point here because this is the packaging and distribution list, and I see this as an emerging pattern for the packaging and distribution of bootstrap scripts and this is something that has not been discussed much before. Conceptually I find these no different from setup.py scripts, and these have been mostly normalized (or at the minimum have a conventional name and a conventional if not specified interface.) Yet today, for the all important core package and environment management tools, we have bootstrap scripts each with different interfaces and different approaches to self containment or no containment. I feel this is worth discussing as bootstrapping is where everything begins :) -- Philippe Ombredanne +1 650 799 0949 | pombreda...@nexb.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Merge catalog-sig and distutils-sig
On Thu, Mar 28, 2013 at 7:22 PM, Donald Stufft wrote: > Is there much point in keeping catalog-sig and distutils-sig separate? > > It seems to me that most of the same people are on both lists, and the topics > almost always have consequences to both sides of the coin. So much so that > it's often hard to pick *which* of the two (or both) lists you post too. > Further confused by the fact that distutils is hopefully someday going to go > away :) > > Not sure if there's some official process for requesting it or not, but I > think we should merge the two lists and just make packaging-sig to umbrella > the entire packaging topics. +1 -- Philippe Ombredanne +1 650 799 0949 | pombreda...@nexb.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Help with buildout
On Sun, Mar 31, 2013 at 12:32 AM, John Anderson wrote: > I'm trying to use buildout to pull in 4 github repositories and combine > their wsgi applications together using paste's urlmap. A few things aren't > working the way I expect and so I'm looking for some more information: > This is my buildout.cfg: > https://github.com/sontek/notaliens.com/blob/master/buildout.cfg There are issues with your buildout (git urls, colander, etc) and it does not work as-is and errors out. This is probably why you could not get to import paste. This works and has no issue https://gist.github.com/pombredanne/5281269 -- Philippe Ombredanne +1 650 799 0949 | pombreda...@nexb.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP: Improving Python ZIP Application Support
On Sat, Mar 30, 2013 at 8:22 PM, Daniel Holth wrote: > Python ZIP Application Support - > https://docs.google.com/document/d/1MKXgPzhWD5wIUpoSQX7dxmqgTZVO6l9iZZis8dnri78/edit?usp=sharing > PEP: 4XX > Title: Improving Python ZIP Application Support So I guess that this already-available-yet-hidden-or-little-known feature we had since Python 2.6 will be getting a little light. Let me ask a few silly questions: Does this means that any zip with a __main__.py is de-facto already executable? What about a wheel with a __main__ ? or an egg? Or a source archive where the __main__ calls setup.py install or buildout bootstrap? Is this something to promote? How is this overlapping with other packaging approaches? or possibly replacing them all? -- Philippe Ombredanne +1 650 799 0949 | pombreda...@nexb.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP: Improving Python ZIP Application Support
On Tue, Apr 2, 2013 at 12:40 AM, Nick Coghlan wrote: > On Tue, Apr 2, 2013 at 6:12 AM, M.-A. Lemburg wrote: >> c) The feature has never really been documented outside the ticket >> where it got added to Python :-) > Not true, it's covered in the command line docs at > http://docs.python.org/2/using/cmdline.html#interface-options The doc has a bug BTW: "Changed in version 2.5: Directories and zipfiles containing a __main__.py file at the top level are now considered valid Python scripts." I tested it and this does not work in Pythin 2.5, only in 2.6 -- Philippe Ombredanne +1 650 799 0949 | pombreda...@nexb.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] HTTPS and certificate check update for distribute ?
On Thu, Apr 25, 2013 at 4:42 PM, M.-A. Lemburg wrote: > The latest pip supports HTTPS URLs and certificate checks > (according to the change log). > Will there be a release of distribute that implements the > same changes ? And FWIW, the same question would be relevant for buildout that depends on distribute now. -- Philippe Ombredanne +1 650 799 0949 | pombreda...@nexb.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] buildout in offline mode - install receipts also from cache?
On Mon Dec 17 15:35:47 CET 2012, Anton Koval' psihonavt at gmail.com wrote: > in our project we've decided to completely move > buildout process to offline mode. > according to documentation > (http://pypi.python.org/pypi/zc.buildout/2.0.0a5) > I made updates in our main buildout configuration file, like: > download-cache = ... > install-from-cache = true > offline = true > But when buildout tries to install any part with recipe specified, > it is (obviously) failing, e.g.: > Error: Couldn't find a distribution for 'iw.recipe.cmd'. > So, my question is: are there ways to fetch recipes also from > some cache (or whatever) on local disk? I have experienced the same issue See https://github.com/buildout/buildout/issues/41 This is a bug and also per Jim Fulton a lack of specification/documentation of what offline really means. IMHO it is not possible today to bootstrap and buildout offline. Anyone with a better story there? That said the fixes are rather easy ... And I have an ugly but nicely working monkey patching extension that I will publish on pypi in a couple days... Which ironically enough needs to be installed with pip before a boostrap. Now Jim suggested I should bring the discussion here. In general installing packages without a network connection should be straight forward, simple and easy. That would be the common understanding of what offline means. It works fine with setuptools/distribute and pip albeit a tad circumvoluted and under documented. It does not with buildout and fails because with the offline flag/config, buildout calls its install function with a None dest argument making setuptools/distribute fail downstream with a weird exception as you reported. This affects all current and recent versions of buildout. -- Philippe Ombredanne nexB ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Re: Provide a way to bundle and extract license files
Steve, Nick: On Sun, Feb 23, 2020 at 10:04 AM Steve Jorgensen wrote: > > Nick Coghlan wrote: > > On Sun, 23 Feb 2020 at 08:40, Ian Stapleton Cordasco > > graffatcolmin...@gmail.com wrote: > > > > > > Forgive me if I'm missing something but doesn't > > > license-file provides this functionality (see > > > https://stackoverflow.com/a/48691876) for > > > an example. > > > I surmise not enough people use it although it's readily available? > > > This is likely to be the case, as license-file[s] is a setuptools > > feature aimed at ensuring the license file ends up in the sdist/wheel > > archive, rather than a published metadata field aimed at allowing > > other tools to find that license file within the sdist/wheel > > archive. > > There's a pre-draft PEP in discussion at > > https://github.com/pombredanne/spdx-pypi-pep/pull/2 > > and > > https://discuss.python.org/t/improving-license-clarity-with-better-package-m... > > that looks at clarifying licensing metadata through the use of SPDX > > classifiers. That draft PEP also formalises the "License-File" field. > > The approach I'm currently taking to this problem is to combine > > https://github.com/nexB/scancode-toolkit/blob/develop/README.rst > > for > > finding component licenses with > > https://github.com/nexB/aboutcode-toolkit > > to generate an open source > > attribution bundle for those components. The one key caveat on that > > approach is that the initial scancode output requires some non-trivial > > cleanup before you can feed it into the aboutcode ABOUT file generator > > when first applying it to a project: > > https://github.com/nexB/aboutcode-toolkit/issues/416 > > Cheers, > > Nick. > > P.S. As with a lot of distribution related issues, the key challenge > > with making improvements in this space is that developers really need > > tools that work today to meet their open source attribution > > obligations (such as nexB's scancode & aboutcode toolkits), while > > metadata level improvements (like Philippe's draft PEP) will take > > years to cover a significant proportion of published packages (and > > there's a long tail of rarely updated projects that may never catch > > up). > > Thanks for your very informative & useful reply. :) Nick: What you are doing with the scancode and aboutcode toolkits seems super yummy and would likely be super useful elsewhere! If you think there is something we could extract to make it part of the tools, I am game to help. And I need to submit that draft PEP BTW :] Steve: that PEP eventually documents the de-facto undocumented thing that includes license_file(s) in built wheels. The field already exists and is supported already so it can be used. To Nick's point it is going to take a long while to fix it all in the actual packages. That said, I am also involved in an initiative to help along the way and hopefully will help take only 100 years instead of the original thousand years needed to fix the problem (See https://clearlydefined.io ) There we are 1. scanning with scancode ALL the packages (Python + everything else if there is such a thing ;) ) 2. licensing data quality is "scored" with this approach https://github.com/clearlydefined/license-score/blob/master/ClearlyLicensedMetrics.md The license scoring includes if the full license text is present or not in the package (which is your original concern). 3. volunteers are reviewing that data for accuracy and correctness and fixing it if needed. 4. eventually fixes are pushed back upstream. There is also some Google summer of project https://github.com/nexB/aboutcode/wiki/Project-Ideas-Improve-License-Detection-Accuracy to do some large scale analysis of the 10M scans we have on hand. Do not hesitate to reach out on our off list. -- Cordially Philippe Ombredanne p...@nexb.com (scancode-toolkit maintainer) -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/archives/list/distutils-sig@python.org/message/NF7V7Z2LOBEQNAZ6WQBSXDAQFU6RDG7C/
