Re: [Distutils] pip/warehouse feature idea: "help needed"
On Mon, Apr 13, 2015 at 10:13 PM Donald Stufft wrote: > > > On Apr 13, 2015, at 8:57 PM, Ben Finney > wrote: > > > > Nick Coghlan writes: > > > >> On 11 Apr 2015 12:22, "Alexander Walters" > wrote: > >>> Is the package index really the best place to put this? This is a > >>> very social-networking feature for the authoritative repository of > >>> just about all the third party module, and it feels like either it > >>> could corrupt the 'sanctity' of the repository (in the absolute > >>> worst case) > >> > >> If you're concerned that this feature might weaken the comforting > >> illusion that PyPI published software is contributed and maintained by > >> faceless automatons rather than living, breathing human beings, then > >> yes, encouraging folks to think more about where the software they use > >> is coming from would be a large part of the point of adding such a > >> feature. > > > > I can't speak for Alexander, but I'm also −1 to have this *on PyPI*. > > > > I'm all for such features existing. What is at issue is whether PyPI is > > the place to put them. > > > > We have been gradually improving the function of PyPI as an > > authoritative *index* of packages; that's possible because it is a > > repository of uncontroversial facts, not opinions (i.e. “what is the > > packaging metadata of this distribution”, “where is its documentation”, > > “where is its VCS”, etc.). > > > >>> I am not saying the PSF shouldn't do this, but is pypi REALLY the > >>> best part of python.org to put it? > >> > >> I personally believe so, yes - sustaining software over the long term is > >> expensive in people's time, but it's often something we take for > granted. > >> The specific example Guido brought up in his keynote was the challenge > of > >> communicating a project's openness to Python 3 porting assistance. > > > > The people doing the work of maintaining PyPI have said many times in > > recent years that there just isn't enough person-power to add a whole > > bunch of features that have been requested. Why would we think > > moderating a social-networking rating, opinion, discussion, or other > > non-factual database is something reasonable to ask of the PyPI > > maintainers? > > > > Conversely, if we are under the impression that adding ratings, > > feedback, reviews, discussion, and other features to PyPI is *not* going > > to be a massive increase in workload for the maintainers, I think that's > > a foolish delusion which will be quite costly to the reputation PyPI has > > recently gained through hard effort to clarify its role. > > > > By all means, set up a well-maintained social ecosystem around Python > > packages. But not on PyPI itself: The Python Package Index is feasible > > in part because it has a clear and simple job, though, and that's not > > it. > > > > -- > > \“If you can't hear me sometimes, it's because I'm in | > > `\ parentheses.” —Steven Wright | > > _o__) | > > Ben Finney > > > > ___ > > Distutils-SIG maillist - Distutils-SIG@python.org > > https://mail.python.org/mailman/listinfo/distutils-sig > > > I don’t see any problem with the general idea of adding features to PyPI to > enable package maintainers to find more help maintaining specific parts of > their projects. I do have a problem with expecting the PyPI administrators > to fill out or otherwise populate this information. Saying “Here’s a place > you can donate to me” is still a fact, it’s just a more social fact than > what we currently enable. > > I’m kind of down on the idea of linking to CVs or linkedin as part of the > project metadata because that’s not project specific and is really more > maintainer specific. I think that particular feature would be better suited > to some sort of global “Python profile” that could then be linked to from > PyPI instead of trying to bake it into PyPI itself. > > However things like “Looking for New Maintainers / Orphan a Project”, > or some call to actions on “here are some issues that need fixed” or other > things doesn’t seem unreasonable to me. Particularly the ability to orphan > a project or look for new maintainers seems like a useful thing to me that > really can’t live anywhere other than PyPI reasonably. > I agree. Even something as simple as a boolean that triggers a banner saying "this project is looking for a new maintainer" would be useful both from the perspective of project owners who want to move on or from the perspective of users who can't tell if a project is maintained based on how long it has been since a project uploaded a new version (which is why I think someone suggested sending an annual email asking for a human action to say "alive and kicking" to help determine if a project is completely abandoned). -Brett ___ Distutils-SIG maillist - Distutils-SIG@pyt
Re: [Distutils] pip/warehouse feature idea: "help needed"
On Tue, Apr 14, 2015 at 8:34 PM Ben Finney wrote: > Nick Coghlan writes: > > > Yep, Guido's keynote was the genesis of the thread. > > I can't find it online, can you give a URL so we can see the talk? > https://www.youtube.com/watch?v=G-uKNd5TSBw > > > Past suggestions for social features have related to providing users > > with a standard way to reach maintainers and each other, and I'd > > prefer to leave maintainers in full control of that aspect of the > > maintainer experience. I'm not alone in feeling that way, hence why > > such features tend not to be viewed especially positively. > > Thanks for this detailed response differentiating this proposal from > previous ones, it's exactly what I was asking for. > > -- > \ “For mad scientists who keep brains in jars, here's a tip: why | > `\ not add a slice of lemon to each jar, for freshness?” —Jack | > _o__) Handey | > Ben Finney > > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Accessing package data files in wheels
I'm traveling so I can't do a thorough reply, but a goal of mine for Python 3.6 is finally solve the data access problem for packages based on Donald's importlib.resources proposal as well as pkg_resources to try and learn from previous mistakes. On Mon, Jun 29, 2015, 04:52 Paul Moore wrote: > On 29 June 2015 at 10:26, Paul Sokolovsky wrote: > > and yet we're stuck at the old base PEPs which > > overlooked providing stream access protocol for package resources > > access. > > The PEP did not "overlook" stream access. Rather, the compatibility > constraints and the need to support existing code meant that we needed > to ensure that we required the minimal possible interface from > loaders. Even get_data was an optional interface. > > In practice, many of the constraints around at the time no longer > apply, and zip and filesystem loaders remain the most common examples, > so the conservative approach of PEP 302 can be revisited (as I said). > But someone needs to step up and manage such a change before it will > happen. > > > Let that be rhetoric question then, and let everyone assume that so > > far trading eggs for wheels was more important than closing a visible > > accidental gap in the stdlib/loader API. > > The egg->wheel transition was about *distribution* formats. The loader > API is a runtime facility. The two are unrelated. > > One of the problems with eggs was the fact that they were a combined > installation and runtime format, so confusing the two aspects is > understandable (but still incorrect). > > > There was recent discussion on python-dev how other languages are > > cooler because they allow to create self-contained executables. Python > > has all parts of the equation already - e.g. the standard way to put > > Python source inside an executable is using frozen modules. So, that's > > another usecase for accessing package resources - was support for > > frozen modules was implemented at all? > > See > https://docs.python.org/3.5/library/importlib.html#importlib.machinery.FrozenImporter > > From that, it appears that the frozen module importer does not > implement the ResourceLoader API. So no, get_data is not supported for > frozen modules. Of course, you can write your own extension of > FrozenImporter for your application, so it's entirely possible to get > this to work. But the standard Python bootstrap process (which is > FrozenImporter's main use case, AFAIK) doesn't need that feature, > which is probably why it's not present. > > Anyway, as you can see, all the various mechanisms are available, and > extending importlib is certainly possible, so as we've said it's > really only about someone with the motivation doing the work. It could > probably even be done as a 3rd party project in the first place (much > like pkg_resources was) and then proposed for inclusion in the stdlib > once it has been found to be useful. > > Paul > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Making install a no-op
The way I handled itnin importlib was to version check in setup.py and if importlib would be in the stdlib I would use an empty list to represent what to install. That way users didn't have use markers and need to know importlib was in some specific version of Python (basically argparse has made people expect backports to just magically work). On Fri, Jul 10, 2015, 14:38 Ethan Furman wrote: > I have recently received a request to make installing enum34 a no-op on > Python3.4 and later so that wheels, etc, don't have to worry about the > Python version when dealing with Enum. > > From an enum34 point-of-view this makes sense since Enum is in the stdlib > in 3.4+, and enum34 has no purpose -- but how? Is it a simple matter of > checking for the Python version and raising > SystemExit if enum34 is not needed? > > -- > ~Ethan~ > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] dump of all PyPI project metadata available?
When I wrote https://nothingbutsnark.svbtle.com/python-3-support-on-pypi I wrote a script to download every project's JSON metadata by scraping the simple index and then making the appropriate GET request for the JSON metadata. It worked, but somewhat of a hassle. Is there some dump somewhere that is built daily, weekly, or monthly of all the metadata on PyPI for offline analysis? ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] dump of all PyPI project metadata available?
On Wed, Jul 22, 2015 at 2:19 PM Wes Turner wrote: > https://github.com/dstufft/pypi-stats > > https://github.com/dstufft/pypi-external-stats > I'm not quite sure what I'm supposed to get from those links, Wes, as that code still scrapes every project individually and downloads them while all I'm trying to avoid having to scrape PyPI and instead just download a single file (plus I don't want the files but just the metadata already returned by the JSON API). -Brett > - [ ] a flat bigquery w/ pandas.io.gbq ala GitHub Archive would be great > - [ ] it's probably worth it to add RDFa to PyPi and warehouse pages (in > addition to the auxiliary executed/extracted JSON) for #search > On Jul 22, 2015 4:08 PM, "Brett Cannon" wrote: > >> When I wrote https://nothingbutsnark.svbtle.com/python-3-support-on-pypi >> I wrote a script to download every project's JSON metadata by scraping the >> simple index and then making the appropriate GET request for the JSON >> metadata. It worked, but somewhat of a hassle. >> >> Is there some dump somewhere that is built daily, weekly, or monthly of >> all the metadata on PyPI for offline analysis? >> >> ___ >> Distutils-SIG maillist - Distutils-SIG@python.org >> https://mail.python.org/mailman/listinfo/distutils-sig >> >> ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] dump of all PyPI project metadata available?
On Wed, Jul 22, 2015 at 9:12 PM Wes Turner wrote: > > On Jul 22, 2015 5:12 PM, "Brett Cannon" wrote: > > > > > > > > On Wed, Jul 22, 2015 at 2:19 PM Wes Turner wrote: > >> > >> https://github.com/dstufft/pypi-stats > >> > >> https://github.com/dstufft/pypi-external-stats > > > > > > I'm not quite sure what I'm supposed to get from those links, Wes, as > that code still scrapes every project individually and downloads them while > all I'm trying to avoid having to scrape PyPI and instead just download a > single file (plus I don't want the files but just the metadata already > returned by the JSON API). > > An online query or an offline dump? > Offline dump. I literally just want a single file to download. Anyway, it's sounding like there isn't one currently so it would need to be a new feature for Warehouse. -Brett > > > > -Brett > > > >> > >> - [ ] a flat bigquery w/ pandas.io.gbq ala GitHub Archive would be great > > http://pandas.pydata.org/pandas-docs/version/0.16.2/io.html#io-bigquery > > >> - [ ] it's probably worth it to add RDFa to PyPi and warehouse pages > (in addition to the auxiliary executed/extracted JSON) for #search > > https://github.com/pypa/warehouse/blob/master/warehouse/packaging/models.py > > > https://github.com/pypa/warehouse/blob/master/tests/unit/packaging/test_models.py > > https://github.com/pypa/warehouse/blob/master/warehouse/packaging/views.py > > > https://github.com/pypa/warehouse/blob/master/warehouse/templates/packaging/detail.html > > https://github.com/pypa/warehouse/blob/master/warehouse/routes.py > > > https://github.com/pypa/warehouse/blob/master/tests/unit/legacy/api/test_json.py > > https://github.com/pypa/warehouse/blob/master/warehouse/legacy/api/json.py > > >> > >> On Jul 22, 2015 4:08 PM, "Brett Cannon" wrote: > >>> > >>> When I wrote > https://nothingbutsnark.svbtle.com/python-3-support-on-pypi I wrote a > script to download every project's JSON metadata by scraping the simple > index and then making the appropriate GET request for the JSON metadata. It > worked, but somewhat of a hassle. > >>> > >>> Is there some dump somewhere that is built daily, weekly, or monthly > of all the metadata on PyPI for offline analysis? > >>> > >>> ___ > >>> Distutils-SIG maillist - Distutils-SIG@python.org > >>> https://mail.python.org/mailman/listinfo/distutils-sig > >>> > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP for dependencies on libraries like BLAS (was: Re: Working toward Linux wheel support)
On Thu, 20 Aug 2015 at 10:16 Wes Turner wrote:
>
> On Aug 20, 2015 5:05 AM, "Nick Coghlan" wrote:
> >
> > [Catching up on distutils-sig after travel]
> >
> > On 13 August 2015 at 16:08, Nathaniel Smith wrote:
> > > It seems like a reasonable effort at solving this problem, and I guess
> > > there are probably some people somewhere that have this problem, but
> > > my concern is that I don't actually know any of those people. The
> > > developers I know instead have the problem of, they want to be able to
> > > provide a small finite number of binaries (ideally six binaries per
> > > Python version: {32 bit, 64 bit} * {windows, osx, linux}) that
> > > together will Just Work on 99% of end-user systems. And that's the
> > > problem that Enthought, Continuum, etc., have been solving for years,
> > > and which wheels already mostly solve on windows and osx, so it seems
> > > like a reasonable goal to aim for. But I don't see how this PEP gets
> > > us any closer to that.
> >
> > The key benefit from my perspective is that tools like pyp2rpm, conda
> > skeleton, the Debian Python packaging tools, etc, will be able to
> > automatically generate full dependency sets automatically from
> > upstream Python metadata.
> >
> > At the moment that's manual work which needs to be handled
> > independently for each binary ecosystem, but there's no reason it has
> > to be that way - we can do a better job of defining the source
> > dependencies, and then hook into release-monitoring.org to
> > automatically rebuild the downstream binaries (including adding new
> > external dependencies if needed) whenever new upstream releases are
> > published.
>
> JSON (JSON-LD) would likely be most platform compatible (and designed for
> interoperable graph nodes and edges with attributes).
>
> JSON-LD does not require a specific library iff the @context is not
> necessary.
>
> Notes about JSON-LD and interoperable software package metadata:
> https://mail.python.org/pipermail/distutils-sig/2015-April/026108.html
>
What does JSON-LD have to do with this conversation, Wes? No discussion of
implementation has even begun, let alone worrying about data formats. This
is purely a discussion of problem space and what needs to be solved and is
not grounded in concrete design yet.
___
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Unable to login to PyPi
On Tue, 29 Sep 2015 at 11:34 Mike O'Driscoll wrote: > Would someone be able to point me to the correct package/developer bring > up/testing instructions. > > Should I find extra cycles I could potentially investigate a fix. > > I think these might be the instructions you're looking for: https://wiki.python.org/moin/CheeseShopDev -Brett > > -- > Mike O'Driscoll > > On Tue, Sep 29, 2015 at 9:09 AM, Mike O'Driscoll > wrote: > >> I'm completely locked out. >> >> If there is/was a way to add pure username password I'll do that when I >> can get back in. >> >> >> -- >> Mike O'Driscoll >> >> On Mon, Sep 28, 2015 at 10:57 PM, Glyph Lefkowitz < >> gl...@twistedmatrix.com> wrote: >> >>> Mike, do you have another way to authenticate to the site, or are you >>> locked out until OpenID works again? >>> >>> -g >>> >>> On Sep 28, 2015, at 14:02, Richard Jones wrote: >>> >>> Hi Mike, >>> >>> Sorry, but this is a known problem that no-one has time to investigate >>> or fix. >>> >>> >>> Richard >>> >>> On 29 September 2015 at 01:31, Mike O'Driscoll >>> wrote: >>> Hello, I have been unable to login to the PyPi site for nearly a month now via OpenID (launchpad). I have the following ticket open but have gotten no traction: https://bitbucket.org/pypa/pypi/issues/333/unable-to-login-via-openid Any support would be appreciated. Thanks, -- Mike O'Driscoll ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig >>> ___ >>> Distutils-SIG maillist - Distutils-SIG@python.org >>> https://mail.python.org/mailman/listinfo/distutils-sig >>> >>> >>> >> > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Towards a simple and standard sdist format that isn't intertwined with distutils
On Fri, 2 Oct 2015 at 05:08 Donald Stufft wrote: > On October 2, 2015 at 12:54:03 AM, Nathaniel Smith (n...@pobox.com) wrote: > > > We realized that actually as far as we could tell, it wouldn't > > be that > > hard at this point to clean up how sdists work so that it would be > > possible to migrate away from distutils. So we wrote up a little > > draft > > proposal. > > > > The main question is, does this approach seem sound? > > I've just read over your proposal, but I've also just woken up so I might > be > a little slow still! After reading what you have, I don't think that this > proposal is the right way to go about improving sdists. > > The first thing that immediately stood out to me, is that it's recommending > that downstream redistributors like Debian, Fedora, etc utilize Wheels > instead > of the sdist to build their packages from. However, that is not really > going to > fly with most (all?) of the downstream redistributors. Debian for instance > has > policy that requires the use of building all of it's packages from Source, > not > from anything else and Wheels are not a source package. While it can > theoretically work for pure python packages, it quickly devolves into a > mess > when you factor in packages that have any C code what so ever. > So wouldn't they then download the sdist, build a wheel as an intermediate, and then generate the .deb file? I mean as long as people upload an sdist for those that want to build from source and a wheel for convenience -- which is probably what most people providing wheels do anyway -- then I don't see the problem. > > Overall, this feels more like a sidegrade than an upgrade. One major theme > throughout of the PEP is that we're going to push to rely heavily on > wheels as > the primary format of installation. While that works well for things like > Debian, I don't think it's going to work as wheel for us. If we were only > distributing pure python packages, then yes absolutely, however given that > we > are not, we have to worry about ABI issues. Given that there is so many > different environments that a particular package might be installed into, > all > with different ABIs we have to assume that installing from source is still > going to be a primary path for end users to install and that we are never > going > to have a world where we can assume a Wheel in a repository. > > One of the problems with the current system, is that we have no mechanism > by > which to determine dependencies of a source distribution without > downloading > the file and executing some potentially untrusted code. This makes > dependency > resolution harder and much much slower than if we could read that > information > statically from a source distribution. This PEP doesn't offer anything in > the > way of solving this problem. > Isn't that what the requirements and requirements-file fields in the _pypackage file provide? Only if you use that requirements-dynamic would it require execcuting arbitrary code to gather dependency information, or am I missing something? > > To a similar tune, this PEP also doesn't make it possible to really get at > any other metadata without executing software. This makes it pratically > impossible to safely inspect an unknown or untrusted package to determine > what > it is and to get information about it. Right now PyPI relies on the > uploading > tool to send that information alongside of the file it is uploading, but > honestly what it should be doing is extracting that information from > within the > file. This is sort of possible right now since distutils and setuptools > both > create a static metadata file within the source distribution, but we don't > rely > on that within PyPI because that information may or may not be accurate > and may > or may not exist. However the twine uploading tool *does* rely on that, and > this PEP would break the ability for twine to upload a package without > executing arbitrary code. > Isn't that only if you use the dynamic fields? > > Overall, I don't think that this really solves most of the foundational > problems with the current format. Largely it feels that what it achieves is > shuffling around some logic (you need to create a hook that you reference > from > within a .cfg file instead of creating a setuptools extension or so) but > without fixing most of the problems. The largest benefit I see to > switching to > this right now is that it would enable us to have build time dependencies > that > were controlled by pip rather than installed implicitly via the execution > of > the setup.py. That doesn't feel like a big enough benefit to me to do a > mass > shakeup of what we recommend and tell people to do. Having people adjust > and > change and do something new requires effort, and we need something to > justify > that effort to other people and I don't think that this PEP has something > we > can really use to justify that effort. > >From my naive perspective this proposal seems to
Re: [Distutils] Towards a simple and standard sdist format that isn't intertwined with distutils
On Sat, 3 Oct 2015 at 04:51 Paul Moore wrote: > On 3 October 2015 at 02:03, Nathaniel Smith wrote: > > In particular I hesitate a little bit to just drop in everything from > > PEP 426 and friends, because previous specs haven't really thought > > through the distinction between sdists and wheels -- e.g. if an sdist > > generates two wheels, they probably won't have the same name, > > description, trove classifiers, etc. They may not even have the same > > version (e.g. if two different tools with existing numbering schemes > > get merged into a single distribution -- esp. if one of them needs an > > epoch marker). So it may well make sense to have an "sdist description > > field", but it's not immediately obvious that it's identical to a > > wheel's description field. > > I can only assume you're using the term sdist differently from the way > it is normally used (as in the PUG glossary), because for me the idea > that a sdist generates multiple wheels makes no sense. > > If I do "pip wheel " I expect to get a single wheel that > can be installed to give the same results as "pip install > " would, The wheel install will work on my build machine, > and on any machine where the wheel's compatibility metadata (the > compatibility tags, currently) says it's valid. > > The above behaviour is key to pip's mechanism for auto-generating and > caching wheels when doing an install, so I don't see how it could > easily be discarded. > > If what you're calling a "sdist" doesn't work like this, maybe you > should invent a new term, so that people don't get confused? If it > *does* work like this, I don't see what you mean by a sdist generating > two wheels. > I think sdist is getting a bit overloaded in this discussion. From my understanding of what Paul and Donald are after, the process should be: VCS -> sdist/source wheel -> binary wheel. Here, "VCS" is literally a git/hg clone of some source tree. A "sdist/source wheel" is carefully constructed zip file that contains all the source code from the VCS necessary to build a binary wheel for a project as long as the proper dependencies exist (e.g., proper version of NumPy, BLAS in one of its various forms, etc.). The binary wheel is obviously the final artifact that can just be flat-out loaded by Python without any work. So Paul doesn't see sdist/source wheels producing more than one binary wheel because in its own way an sdist/source wheel is a "compiled" artifact of select source code whose only purpose is to generate a binary wheel for a single project. So while a VCS clone might have multiple subprojects, each project should generate a single sdist/source wheel. Now this isn't to say that an sdist/source wheel won't generate different *versions* of a binary wheel based on whether e.g. BLAS is system-linked, statically linked, or dynamically loaded from a Linux binary wheel. But the key point is that the sdist/source wheel still **only** makes one kind of project. >From this perspective I don't see Nathaniel's desire for installing from a VCS as something other than requiring a step to bundle up the source code into an sdist/source wheel that pip knows how to handle. But I think what Paul and Donald are saying is pip doesn't want to have anything to do with the VCS -> sdist/source wheel step and that is entirely up to the project to manage through whatever tooling they choose. I also view the sdist//source wheel as almost a mini-VCS checkout since it is just a controlled view of the source code with a bunch of helpful, static metadata for pip to use to execute whatever build steps are needed to get to a binary wheel. Or I'm totally wrong. =) But for me I actually like the idea of an sdist/source wheel being explained as "a blob of source in a structured way that can produce a binary wheel for the package" and a binary wheel as "a blob of bytes for a package that Python can import" and I'm totally fine having C extensions not just shuttle around a blind zip but a structured zip in the form of an sdist/source wheel. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Towards a simple and standard sdist format that isn't intertwined with distutils
Listening in to an On Air session is unbounded, but direct participants are capped at 10. On Fri, 9 Oct 2015 at 09:17 Ionel Cristian Mărieș wrote: > > On Fri, Oct 9, 2015 at 7:05 PM, Chris Barker > wrote: > >> How many people can join a hangout? we may be bumping up against that >> limit :-) > > > AFAIK there's no limit on the number of people that can listen in. Also, > Hangouts can record video (the "On Air" thing). > > > > Thanks, > -- Ionel Cristian Mărieș, http://blog.ionelmc.ro > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] py2dsc --with-python3=True not producing python3- packages
Distutils-sig has nothing to do with py2dsc so I doubt anyone can really help here unfortunately. I would try reaching out to py2dsc or the flask-login team. On Wed, 21 Oct 2015 at 13:51 Marques Johansson wrote: > I assume I am doing something wrong, but the man page seems a bit light on > the matter. Does it have something to do with "python setup.py" being > called instead of "python3 setup.py". I have python3-stdeb=0.8.2-4 > installed. > > wget > https://pypi.python.org/packages/source/F/Flask-Login/Flask-Login-0.3.2.tar.gz > py2dsc --with-python3=True --no-python2-scripts=True > --no-python3-scripts=False --with-python2=False Flask-Login-0.3.2.tar.gz > > -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- > running the following command in directory: > deb_dist/tmp_py2dsc/Flask-Login-0.3.2 > /usr/bin/python setup.py --command-packages stdeb.command sdist_dsc > --dist-dir=/home/marques/src/deb_dist > --use-premade-distfile=/home/marques/src/Flask-Login-0.3.2.tar.gz > --with-python2=False --with-python3=True --no-python2-scripts=True > --no-python3-scripts=False > -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- > running sdist_dsc > running egg_info > writing requirements to Flask_Login.egg-info/requires.txt > writing Flask_Login.egg-info/PKG-INFO > writing top-level names to Flask_Login.egg-info/top_level.txt > writing dependency_links to Flask_Login.egg-info/dependency_links.txt > reading manifest file 'Flask_Login.egg-info/SOURCES.txt' > reading manifest template 'MANIFEST.in' > writing manifest file 'Flask_Login.egg-info/SOURCES.txt' > CALLING dpkg-source -b flask-login-0.3.2 flask-login_0.3.2.orig.tar.gz (in > dir /home/marques/src/deb_dist) > dpkg-source: info: using source format `3.0 (quilt)' > dpkg-source: info: building flask-login using existing > ./flask-login_0.3.2.orig.tar.gz > dpkg-source: info: building flask-login in > flask-login_0.3.2-1.debian.tar.xz > dpkg-source: info: building flask-login in flask-login_0.3.2-1.dsc > dpkg-buildpackage: source package flask-login > dpkg-buildpackage: source version 0.3.2-1 > dpkg-buildpackage: source distribution unstable > dpkg-buildpackage: source changed by Matthew Frazier < > leafstormr...@gmail.com> > dpkg-source --before-build flask-login-0.3.2 > fakeroot debian/rules clean > dh clean --with python3 --buildsystem=pybuild >dh_testdir -O--buildsystem=pybuild >dh_auto_clean -O--buildsystem=pybuild > I: pybuild base:170: python3.4 setup.py clean > running clean > removing > '/home/marques/src/deb_dist/flask-login-0.3.2/.pybuild/pythonX.Y_3.4/build' > (and everything under it) > 'build/bdist.linux-x86_64' does not exist -- can't clean it > 'build/scripts-3.4' does not exist -- can't clean it >dh_clean -O--buildsystem=pybuild > dpkg-source -b flask-login-0.3.2 > dpkg-source: info: using source format `3.0 (quilt)' > dpkg-source: info: building flask-login using existing > ./flask-login_0.3.2.orig.tar.gz > dpkg-source: warning: ignoring deletion of directory Flask_Login.egg-info > dpkg-source: warning: ignoring deletion of file > Flask_Login.egg-info/dependency_links.txt, use --include-removal to override > dpkg-source: warning: ignoring deletion of file > Flask_Login.egg-info/not-zip-safe, use --include-removal to override > dpkg-source: warning: ignoring deletion of file > Flask_Login.egg-info/PKG-INFO, use --include-removal to override > dpkg-source: warning: ignoring deletion of file > Flask_Login.egg-info/requires.txt, use --include-removal to override > dpkg-source: warning: ignoring deletion of file > Flask_Login.egg-info/SOURCES.txt, use --include-removal to override > dpkg-source: warning: ignoring deletion of file > Flask_Login.egg-info/top_level.txt, use --include-removal to override > dpkg-source: warning: ignoring deletion of file > Flask_Login.egg-info/version_info.json, use --include-removal to override > dpkg-source: info: building flask-login in > flask-login_0.3.2-1.debian.tar.xz > dpkg-source: info: building flask-login in flask-login_0.3.2-1.dsc > dpkg-genchanges -S -sa >../flask-login_0.3.2-1_source.changes > dpkg-genchanges: including full source code in upload > dpkg-source --after-build flask-login-0.3.2 > dpkg-buildpackage: full upload (original source is included) > dpkg-source: warning: extracting unsigned source package > (flask-login_0.3.2-1.dsc) > dpkg-source: info: extracting flask-login in flask-login-0.3.2 > dpkg-source: info: unpacking flask-login_0.3.2.orig.tar.gz > dpkg-source: info: unpacking flask-login_0.3.2-1.debian.tar.xz > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Time for a setuptools_lite??
On Wed, 21 Oct 2015 at 10:17 Paul Moore wrote: > On 21 October 2015 at 17:42, Chris Barker wrote: > > So why not have a setuptools-lite that only does the building? We need to > > keep the full over-burdened setuptools around, because lot sof folks are > > using those features. But for those of us that are doing something fairly > > new, and don't want to use stuff that setuptools "shouldn't" be doing, > I'd > > love to have a setuptools-lite that only did what I really need, and was > > guaranteed NOT to accidentally introduce easy_install, etc... > > In general, this sounds like a reasonable idea. But, there are the > usual problems: > > 1. Someone needs to do the work. > 2. Backward compatibility (as in, there's bound to be some projects > out there using whatever functionality you can think of that "nobody > would ever need"...) > > And a third issue specific to this idea. > > 3. You can't call it setuptools, and pip "injects" the real setuptools > into projects when building/installing. Pretending to be setuptools > was what made distribute such a PITA to deal with, and I don't imagine > we'd ever want to go there again. > > Sorry, I'm being a downbeat grump today. The reality is that far and > away the best shot we've had at getting out of this mess is the plan > we're currently working to, which is to standardise interfaces, and > once we have some standards, we can start decoupling the various bits > and *then* replacing them as needed. We need people working on that in > the first instance, work is currently stalled by lack of time from the > key participants. So anyone with energy for moving this whole area > forward would be really welcome to help out with the interoperability > standards (metadata 2.0, defining a sdist format, pinning down pip's > interfaces to build systems, things like that). But if you want to > rework the existing toolset, you're probably going to be on your own > (and there's unfortunately a risk that you'll pull attention away from > other work via debates on this list etc). > > I really appreciate the enthusiasm we're seeing from people on the > list at the moment, it's a matter of harnessing that into the right > direction. As a thought, does anyone feel like working on an > "interoperability standards roadmap" that provides a reference for > where the PyPA see things going? It would be useful to keep focus, but > the information's currently scattered about various mailing lists and > discussions - some of the posts around the recent "new source > distribution format" threads might be a good place to start. > Would it makes sense to start a roadmap doc/repo under the PyPA account so the current grand vision is written down in a very high-level overview and then what the issues needing solving for each bit are along with any in-progress work are kept in a single place (I don't care if this is one big doc or there is the overview doc and then an individual doc per part of the grand vision)? Putting it up on GitHub allows for quick PRs on some Markdown doc that people can do quickly after an email thread reaches some form of a conclusion. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] wacky idea about reifying extras
On Tue, 27 Oct 2015 at 02:17 Robert Collins wrote: > On 27 October 2015 at 21:47, David Cournapeau wrote: > > > Another simple solution for this particular case is to add conflict rules > > between packages that provide the same requirement (that's what php's > > composer do IIRC). > > > > The case of safety against malicious forks is handled quite explicitly in > > composer, we may want to look at how they do it when considering > solutions > > (e.g. https://github.com/composer/composer/issues/2690, though it has > > changed a bit since then) > > > > Adding the provides/conflict concepts to pip resolver will complexify it > > quite significantly, both in terms of running time complexity (since at > that > > point you are solving a NP-complete problem) and in terms of > implementation. > > But we also know for real cases this is doable, even in pure python > > (composer handles all the cases you are mentioning, and is in pure php). > > We already require a full NP-complete solver because of <, <= and ~ > version operators. > > I haven't adsorbed this proposal enough to comment on the reification > aspect yet. > > I'm worried about provides and conflicts in general, but not from a > resolver code perspective - thats a one-ish-time-cost, but instead > from a user experience perspective. > So from my perspective as someone who (I think) grasps what the problems that everyone is trying to solve is but not knowing enough to know how stuff is done now (all my projects on PyPI are pure Python), Nathaniel's proposal makes total sense to me. I would think it would be easy to explain to a scientist that "to get scipy, run `python3.5 -m pip install numpy`, but if you want fast over open source and use Intel's MKL library, do `python3.5 -m pip install numpy[mkl]`. I think the syntax clearly shows it's a modification/tweak/special version of numpy and it makes sense that I want to install something that provides numpy while relying on MKL. Nathaniel's comment about how this might actually give pip a leg up on conda also sounds nice to me as I have enough worry about having a fissure in 1D along the Python 2/3 line, and I'm constantly worried that the scientific community is going to riot and make it a 2D fissure along Python 2/3, pip/conda axes and split effort, documentation, etc. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] A smaller step towards de-specializing setuptools/distutils
On Thu, 29 Oct 2015 at 12:03 Donald Stufft wrote: > On October 29, 2015 at 2:54:19 PM, Daniel Holth (dho...@gmail.com) wrote: > > I think that would be very handy, especially making setuptools not a > > special case. You could get it down to 3 lines in setup.cfg, a file that > > already exists and already gathers random settings. > > > > > > I don’t think we should try to use an entry point (at least not yet). One > of the key points is to minimize the conceptual churn and to move forward > slowly while still progressing. I don’t think that using a class really > buys us much because anyone who is capable of writing that class is also > capable of writing a small CLI interface to be used within a setup.py. It > feels more like churn for churn’s sake than anything else and some sort of > idealogical purity over pragmatism. > There is also the familiarity of standardizing the CLI that pip will use when calling setup.py. The one thing that Daniel's case 3 proposal has over Donald's, though, is that all data can be self-contained in setup.cfg. If a build tool chooses to have the user specify all data in setup.cfg then you just copy-and-paste the `[bootstrap]` part and then in some other section they specify the info the build tool wants. Donald's approach could do something similar but it also requires copying around a boilerplate setup.py that is never modified. It's a familiarity vs. simplicity issue. Personally I still like the simple setup.py approach for the familiarity of it as well as the backwards-compatibility/transition bit being so simple with older versions of setuptools. > > I also don’t want to base any standard on setuptools entry points because > they are not a standard (and if we did, the key should not mention pip at > all, pip shouldn’t be special cased any more than setuptools should). > Well, it kind of is a standard thanks to Paul and zipapp: https://docs.python.org/3/library/zipapp.html#module-zipapp > > - > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 > DCFA > > > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Please don't impose additional barriers to participation (was: build system abstraction PEP)
On Thu, 29 Oct 2015 at 22:53 Marcus Smith wrote: > > >> If python-dev ends up adopting GitLab for the main PEPs repo, then we >> should be able to move the whole process there, rather than needing to >> maintain a separate copy. >> > will that be as open as pypa/interoperability-peps? > Proof of concept and final proposal isn't in front of me yet so it isn't known. I think Nick's hope is that it will be as open. -Brett > if it's closed off such that only python devs can log PRs against PEPs > once they're in the system, then that's no fun. > If so, I'd still want to keep pypa/interoperability-peps as the front end > tool for the actual change management. > > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PyPA Roadmap
Thanks for writing the roadmap, Marcus! Heck of a lot easier to understand what is or is not sitting at a PEP at the moment. On Mon, 2 Nov 2015 at 18:13 Marcus Smith wrote: > Based on discussions in another thread [1], I've posted a PR to pypa.io > for a "PyPA Roadmap" > > PR: https://github.com/pypa/pypa.io/pull/7 > built version: http://pypaio.readthedocs.org/en/roadmap/roadmap/ > > To be clear, I'm not trying to dictate anything here, but rather just > trying to mirror what I think is going on for the sake of new (or old) > people, who don't have a full picture of the major todo items. > > I'm asking for help to make this as accurate as possible and to keep it > accurate as our plans change. > > thanks, > Marcus > > [1] > https://mail.python.org/pipermail/distutils-sig/2015-October/027346.html > , although it seems a number of emails in this thread never made it to the > archive due to the python mail server failure. > > > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] The future of invoking pip
On Wed, 11 Nov 2015 at 04:06 Paul Moore wrote: > On 11 November 2015 at 06:35, Nick Coghlan wrote: > > Windows Python 2 installations require manual PATH modifications > > regardless, but it's more common for people to know how to make > > "python -m pip install X" work, than it is for them to remember to > > also add the "Scripts" directory needed to make "pip install X" work. > > ... and "py -m pip install X" works without any PATH modification on > all Windows systems with the launcher installed (I can't recall if > it's included with Python 2.7 - but if not, maybe it should be > backported? There's a standalone version people can get as well). > While the discussion to try and get UNIX to adopt `py` is nice, I think that decision falls under python-dev's jurisdiction. So if people here decide "we should be pushing for that" then that's great, but that means someone needs to go to python-dev and say "distutils-sig is trying to solve the issue of `pip` being ambiguous as to what Python installation it works with and we thought making `py` a thing on UNIX was the best solution forward for `py -m pip`". And if that's the case then the stop-gap is `python -m pip`. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package classifiers - both major and minor Python versions?
On Thu, 21 Jan 2016 at 06:48 John Whitlock wrote: > A discussion about the Python language classifiers came up in a pull > request [1], and I couldn't find a definite answer. The question is - > should a packager specify the major Python versions, minor Python versions, > or both? > > The Python Packaging User Guide's example [2] has both: > > # Specify the Python versions you support here. In particular, ensure > # that you indicate whether you support Python 2, Python 3 or both. > 'Programming Language :: Python :: 2', > 'Programming Language :: Python :: 2.6', > 'Programming Language :: Python :: 2.7', > 'Programming Language :: Python :: 3', > 'Programming Language :: Python :: 3.2', > 'Programming Language :: Python :: 3.3', > 'Programming Language :: Python :: 3.4', > > In the example, 'Programming Language :: Python :: 2' is a major version, > and 'Programming Language :: Python :: 2.7' is a minor version. > > pyroma [3], which I use as a packaging linter, has insisted on both the > major and minor versions since the first release in 2011 [4]. > > These were added in 2008, but the announcement on this mailing list didn't > include guidance on usage [5]. I can't find any guidance in PEPs either. > You should at least do the major versions, and if you are up for maintaining them, then do the minor ones as well. You want the major ones to tell people whether you even support Python 3 or not. Various tools like caniusepython3 rely on at least the major version classifier existing to programmatically know about Python 3 support. Minor support is good to let people know what versions you have tested against and officially support. This is especially useful right after a new version of Python is released as it lets people know whether you have tested against the new release yet or not. It also lets people know if you have dropped support for an older version of Python. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 513: A Platform Tag for Portable Linux Built Distributions Version
On Tue, 26 Jan 2016 at 10:59 Robert T. McGibbon wrote: > On Mon, Jan 25, 2016 at 1:51 PM, Robert T. McGibbon > wrote: > >> >> HTML version: >> https://github.com/manylinux/manylinux/blob/master/pep-513.rst >> > > Nick, at some point would it be possible to put update the version in the > PEPs repo to > this version? > The easiest way to get a PEP updated for python.org/dev/peps is to email p...@python.org with a patch containing the update. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 513: A Platform Tag for Portable Linux Built Distributions Version
On Tue, 26 Jan 2016 at 14:36 Nathaniel Smith wrote: > On Mon, Jan 25, 2016 at 8:37 PM, Robert T. McGibbon > wrote: > > I agree that this is an important detail. I generally use machines that > have > > many different Python interpreters installed (some distro-provided and > > others in my home directory), and can easily imagine wanting them to have > > different behavior w.r.t. manylinux1 wheels. > > > > Perhaps the option could be put in site.py, or somewhere in > > lib/pythonX.Y/configX.Y/? I'm not sure what the appropriate solution here > > is. > > On further thought, the site.py solution has sorta grown on me... > What if someone runs Python with `-S`? ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] How to declare optional dependencies?
Maybe I'm totally overlooking something or misreading the docs, but I can't find a way to say in a requirements.txt file that a dependency is optional and its failure to install is okay. E.g., aiohttp supports using cchardet as an accelerator of chardet ( http://aiohttp.readthedocs.org/en/stable/#library-installation). I would like to be able to specify in my requirements.txt that I would like cchardet to be installed if possible, but it not being available -- which it might be as it doesn't have any Python 3.5 wheels ATM -- is fine and not the end of the world. I'm also happy to push a patch upstream if there is something aiohttp should be setting in their setup.py instead (I thought maybe extras, but those seem to be hard requirements as well). ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Ensuring source availability for PyPI entries / PEP: Build system abstraction for pip/conda etc
On Thu, 11 Feb 2016 at 10:01 M.-A. Lemburg wrote: > On 11.02.2016 17:48, Donald Stufft wrote: > > > >> On Feb 11, 2016, at 11:08 AM, M.-A. Lemburg wrote: > >> > >> Then why not fix distutils' sdist command to add the needed > >> information to PKG-INFO and rely on it ? > >> > >> Or perhaps add a new distutils command which outputs the needed > >> information as JSON file and fix the sdist command to call this > >> by default ? > >> > >> There are many ways to address such issues, but defining a new > >> standard for every issue we have instead of fixing the existing > >> distutils implementation is not the best way to approach this. > > > > > > The very nature of distutils (later inherited by setuptools) is the > problem to > > be honest. The reason we're adding new standards and moving away from > these > > systems is that fixing them is essentially fundamentally altering them. > > Of course. We're doing that constantly in Python, so why not > in distutils too ? > IMO, I think we should work towards a goal where we strip distutils down to only the parts that are required to be provided by Python to make it easier to maintain. Distutils served its purpose, but now I think we should push what distutils does out to the community to allow for more rapid updates and easier maintenance and evolution which will better align with things like compiler release schedules and support alternative compilers and such more easily. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] What's up with PyPi maintenance?
On Thu, 17 Mar 2016 at 17:51 Chris Barker - NOAA Federal < chris.bar...@noaa.gov> wrote: > > b) Is there a serious lack of folks available to address such issues? >> > > This is the correct answer. I can't even keep on top of the request from > people wanting to take over packages. > > > if (b), should we make a concerted effort to recruit new folks to assist >> PyPi maintenance / development??? >> > > That would be nice, but would be pointless until Warehouse goes live. > > > When do we expect that? A lot of people rely on the current system, we > really need to find a way to maintain it 'till it's replaced. > Do realize that this is in regards to the search box, not the ability for PyPI to keep standing to serve files. Obviously if something happened to file serving or some critical aspect of PyPI then that would get addressed ASAP. But fixing a search box is not critical infrastructure so I don't think it warrants a "all hands on deck" call for help over trying to get the replacement code done instead (I get people use the search box, but there are well-known alternatives for searching PyPI). In all honesty, if people want a functioning search box on PyPI I would say they should submit a patch to use https://cse.google.com/cse/ . ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] What's up with PyPi maintenance?
On Tue, 15 Mar 2016 at 09:42 Chris Barker wrote: > Probably not the right list but: > > There are a number of folks having issue siwth new PyPi pacakges not being > found by search: > > > https://bitbucket.org/pypa/pypi/issues/412/my-package-doesnt-show-up-in-search > > clearly issues have been posted to the project, but has been going on a > for a while, with seemingly no action, or response from PyPi > developers/maintainers. > Donald can correct me where I'm wrong, but from my understanding ... > > a) Is there somewhere else we can ping the right fols? > This is as good as any. > > b) Is there a serious lack of folks available to address such issues? > Yes. > > if (b), should we make a concerted effort to recruit new folks to assist > PyPi maintenance / development??? > There are been a couple of calls for help with Warehouse which will replace PyPI as soon as it's finished: https://github.com/pypa/warehouse (the current PyPI code base is really antiquated and from my understanding a real burden to work with at this point). -Brett > > Thanks, > > -CHB > > > > > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R(206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > chris.bar...@noaa.gov > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python Project distribution
https://packaging.python.org/en/latest/ should have all the information you need to learn how to package up and distribute your project. On Tue, 26 Apr 2016 at 05:42 akash chaudhary wrote: > Hi > > I want to distribute my python project that select best proxy and changes > it to chrome browser or running current application that uses internet > behind proxy.it has also features of username and password in it. > I want to know how to distribute my python setup using distutils. > Please response me soon as possible. > > Thanking You. > > Akash Chaudhary > +918979078763 > MNNIT,Allahabad > India > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] moving things forward (was: wheel including files it shouldn't)
On Wed, 27 Apr 2016 at 10:53 Donald Stufft wrote: > This isn't really a problem with what you're doing. Rather it's an issue > with the toolchain and and open question whether or not wheels should > conceptually be able to be produced from a checkout, or if they should only > be produced from a sdist. Problems like this are why I advocate the > Checkout -> sdist -> wheel being the only path, but others feel differently. > And Daniel Holth said he did feel differently, so obviously this hasn't moved forward in terms of finding consensus. Where does all of this sit in regards to trying to separate building from installation? From my perspective as mailing list lurker, it's sitting at an impasse as Nick hasn't made a final call nor has a BDFL delegate on the topic been chosen to settle the matter (obviously if I missed something then please let me know). Could we choose a Great Decider on this topic of build/installation separation and get final RFC/PEPs written so we can get passed this impasse and move forward? -Brett > > > — Donald Stufft > > On Apr 27 2016, at 1:38 pm, Ethan Furman wrote: > >> On 04/26/2016 07:10 AM, Donald Stufft wrote: >> >> > Alternatively, he could have just produced a wheel from any checkout at >> > all if the MANIFEST.in excluded a file that would otherwise have been >> > installed. >> >> Yes. My MANIFEST.in starts with an 'exclude enum/*' and then includes >> all files it wants. >> >> > This sort of thing is why I'm an advocate that we should only >> > build sdists from checkouts, and wheels from sdists (at the low level >> > anyways, even if the UI allows people to appear to create a wheel >> > straight from a checkout). >> >> My current process is: >> >>python3.5 setup.py sdist --format=gztar,zip bdist_wheel upload >> >> What should I be doing instead? >> >> -- >> ~Ethan~ >> >> ___ >> Distutils-SIG maillist - Distutils-SIG@python.org >> https://mail.python.org/mailman/listinfo/distutils-sig >> > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] moving things forward (was: wheel including files it shouldn't)
Thanks for the update! Glad this is still moving forward. I'll continue to prod the list if things stall again as I want to respond to "Python packaging is broken" with "actually your knowledge is just outdated, go read packaging.python.org". :) On Tue, 3 May 2016 at 11:28 Nathaniel Smith wrote: > On Tue, May 3, 2016 at 10:10 AM, Paul Moore wrote: > > On 3 May 2016 at 17:47, Donald Stufft wrote: > >> It will likely get decided as part of the build system PEP, whenever > that > >> gets picked up again. > > > > Yes, but on 15th March > > (https://mail.python.org/pipermail/distutils-sig/2016-March/028457.html) > > Robert posted > > > >> Just to set expectations: this whole process seems stalled to me; I'm > >> going to context switch and focus on things that can move forward. > >> Someone please ping me when its relevant to put effort in again :). > > > > And I think that's right. The whole build system PEP issue appears > > stalled from a lack of someone willing (or with the time) to make a > > call on the approach we take. > > No, no, Nick's not the blocker. I'm the blocker! (Sorry) > > Donald + Robert + I had a longish conversation about this on IRC a > month ago [1]. I volunteered to summarize back to the mailing list, > and then I flaked -- so I guess this is that belated email :-). > > Here's the tentative conclusions we came to: > > Blocker 1 is figuring out what to do about the sdist format. The > debate is between keeping something that's basically the current > format, versus somehow cleaning it up (e.g. Donald's "source wheel" > ideas). To move forward: > - I'll write up a PEP that attempts to just document/standardize the > current de facto sdist format and send it to the mailing list > (basically: filename convention, PKG-INFO + a list of which fields in > PKG-INFO pypi actually cares about, presence of setup.py), and adds > some sort of optional-defaulting-to-1 SDist-Version (I guess in a file > called SDIST by analogy with WHEEL). And also contains a rationale > section explaining the trade-offs of standardizing this versus > creating a new extension.) > - Donald will make his case for the new extension approach on the mailing > list > - We beg Nick to read over both things and make a ruling so we can move on > > Blocker 2 is figuring out whether the new pip <-> build system "hook" > interface should be command-line based (like the current draft of PEP > 516) or Python function call based (like the current draft of PEP > 517). It sounds like currently Donald and I are in favor of the python > hooks approach, and Robert is indifferent between them and just wants > to move forward, so we agreed that unless anyone objects we'll drop > the command-line approach and go ahead with refining the Python > function call approach. So... if you want to object then speak up now. > > Then there are a bunch of details to work out about what hooks to > provide exactly and what their semantics should be, but hopefully once > we've settled the two issues above that will be an easier discussion > to have. > > So yeah, basically the next step is for me [2] to write up a spec for > how sdists currently (really) work. > > -n > > [1] Logs (split across two pages in the log viewer): > http://chat-logs.dcpython.org/day/pypa-dev/2016-03-30#18.23.46.njs > http://chat-logs.dcpython.org/day/pypa-dev/2016-03-31#00.29.32.lifeless > [2] Or if someone else wants to raise their hand and volunteer I > wouldn't object, obviously I am a bit swamped right now :-) > > -- > Nathaniel J. Smith -- https://vorpus.org > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] who is BDFL for the boostrap/requires declaration? (was: moving things forward)
The emails seem to have reached an equilibrium point of bikeshedding on the (bootstrap|setup)_requires issue that is being discussed (as Daniel points out below, this has nothing to do with how building works and instead is only about statically declaring what tools need to be installed to simply run your builder to do whatever the heck it wants; this is the first baby step to build decoupling/customization). So who is the BDFL on this decision? It seems we need someone to stop the bikeshedding on the field name and what file is going to house this configuration data. And do we need someone to write a PEP for this proposal to have something to target? On Thu, 5 May 2016 at 13:54 Daniel Holth wrote: > This is a recurring point of confusion. setup.py is not ever going away. > In general it is necessary for you to be able to write software to build > your software, and there is no intention to take that feature away. > > People repeatedly come to the conclusion that static metadata means the > entire build is static. It's only the dependencies that need to be static > to enable better dependency resolution in pip. The build does not need to > be static. > > The proposed feature means you will be able to have a simpler setup.py or > no setup.py it by using something like flit or pbr that are configured with > .cfg or .ini. setup.py is not going away. > > Static metadata means the list of dependencies, author name, trove > classifiers is static. Not the build itself. > > Enforced staticness of the build script is right out. > > On Thu, May 5, 2016 at 4:34 PM Alex Grönholm > wrote: > >> I think it would be best to gather a few extreme examples of setup.py >> files from real world projects and figure out if they can be implemented in >> a declarative fashion. That at least would help us identify the pain points. >> >> For starters, gevent's setup.py looks like it needs a fair bit of custom >> logic: >> https://github.com/gevent/gevent/blob/master/setup.py >> >> 05.05.2016, 23:30, Chris Barker kirjoitti: >> >> On Wed, May 4, 2016 at 7:45 PM, Nick Coghlan wrote: >> >> >>> This configuration vs customisation distinction is probably worth >>> spelling out for folks without a formal software engineering or >>> computer science background, so: >>> >> >> fair enough -- good to be clear on the terms. >> >> >>> Configuration is different: you're choosing amongst a set of >>> possibilities that have been constrained in some way, and those >>> constraints are structurally enforced. >> >> >> That's a key point here -- I guess I'm skeptical that we can have the >> flexibility we need with a purely configuration-based system -- we probably >> don't WANT to constrain the options completely. If you think about it, >> while distutils has it's many, many flaws, what made it possible for it to >> be as useful as it is, and last as long as it has because is CAN be >> customized -- users are NOT constrained to the built-in functionality. >> >> I suspect the idea of this thread is to keep the API to a build system >> constrained -- and let the build systems themselves be as customizable as >> the want to be. And I haven't thought it out carefully, but I have a >> feeling that we're going to hit a wall that way .. but maybe not. >> >> >>> Usually that enforcement is >>> handled by making the configuration declarative - it's in some passive >>> format like an ini file or JSON, and if it gets too repetitive then >>> you introduce a config generator, rather than making the format itself >>> more sophisticated. >>> >> >> OK -- that's more or less my thought -- if it's python that gets run, >> then you've got your config generator built in -- why not? >> >> >> >>> The big advantage of configuration over customisation is that you >>> substantially increase the degrees of freedom in how *consumers* of >>> that configuration are implemented - no longer do you need a full >>> Python runtime (or whatever), you just need an ini file parser, or a >>> JSON decoder, and then you can look at just the bits you care about >>> for your particular use case and ignore the rest. >>> >> >> Sure -- but do we care? this is about python packaging -- is it too big a >> burden to say you need python to read the configuration? >> >> -CHB >> >> -- >> >> Christopher Barker, Ph.D. >> Oceanographer >> >> Emergency Response Division >> NOAA/NOS/OR&R(206) 526-6959 voice >> 7600 Sand Point Way NE (206) 526-6329 fax >> Seattle, WA 98115 (206) 526-6317 main reception >> >> chris.bar...@noaa.gov >> >> >> >> ___ >> Distutils-SIG maillist - >> Distutils-SIG@python.orghttps://mail.python.org/mailman/listinfo/distutils-sig >> >> ___ >> Distutils-SIG maillist - Distutils-SIG@python.org >> https://mail.python.org/mailman/listinfo/distutils-sig >> > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > htt
Re: [Distutils] who is BDFL for the boostrap/requires declaration? (was: moving things forward)
On Fri, 6 May 2016 at 09:40 Donald Stufft wrote: > > On May 6, 2016, at 12:36 PM, Brett Cannon wrote: > > So who is the BDFL on this decision? It seems we need someone to stop the > bikeshedding on the field name and what file is going to house this > configuration data. And do we need someone to write a PEP for this proposal > to have something to target? > > > We need someone to write the PEP and someone to offer to be BDFL for it. > For this particular change the default would be Nick for BDFL but if he’s > busy someone else can take it over for this PEP. Though I think we need > someone writing an actual PEP first. > OK, assuming the Nick will be pronouncing, who wants to write the PEP? ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] comparison of configuration languages
On Sat, 7 May 2016 at 07:49 Nick Coghlan wrote: > > On 7 May 2016 13:00, "Nathaniel Smith" wrote: > > > > Here's that one-stop writeup/comparison of all the major configuration > > languages that I mentioned: > > > > https://gist.github.com/njsmith/78f68204c5d969f8c8bc645ef77d4a8f > > Thanks for that, and "yikes" on the comment handling variations in > ConfigParser - you can tell I've never even tried to use end-of-line > comments in INI files, and apparently neither has anyone I've worked with :) > Yeah, that's pretty bad. :/ I checked when ConfigParser was added to Python and it's late 1997: https://hg.python.org/cpython/rev/5b24cbb1f99b, so rather old and predates our stricter code quality rules for additions to the stdlib. > For YAML, my main concern isn't quirkiness of the syntax, or code quality > in PyYAML, it's the ease with which you can expose yourself to security > problems (even if *pip* loads the config file safely, that doesn't mean > every other tool will). Since we don't need the extra power, the easiest > way to reduce the collective attack surface is to use a strictly less > powerful (but still sufficient) format. > > For ast.literal_eval, we'd still need to come up with a way to do > sections, key:value mappings and define rules for comments. > > For completeness, I'll note that XML combines even more user unfriendly > syntax than JSON with similar security risks to YAML. > > So with the trade-offs laid out like that (and particularly the > inconsistent comment and Unicode handling in ConfigParser), I'm prompted to > favour following Rust in adopting TOML. > +1 for TOML from me as well. I know Paul brought up the lack of familiarity, but the format is simple and the Rust community is already fully dependent on it so at worst Rust + us could always just ignore future format versions if necessary. If TOML is the chosen format we could ask how long until a 1.0 release to know if we waited a month or so to implement we could make sure we're compliant with that version. I also checked pytoml at https://github.com/avakar/pytoml and it looks like it's pretty stable; no changes in the past 5 months except to support Python 3.5 and only 3 issues. And the format is simple enough that if someone had to fork the code like Nathaniel suggested or we did it from scratch it wouldn't be a huge burden. -Brett > Cheers, > Nick. > > P.S. I particularly like the idea of using extension sections to > eventually consolidate other static config into a common file - that nicely > addresses my concern with config file proliferation, since it opens the > door to eventually subsuming other files like MANIFEST.in and setup.cfg as > archiving and build systems are updated > > > > > -n > > > > -- > > Nathaniel J. Smith -- https://vorpus.org > > ___ > > Distutils-SIG maillist - Distutils-SIG@python.org > > https://mail.python.org/mailman/listinfo/distutils-sig > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] who is BDFL for the boostrap/requires declaration? (was: moving things forward)
On Fri, 6 May 2016 at 16:58 Nathaniel Smith wrote: > On Fri, May 6, 2016 at 11:14 AM, Brett Cannon wrote: > > > > > > On Fri, 6 May 2016 at 09:40 Donald Stufft wrote: > >> > >> > >> On May 6, 2016, at 12:36 PM, Brett Cannon wrote: > >> > >> So who is the BDFL on this decision? It seems we need someone to stop > the > >> bikeshedding on the field name and what file is going to house this > >> configuration data. And do we need someone to write a PEP for this > proposal > >> to have something to target? > >> > >> > >> We need someone to write the PEP and someone to offer to be BDFL for it. > >> For this particular change the default would be Nick for BDFL but if > he’s > >> busy someone else can take it over for this PEP. Though I think we need > >> someone writing an actual PEP first. > > > > > > OK, assuming the Nick will be pronouncing, who wants to write the PEP? > And Paul also stepped forward to pronounce if Nick didn't want to, so we have the role of Great Decider covered one way or another. > > I've just been writing up a comparison of the different file formats, > partly in case it's useful to others and partly just for my own use in > looking at them against each other and seeing how much it actually > matters. This might also be reusable for the > rationale/rejected-alternatives section of a PEP, if anyone wants it, > or I could go ahead and add a few paragraphs to turn it into a proper > PEP. > What does the PEP need to cover? 1. The syntax of the file (which based on the replies to your great overview, Nathaniel, looks to be TOML). 2. The name of the file (although I'm assuming it will be setup.* out of tradition, probably setup.toml if TOML wins the format battle). 3. What fields there will be and their semantics ... 1. Format version (so just deciding on a name -- which also includes whether it should be top-level or in a subsection -- and initial value) 2. The actual bootstrap field (so its name and what to do if a dependency is already installed but at a version that doesn't match what the bootstrap specification asks for) Am I missing anything? And since I keep pushing on this I'm willing to be a co-author on any PEP if there's no hard deadline in getting the PEP written (i.e. I can help write the prose, but I don't have the time to do the coding as this will be the fourth PEP I have going in some state; got to catch up to Nick's 35 PEPs somehow ;). ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] who is BDFL for the boostrap/requires declaration? (was: moving things forward)
On Sat, May 7, 2016, 12:16 Chris Barker wrote: > On Sat, May 7, 2016 at 11:18 AM, Brett Cannon wrote: > >> What fields there will be and their semantics ... >> >>1. Format version (so just deciding on a name -- which also includes >> whether it should be top-level or in a subsection -- and initial value) >> 2. The actual bootstrap field (so its name and what to do if a >> dependency is already installed but at a version that doesn't match >> what >> the bootstrap specification asks for) >> >> Am I missing anything? >> > > So what is this new configuration file supposed to cover? > How to specify what has to be installed to simply build a project, e.g. is setuptools needed to run setup.py, and if so what version? All the package meta-data? i.e. everything that would be needed by a > package manager to properly install the package? > > or the build meta-data: everything needed by the build system to build the > package? > > both in one file? > > And am missing something? > You're missing that you're talking several PEPs down the road. :) Right now all we are discussing is how to specify what build tools a project needs (historically setuptools, but in the future it could be flit or something else). how is this about "bootstrapping" -- to me, bootstrapping is when you need > X to build X. Isn't this just regular old configuration: you need x,y to > build z? > Sure, if you don't like the term "bootstrap" then you can call it "build requirements". We have not been calling it " configuration" in a general sense as this doesn't cover how to invoke the build step (that will probably be the next PEP), just what needs to be installed to even potentially do a build. -Brett > -CHB > > > > > > > >> And since I keep pushing on this I'm willing to be a co-author on any PEP >> if there's no hard deadline in getting the PEP written (i.e. I can help >> write the prose, but I don't have the time to do the coding as this will be >> the fourth PEP I have going in some state; got to catch up to Nick's 35 >> PEPs somehow ;). >> >> ___ >> Distutils-SIG maillist - Distutils-SIG@python.org >> https://mail.python.org/mailman/listinfo/distutils-sig >> >> > > > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R(206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > chris.bar...@noaa.gov > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] comparison of configuration languages
On Sat, May 7, 2016, 15:47 Donald Stufft wrote: > > > On May 7, 2016, at 5:05 PM, Robert Collins > wrote: > > > > Either we are defining the long term thing now, in which case that > > huge pile of complexity lands on us, and we have to get everything > > right. > > > > Or we are defining a thing which solves the present bug, and as long > > as we make sure it does not bind us in future, we're not hamstrung. > > > > E.g. use setup.cfg now. Add pybuild.toml later. (btw, terrible name, > > as pybuild is a thing in the debian space, and this will confuse the > > heck out of folk). https://wiki.debian.org/Python/Pybuild > > I think this is roughly true, we could either do the simplest thing and > just > add ``setup_requires`` to ``setup.cfg`` and teach pip how to understand > them > and then worry about a new format later, or we can do a new format now and > add > a bit of complexity to what we need to specify (though I don't think _too_ > much > complexity, we don't have to define the build system stuff now, just make > sure > we don't back ourselves into a corner with that). > > I think either answer is OK, just the second one is a bit more work and we > might either get the start of a better format _now_ or end up regretting > what > we pick when we add more things to it. > For both options I hear "pick a new format", which suggests we might as well do it from the get-go for clear separation of the new stuff and just bite the bullet instead of simply postponing a decision; it isn't like our format options are going to significantly change between now and later in the year. -Brett > - > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 > DCFA > > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] comparison of configuration languages
Based on this email and Nathaniel's evaluation I've gone ahead and taken it upon myself to start writing a PEP so we have something concrete to work from. I'm hoping to have it done some time this week. On Sun, 8 May 2016 at 08:43 Donald Stufft wrote: > > > On May 8, 2016, at 9:13 AM, Nick Coghlan wrote: > > > > So let's reduce our scope to: "We want *current users* of d2to1 and > > pbr to be able to declare those dependencies to pip and other > > installation tools in a way that avoids the implicit invocation of > > easy_install by setuptools" > > I think it's less pbr and d2to1 that we're designing for here (although > they'll > benefit from it) but more like numpy.distutils. The reasoning is that pbr > and > such have already been able to architecture themselves so that they work > within > the current system. One of the goals here is to enable projects that > haven't > been able to (or can't reasonably) be written to work as a setuptools > extension > to be used here without requiring a manual pre-installation step. > > The other side of this is anytime you have dependencies that aren't > installed > by pip (such as setup_requires being installed by setuptools) you end up > with > a confusing situation where settings don't get passed down into the "inner" > installer (like ``--index-url``) or when they have different resolution > algorithms (like setuptools supporting pre-releases by default) or when > they > support different formats (like pip not supporting eggs, but setuptools not > supporting wheels). > > However, while I think that a new format is more work just to make sure we > don't back ourselves into a corner, I also don't think it's so much more > work > that it's really worth it to skip doing it now. Their isn't a whole lot of > ways > to solve the very narrow problem of "we need some dependencies installed > prior > to building". We need a field in a file that takes a list of PEP 508 style > specifiers. We can skip any of the build system support right now and add > it in > later (and when we add it in, we can just make it so that a lack of a > declaration is an implicit setuptool). So the only real things we need to > decide on are: > > 1) What format to use? For this I think that it's been pretty clearly that > TOML >has had the greatest amount of support here and I think it represents > the >best set of trade offs for us. > > 2) What do we want to call this file? This one is pure bikeshed but also a > bit >important since this is one of the things that will be the hardest to > change >once we actually pick the name. Ideas I've seen so far (using the toml >extension) are: > >* pypa.toml - This one is specific to us which is good, but it's also a > bit > bad in that I don't think it's very widely known what the PyPA even > is and > I think that the PyPA works better if it's just sort of the thing in > the > background. > >* pybuild.toml - This one might be a bit too oriented towards building > rather than all of the possible uses of this file, but the bigger > problem > with it I think is that hte name clashes with pybuild on Debian which > is > their tool used for building Python packages. > >* pip.toml - Also specific to us which is good, but I think it's a bit > too > overly specific maybe? One of our goals has been to make this stuff > not > dependent on a specific implementation so that we can replace it if we > need to (as we more or less replaced distutils with setuptools, or > easy_install with pip) however this also isn't exactly depending on an > implementation-- just the name of a particular implementation. It does > have the benefit that for a lot of people they associate "pip" with > everything packaging related (e.g. I see people calling them > "pip packages" now). > >* pymeta.toml - This one might be reasonable, it's a bit generic but at > least it has the ``py`` prefix to tie it to us. Not much to say about > it > otherwise. > >Overall from those names, I think I probably like pymeta.toml the best > (or >maybe just ``meta.toml`` if we don't like the py prefix) but maybe other >people have ideas/opinions that I haven't seen yet. > > 3) How do we structure the file and what keys do we use for just this part > of >the overall feature we're hoping to eventually get (and what semantics > do >we give these keys). We could just call it ``setup_requires``, but I > think >that's a bad name for something we're not planning on getting rid of > since >one of the eventual goals is to make ``setup.py`` itself optional. > Another >option is ``build_requires`` but that isn't quite right because we don't >*just* need these dependencies for the build step (``setup.py > bdist_wheel``) >but also for the metadata step (``setup.py egg_info``). Although > perhaps it >doesn't really matter. Another option is to call them > ``bootstrap_
Re: [Distutils] who is BDFL for the boostrap/requires declaration? (was: moving things forward)
On Mon, 9 May 2016 at 06:42 Nick Coghlan wrote: > On 7 May 2016 at 08:21, Paul Moore wrote: > > On 6 May 2016 at 19:14, Brett Cannon wrote: > >> OK, assuming the Nick will be pronouncing, who wants to write the PEP? > > > > ... and if Nick doesn't want to pronounce, I'm willing to offer to be > > BDFL for this one. But a PEP is the first thing. (And IMO the key > > point of the PEP is to be very clear on what is in scope and what > > isn't - the discussions have covered a *lot* of ground and being clear > > on what's excluded will be at least as important as stating what's in > > scope). > > Answering this specifically: I'm happy to be the arbiter-of-consensus > for this one, as my schedule's pretty clear right now (at least until > I head to PyCon US on the 27th). > OK, I'll list Nick as the BDFL Delegate in the PEP then. Hoping to have a draft to send to the list today or tomorrow. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] comparison of configuration languages
On Mon, 9 May 2016 at 06:29 Nick Coghlan wrote: > On 9 May 2016 at 01:43, Donald Stufft wrote: > > Overall, my suggestion here would be to have a file called > ``pymeta.toml`` (or > > ``meta.toml``) > > pymeta.toml would be fine by me. > > I don't really buy the "collision with Debian build tool" argument > against "pybuild" (if I did, I'd be objecting to "pymeta" colliding > with an existing PyPI package), so it's mainly the fact the metadata > in this file covers more than just building has soured me on it. > > > and have it look like:: > > > > [dependencies] > > build = [ > > "setuptools", > > "wheel>=1.0", > > ] > > > > If at some point we decide we need to add a bootstrap_requires (and then > the > > ability to add dynamic build requirements) we can do that by just saying > that > > if you plan on having dynamic build requirements, you need to omit the > build > > key under the [dependencies] section. > > Looking at my previous ideas for semantic dependencies in PEP 426, > what if we start in the near term by defining development > requirements? > > That can then be used to hold arbitrary development dependencies > (metadata generation, sdist creation, test execution, wheel build, > docs build, etc), everything that you need to work on, build, and test > the software, but don't need if you just want to run the published > releases. > > We may later decide that we want to break that down and separate out > specific requirements for sdist creation and for wheel creation, but > we can handle that by saying that if there's no more specific > dependency definition for an operation, then the tools will fall back > to pre-installing all the listed development dependencies. > > That is, someone might write: > > [dependencies] > develop = [ > "setuptools", > "wheel>=1.0", > "sphinx", > "pytest", >] > > And it would be useful not only for running setup.py commands, but > setting up their local dev environment in general. > I'm not going to touch the concept of development dependencies to keep the PEP focused. But one thing this discussion does bring up is everyone is assuming there is simply going to be a [dependencies] section to the configuration file where build, development, and/or installation dependencies will eventually end up. My current plan for the PEP is to not do that. Instead, I'm going to have a [build] section that will have a `dependencies` field. My thinking is that we don't need to have any end-users who choose to look at this file care about anything related to building or developing a project. Plus if we expose any details about the Future Awesome Build API (aka the FAB API ;) then it should probably be kept near all of the other build-related details, including the dependencies for building instead of inherently spanning multiple sections (yes, tools will quite possibly have their own section, but we can try to keep it to [build] and e.g. [tools.setuptools] and not [dependencies], [build], and [tools.setuptools]). ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] comparison of configuration languages
On Mon, 9 May 2016 at 11:21 Chris Barker wrote: > "pymeta" feels very "inessentially weird" to me [1]. > > > yeah... > > setup.toml > > ??? > You can all stop guessing at file names. The PEP will have a recommendation and you all can either agree or disagree at that point. Please don't give me more names to list in the rejected section. :) -Brett > > -CHB > > > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R(206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > chris.bar...@noaa.gov > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] comparison of configuration languages
Just so everyone knows, I'm ignoring this thread as the PEP I'm drafting with Donald and Nathaniel is nearly finished and thus has already settled on the file format discussion and I haven't heard a new point made on any file format proposal that has already been brought up previously. On Tue, 10 May 2016 at 10:10 Paul Moore wrote: > On 10 May 2016 at 13:40, Wolfgang wrote: > > So why not use the ConfigParser available with Python and extend it to > meet > > the requirements. Custom getters can be written and for the complex > > stuff ast.literal_eval() can be used to safely parse the complex list > > of requirements with comments. > > Sadly, pip needs to work on Python 2.6+, so we can't use these new > features of configparser. > > Paul > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] PEP for specifying build dependencies
Donald, Nathaniel, and I have finished our proposed PEP for specifying a projects' build dependencies. The PEP is being kept at https://github.com/brettcannon/build-deps-pep, so if you find spelling mistakes and grammatical errors please feel free to send a PR to fix them. The only open issue in the PEP at the moment is the bikeshedding topic of what to name the sub-section containing the requirements: `[package.build]` or `[package.build-system]` (we couldn't reach consensus among the three of us on this). Otherwise the three of us are rather happy with the way the PEP has turned out and look forward to this being the first step towards allowing projects to customize their build process better! - PEP: NNN Title: Specifying build dependencies for Python Software Packages Version: $Revision$ Last-Modified: $Date$ Author: Brett Cannon , Nathaniel Smith , Donald Stufft BDFL-Delegate: Nick Coghlan Discussions-To: distutils-sig Status: Draft Type: Standards Track Content-Type: text/x-rst Created: NN-Mmm-2016 Post-History: NN-Mmm-2016 Abstract This PEP specifies how Python software packages should specify their build dependencies (i.e. what dependencies are required to go from source checkout to built wheel). As part of this specification, a new configuration file is introduced for software packages to use to specify their build dependencies (with the expectation that the same configuration file will be used for future configuration details). Rationale = When Python first developed its tooling for building distributions of software for projects, distutils [#distutils]_ was the chosen solution. As time went on, setuptools [#setuptools]_ gained popularity to add some features on top of distutils. Both used the concept of a ``setup.py`` file that project maintainers executed to build distributions of their software (as well as users to install said distribution). Using an executable file to specify build requirements under distutils isn't an issue as distutils is part of Python's standard library. Having the build tool as part of Python means that a ``setup.py`` has no external dependency that a project maintainer needs to worry about to build a distribution of their project. There was no need to specify any dependency information as the only dependency is Python. But when a project chooses to use setuptools, the use of an executable file like ``setup.py`` becomes an issue. You can't execute a ``setup.py`` file without knowing its dependencies, but currently there is no standard way to know what those dependencies are in an automated fashion without executing the ``setup.py`` file where that information is stored. It's a catch-22 of a file not being runnable without knowing its own contents which can't be known programmatically unless you run the file. Setuptools tried to solve this with a ``setup_requires`` argument to its ``setup()`` function [#setup_args]_. This solution has a number of issues, such as: * No tooling (besides setuptools itself) can access this information without executing the ``setup.py``, but ``setup.py`` can't be executed without having these items installed. * While setuptools itself will install anything listed in this, they won't be installed until *during* the execution of the ``setup()`` function, which means that the only way to actually use anything added here is through increasingly complex machinations that delay the import and usage of these modules until later on in the execution of the ``setup()`` function. * This cannot include ``setuptools`` itself nor can it include a replacement to ``setuptools``, which means that projects such as ``numpy.distutils`` are largely incapable of utilizing it and projects cannot take advantage of newer setuptools features until their users naturally upgrade the version of setuptools to a newer one. * The items listed in ``setup_requires`` get implicily installed whenever you execute the ``setup.py`` but one of the common ways that the ``setup.py`` is executed is via another tool, such as ``pip``, who is already managing dependencies. This means that a command like ``pip install spam`` might end up having both pip and setuptools downloading and installing packages and end users needing to configure *both* tools (and for ``setuptools`` without being in control of the invocation) to change settings like which repository it installs from. It also means that users need to be aware of the discovery rules for both tools, as one may support different package formats or determine the latest version differently. This has cumulated in a situation where use of ``setup_requires`` is rare, where projects tend to either simply copy and paste snippets between ``setup.py`` files or they eschew it all together in favor of simply documenting elsewhere what they expect the user to have manually installed prior to attempting to
Re: [Distutils] PEP for specifying build dependencies
Since you can easily read the PEP at
https://github.com/brettcannon/build-deps-pep in a nice, rendered format I
won't repost the whole thing, but here are the changes I have made based on
the comments so far.
1. I rephrased throughout the PEP to not explicitly say this is for
building wheels (for Robert). I was just trying to tie the PEP into what
happens today, not what we plan to have happen in the future.
2. Fixed the quotation marks on the TOML examples (for Nathaniel). It's
just habit from Python why I did it the way I did.
3. Reworked the Abstract to the following (for Robert):
"""
This PEP specifies how Python software packages should specify what
dependencies they have in order to execute their chosen build system.
As part of this specification, a new configuration file is introduced
for software packages to use to specify their build dependencies (with
the expectation that the same configuration file will be used for
future configuration details).
"""
4. Added a bit to the end of the Rationale section about where this fits in
future plans (for Robert):
"""
To provide more context and motivation for this PEP, think of the
(rough) steps required to produce a built artifact for a project:
1. The source checkout of the project.
2. Installation of the build system.
3. Execute the build system.
This PEP covers step #2. It is fully expected that a future PEP will
cover step #3, including how to have the build system dynamically
specify more dependencies that the build system requires to perform
its job. The purpose of this PEP though, is to specify the minimal set
of requirements for the build system to simply begin execution.
"""
5. Added a JSON Schema for the resulting data (for Nick because he must be
writing a lot of specs at work recently if he needs typing information for
2 fields :). This is based on Nathaniel's initial work, but I made the
"requires" key a requirement when [package.build-system] is defined. I did
not change the name of the key to "schema-version" for the reasons
Nathaniel outlined in his response to Nick on the topic.
{
"$schema": "http://json-schema.org/schema#";,
"type": "object",
"additionalProperties": false,
"properties": {
"package": {
"type": "object",
"additionalProperties": false,
"properties": {
"semantics-version": {
"type": "integer",
"default": 1
},
"build-system": {
"type": "object",
"additionalProperties": false,
"properties": {
"requires": {
"type": "array",
"items": {
"type": "string"
}
}
},
"required": ["requires"]
}
}
},
"tool": {
"type": "object"
}
}
}
I didn't add an example using 'distutils.numpy' as Nick asked because it's
just the same example as in the PEP but with one field changed; IOW it
doesn't really add anything. I also didn't rename the file as Randy argued
for because the file is for the project, not just the package(s) the
project contains ("package" is an overloaded term and I don't want to
contribute to that with the filename; I can live with the build details
being in relation to a package in the project and thus named [package], but
other things that may end up in this file might not relate to any package
in the project).
On Tue, 10 May 2016 at 17:39 Brett Cannon wrote:
> Donald, Nathaniel, and I have finished our proposed PEP for specifying a
> projects' build dependencies. The PEP is being kept at
> https://github.com/brettcannon/build-deps-pep, so if you find spelling
> mistakes and grammatical errors please feel free to send a PR to fix them.
>
> The only open issue in the PEP at the moment is the bikeshedding topic of
> what to name the sub-section containing the requirements: `[package.build]`
> or `[package.build-system]` (we couldn't reach consensus among the three of
> us on this). Otherwise the three of us are rather happy with the way the
> PEP has turned out and look forward to this being the first step towards
> allowing projects to customize their build process better!
>
So far the votes for this open issue are:
build-system:
1. Nathaniel
2. Ian
3. Ethan
4. Nick
5. Paul
build:
1. Donald
2. Daniel
build-configuration (write-in candidate):
1. Antoine
___
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP for specifying build dependencies
On Wed, 11 May 2016 at 12:37 Robert Collins wrote: > On 12 May 2016 at 06:32, Brett Cannon wrote: > > Since you can easily read the PEP at > > https://github.com/brettcannon/build-deps-pep in a nice, rendered > format I > > won't repost the whole thing, but here are the changes I have made based > on > > the comments so far. > > > > 1. I rephrased throughout the PEP to not explicitly say this is for > building > > wheels (for Robert). I was just trying to tie the PEP into what happens > > today, not what we plan to have happen in the future. > > > > 2. Fixed the quotation marks on the TOML examples (for Nathaniel). It's > just > > habit from Python why I did it the way I did. > > > > 3. Reworked the Abstract to the following (for Robert): > > Thanks Brett. > Welcome! Just like all of you, I want to get this PEP right as it's the foundational one to start us down the road of *finally* getting building untangled in a way we're all happy and excited about! -Brett > > -Rob > > > -- > Robert Collins > Distinguished Technologist > HP Converged Cloud > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP for specifying build dependencies
On Wed, 11 May 2016 at 16:01 Nathaniel Smith wrote:
> On Wed, May 11, 2016 at 11:32 AM, Brett Cannon wrote:
> [...]
> > the file is for the project, not just the package(s) the project
> > contains ("package" is an overloaded term and I don't want to contribute
> to
> > that with the filename; I can live with the build details being in
> relation
> > to a package in the project and thus named [package], but other things
> that
> > may end up in this file might not relate to any package in the project).
>
> We went back and forth on the overloaded "package" name a bit while
> drafting too, and eventually just gave up and went ahead because it's
> not that important.
>
> To me this feels similar to situations I've encountered in the past,
> where I've spent a bunch of time debating between two things, and it
> turned out that the reason we couldn't agree was because both
> proposals were wrong and a third solution was much better :-).
>
> I still don't think the [package] name part is worth arguing about
> much, but throwing this out there in case it turns out to be that
> "third way" that suddenly makes everyone go "a-ha!":
>
> If you think about it, really the stuff in [package.build-system] is
> there to tell pip how to run the build system. It's associated with
> building the project/package, sure, but that's not what makes it
> special -- everything that goes in this file will be associated with
> building or developing the project/package: [tool.flit],
> [tool.coverage], [tool.pytest], [tool.tox], whatever. The build-system
> stuff could easily and comfortably have gone into
> [tool.pip.build-system] instead... *except* that we don't want it to
> be specific to pip, we want it to be a piece of shared/common
> configuration that's defined by a shared process (PEPs) and used by
> *multiple* tools. That's why it doesn't belong in [tool.pip].
>
> Or another way to put it, contrasting [tool.*] versus [package.*] is
> kinda weird, because those categories aren't actually contradictory --
> it's like having categories [red] versus [square].
>
> So, proposal: maybe we should rename the [package] namespace to
> something that reflects what distinguishes it from [tool], like:
>
> [standard.build-system]
>
> or
>
> [common.build-system]
>
> or
>
> [shared.build-system]
>
>
or
[base.build-system]
or
[super.build-system]
I'm +1 on base, super, or common, +0 on standard, -0 on shared.
___
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP for specifying build dependencies
On Thu, 12 May 2016 at 05:43 Donald Stufft wrote: > > > On May 12, 2016, at 8:31 AM, Nick Coghlan wrote: > > > > We could also keep semantics-version, and just put it inside > [build-system]. > > > > Either way, by allowing access to the [tool.*] namespace without any > > other version check, the key constraint we're placing on ourselves is > > a commitment to only making backwards compatible changes at the top > > level of the schema definition, and that should be a feasible promise > > to keep. While I can't conceive of an eventuality where we'd need to > > break such a promise, even if we did, the change could be indicated by > > switching to a different filename. > > I don't think we should put it inside of [build-system], largely because I > think the chances we ever need to increment the version is very small, and > I > feel like putting it inside of [build-system] means we'll then need one for > any other top level key we put in. Each additional one is additional > complexity > and increases the chance that some tool doesn't accurately check every > single > one of them. Putting it inside of [package] made some sense, because that > was > going to be a container for all of the things that one particular group of > people (distutils-sig / PyPA) "managed" or cared about but I think that > putting > it on each individual sub section is just overkill. > Everything that Nathaniel and Donald said is accurate about the discussion we had offline while drafting the PEP. Top-of-file was originally proposed by me but was viewed as too broad, hence the namespacing of the bits PyPA controlled and putting the field in there. We also considered per-table, but that seemed like overkill. > > We can easily stick it at the top level of the file and just explicitly > state > that the [tool.*] namespace is exempt from deriving any sort of meaning > from > the semantics-version value. I think that is easier to not screw up (only > one > check, vs N checks) and I think that it looks nicer too from a human > writing/editing POV if we ever do bump the version and force people to > write it > out: > I had originally proposed that but I think we didn't like the wording of it or the possibility of someone not realizing that the scoping of the field was special-cased to everything *not* in [tool]. > > semantics-version = 2 > > [build-system] > requires = [ > "setuptools", > "pip", > ] > > [test-runner] # Just an example > command = "py.test --strict" > requires = [ > "pytest", > ] > > [tool.pip] > index-url = "https://index.example.com/simple/"; > > But honestly, I'm of the opinion we could probably just ditch it. I don't > think > it'll be hard to maintain compatibility within the keywords we pick in this > file and I worry that by including it in something that we expect humans to > write, we provide an incentive to using it when perhaps we could think up a > better, backwards compatible syntax. The main argument in favor of adding > it > now with an implicit default of `1` is that if I'm wrong and we end up > needing > it, including it now will mean that projects are actively checking the > version > number so we can safely increase it with the desired effect. If we don't > include it now, then even if we add it at a later date nothing will be > checking > to see if that changed or not. > Both Donald and Nathaniel say to drop it, and since I put it in just to be overly cautious I'm fine with dropping it. So unless Nick says "semantics-version or death!", I agree w/ my co-authors and would update the PEP to say: 1. no semantics-version 2. [build-system] instead of [package.build-system] ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP for specifying build dependencies
On Thu, 12 May 2016 at 20:52 Nick Coghlan wrote: > On 13 May 2016 at 02:33, Brett Cannon wrote: > > Both Donald and Nathaniel say to drop it, and since I put it in just to > be > > overly cautious I'm fine with dropping it. So unless Nick says > > "semantics-version or death!", I agree w/ my co-authors and would update > the > > PEP to say: > > > > 1. no semantics-version > > 2. [build-system] instead of [package.build-system] > > > > I think both of those changes are improvements - having to change the > filename is a reasonable disincentive against making breaking changes, > and with just the two sections initially being defined ([build-system] > and [tool.*]) it makes sense to have them both at the top level. > Seems everyone is in agreement then. I'll update the PEP and send out a new draft later today. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] comparison of configuration languages
On Fri, 13 May 2016 at 10:47 Paul Moore wrote: > On 13 May 2016 at 18:15, Chris Barker wrote: > > I think we're freaking out way too much about what *could* go wrong. > > It's more that:pip would have to vendor pyyaml, and it's not small. I > have no idea whether it's easy to vendor, either (does it have > separate code paths for Python 2 and 3? I don't know, I've never > looked). Also, we'd have to forego the C implementation - I have no > idea how much of a performance hit that would give (of course, > performance is hardly a key issue here) or how likely it is that bugs > exist in the Python version that aren't normally noticed because > people use the C implementation (which is definitely just speculation, > conceded). > > But I think the decision is made, so it's not worth debating any more, > honestly. > No need to think; the decision is made and it's TOML. I know Chris doesn't mean to stir up trouble, but at this point if someone wants to propose something other than TOML they are going to have to write their own PEP. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP for specifying build dependencies
On Fri, 13 May 2016 at 11:34 Chris Barker wrote: > One other question: > > Is it just examples or is "build" being defined as "build a wheel"? > Just an example (clarified previously). -Brett > > i.e. there are times one might want to build a package without building a > wheel -- just it install it yourself, or to put it in another package > format -- conda, rpm, what have you. > > In conda's case, building a wheel, and then installing it would work fine, > but I'm not sure we want to lock that down as the only way to build a > package. > > Granted, if all it means is that someone will download an unnecessary > dependency, big deal. > > I'm also a bit confused about whether we're trying to specify the > dependencies required simply to run the build tool itself, or the > dependencies required to actually do the build -- or the latter being saved > for another day? > The minimal requirements to execute the build system. Providing some way to specify more dependencies in some dynamic fashion and the like is for another PEP. -Brett > > -CHB > > > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R(206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > chris.bar...@noaa.gov > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] build system requirements PEP, 3rd draft
Biggest changes since the initial draft: 1. No more semantics-version 2. No more [package] table 3. Settled on [build-system] as the table name 4. The "requires" key is required if [build-system] is defined 5. Changed the title and clarified that this is all about the minimum requirements for the build system to execute (some added support for things like dynamic dependencies for producing build artifacts is for another PEP) 6. Added a JSON Schema for the resulting data from the table because Nick likes his specs :) -- PEP: NNN Title: Specifying Minimum Build System Requirements for Python Projects Version: $Revision$ Last-Modified: $Date$ Author: Brett Cannon , Nathaniel Smith , Donald Stufft BDFL-Delegate: Nick Coghlan Discussions-To: distutils-sig Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 10-May-2016 Post-History: 10-May-2016, 11-May-2016, 13-May-2016 Abstract This PEP specifies how Python software packages should specify what dependencies they have in order to execute their chosen build system. As part of this specification, a new configuration file is introduced for software packages to use to specify their build dependencies (with the expectation that the same configuration file will be used for future configuration details). Rationale = When Python first developed its tooling for building distributions of software for projects, distutils [#distutils]_ was the chosen solution. As time went on, setuptools [#setuptools]_ gained popularity to add some features on top of distutils. Both used the concept of a ``setup.py`` file that project maintainers executed to build distributions of their software (as well as users to install said distribution). Using an executable file to specify build requirements under distutils isn't an issue as distutils is part of Python's standard library. Having the build tool as part of Python means that a ``setup.py`` has no external dependency that a project maintainer needs to worry about to build a distribution of their project. There was no need to specify any dependency information as the only dependency is Python. But when a project chooses to use setuptools, the use of an executable file like ``setup.py`` becomes an issue. You can't execute a ``setup.py`` file without knowing its dependencies, but currently there is no standard way to know what those dependencies are in an automated fashion without executing the ``setup.py`` file where that information is stored. It's a catch-22 of a file not being runnable without knowing its own contents which can't be known programmatically unless you run the file. Setuptools tried to solve this with a ``setup_requires`` argument to its ``setup()`` function [#setup_args]_. This solution has a number of issues, such as: * No tooling (besides setuptools itself) can access this information without executing the ``setup.py``, but ``setup.py`` can't be executed without having these items installed. * While setuptools itself will install anything listed in this, they won't be installed until *during* the execution of the ``setup()`` function, which means that the only way to actually use anything added here is through increasingly complex machinations that delay the import and usage of these modules until later on in the execution of the ``setup()`` function. * This cannot include ``setuptools`` itself nor can it include a replacement to ``setuptools``, which means that projects such as ``numpy.distutils`` are largely incapable of utilizing it and projects cannot take advantage of newer setuptools features until their users naturally upgrade the version of setuptools to a newer one. * The items listed in ``setup_requires`` get implicily installed whenever you execute the ``setup.py`` but one of the common ways that the ``setup.py`` is executed is via another tool, such as ``pip``, who is already managing dependencies. This means that a command like ``pip install spam`` might end up having both pip and setuptools downloading and installing packages and end users needing to configure *both* tools (and for ``setuptools`` without being in control of the invocation) to change settings like which repository it installs from. It also means that users need to be aware of the discovery rules for both tools, as one may support different package formats or determine the latest version differently. This has cumulated in a situation where use of ``setup_requires`` is rare, where projects tend to either simply copy and paste snippets between ``setup.py`` files or they eschew it all together in favor of simply documenting elsewhere what they expect the user to have manually installed prior to attempting to build or install their project. All of this has led pip [#pip]_ to simply assume that setuptools is necessary when ex
Re: [Distutils] build system requirements PEP, 3rd draft
And the PEP is checked in! https://hg.python.org/peps/file/tip/pep-0518.txt I'm at OSCON and in a tutorial on my work laptop so I didn't have a chance to verify there weren't any reST errors, so if someone with commit privileges can just quickly run `make` on the peps repo to make sure I didn't botch something that would be appreciated. :) On Mon, 16 May 2016 at 01:02 Paul Moore wrote: > On 16 May 2016 at 08:02, Nick Coghlan wrote: > > More seriously, as BDFL-Delegate, I'm entirely happy with this > > version. Thanks for your work in pulling this reduced scope PEP > > together, as well as to Robert, Nathaniel and Donald in driving the > > preceding build system invocation PEPs. > > Excellent news! Thanks to all for moving this forward. > > Paul > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] build system requirements PEP, 3rd draft
Cool, thanks! On Mon, 16 May 2016 at 07:49 Donald Stufft wrote: > > On May 16, 2016, at 10:43 AM, Brett Cannon wrote: > > I'm at OSCON and in a tutorial on my work laptop so I didn't have a chance > to verify there weren't any reST errors, so if someone with commit > privileges can just quickly run `make` on the peps repo to make sure I > didn't botch something that would be appreciated. :) > > > > Seems to work: > > $ make pep-0518.html > pep-0518.txt (text/x-rst) -> pep-0518.html > > - > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 > DCFA > > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Switch PyPA from IRC to Gitter or similar
On Fri, 10 Jun 2016 at 07:25 Ian Cordasco wrote: > On Fri, Jun 10, 2016 at 8:22 AM, Jason R. Coombs > wrote: > > In #pypa-dev, I raised the possibility of moving our PyPA support > channels from IRC to another hosted solution that enables persistence. > Although IRC has served us well, there are systems now with clear feature > advantages, which are crucial to my continuous participation: > > I'm choosing not to read this as a threat. > I don't think it was a threat to begin with. For it to be a threat it would somehow need to affect you personally. I think all Jason was trying to do was to point out this is not some idle conversation to him, but in fact impacts how he participates in communications regarding setuptools and PyPA. > > > - always-on experience; even if one’s device is suspended or otherwise > offline. > > - mobile support — the in-cloud experience is essential for low power > and intermittently connected devices. > > - push notifications allow a project leader to remain largely inactive > in a channel, but attention raised promptly when users make a relevant > mention. > > - continuous, integrated logging for catching up on the conversation. > > So here's a question: Why are these crucial to you? You've explained > potential benefits but not why they're crucial to you and should be > crucial to anyone else. > > Why do you need an "always-on experience"? Why do you feel required to > always be on? Do other people tell you that you need to always be on > chat? > > Push notifications allow for prompt attention to mentions, but are all > mentions push notification worthy? Do we all need to be herded to > platforms that will spam us because someone mentioned us by nick or > name? I personally see this as a net negative. I don't need an email > or push notification to my phone because someone said my name in a > channel. That's a distraction. It prevents me from working on things > because it creates a false sense of alarm. > I think there's a difference between getting a push notification that rings your phone and one that simply sits in your notification bar. For the former that could get annoying if abused, but for the latter it could be handy if e.g. you are working on a bug and you happen to know that Jason could help speed things up for you by answering a question if he happens to be available. To me there's three levels of engagement: (1) I'm willing to be interrupted, (2) if I'm available I'll notice, else I will ignore it, and (3) just collect all the messages and I will check the next time I explicitly log in. You solve (1) by simply being logged into IRC all the time, but I don't know how to get (2) or (3) to work in IRC w/o setting up something like a reflector or something fancy (#3 can be covered by email, and maybe #2 with the right setup). > > Continuous logging is on for #pypa and #pypa-dev as I understand it. > Surely it's not "integrated" into your chat client, but it's not as if > the logging doesn't exist. > For me the constant connection allows for collecting mentions into a single notification area (#3 in the engagement level list I mentioned above). I personally have 4 devices I connect to the internet with and none of them short of my phone is on constantly. With some kind of constant connection I could then have all of the mentions I have not seen collected into a single place so I could address them no matter what device I choose to check in with. Otherwise I have to restrict myself to only using a device which has an IRC client that can reconcile where I left off and pick up on all the messages that mentioned me since I last logged in. And as for mobile access, that's just a matter of occasional convenience. I don't think that messaging all the time from my mobile phone is the best option, but it is one that I can do while e.g. sitting on the bus on the way to work so that I can spend my time at home doing other things potentially. > > > Both Gitter and Slack offer the experience I’m after, with Gitter > feeling like a better fit for open-source projects (or groups of them). > > I've tried using Gitter several times in the past. Unless they've > fixed their bugs related to sending me emails every day about activity > in a channel I spoke in once and left, I think they should be > eliminated. > > Slack has also had several outages lately that should also disqualify > it (besides the fact that it's incredibly closed source and will be > expensive to maintain logs in). > > > I’ve tried using IRCCloud, and it provides a similar, suitable > experience on the same IRC infrastructure, with one big difference. While > Gitter and Slack offer the above features for free, IRCCloud requires a > $5/user/month subscription (otherwise, connections are dropped after two > hours). I did reach out to them to see if they could offer some > professional consideration for contributors, but I haven’t heard from them. > Furthermore, IRCCloud requires an additional account on top
Re: [Distutils] Outdated packages on pypi
On Tue, 12 Jul 2016 at 21:54 Donald Stufft wrote: > > > On Jul 12, 2016, at 4:45 PM, Glyph Lefkowitz > wrote: > > > > My feeling is that there should be a "dead man's switch" sort of > mechanism for this. Require manual intervention from at least one package > owner at least once a year. I believe if you dig around in the archives > there's been quite a bit of discussion around messaging to package owners > and that sort of thing - and the main sticking point is that someone needs > to volunteer to do the work on Warehouse. Are you that person? :) > > [SNIP] > > Another thing we need to be careful about is what do we do once said dead > man’s switch triggers? We can’t just release the package to allow anyone to > register it, that’s just pointing a security shaped footgun at the foot of > every person using that project? It doesn’t make sense to block new uploads > for that project since there’s no point to disallowing new uploads. > Flagging it to allow someone to “take over” (possibly with some sort of > review) has some of the security shaped footguns as well as a problem with > deciding who to trust with a name or not. My assumption was that if a project was flagged as no longer maintained, then it would literally just get some clear banner/label/whatever to let people know that if they start using the project that they shouldn't necessarily expect bug-fixes. And if people wanted to get really fancy, expose this metadata such that some tool could easily warn you that you have dependencies that have been flagged as unsupported code. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Proposal: "Install and save"
On Sat, 23 Jul 2016 at 10:36 Daniel Holth wrote: > Not yet. Someone should fix that 😎 > There is an issue tracking that if someone gets adventurous enough to write up a PR: https://github.com/pypa/pip/issues/3691 . -Brett > > On Sat, Jul 23, 2016, 11:37 Alex Grönholm > wrote: > >> pip doesn't yet support pyproject.toml does it? >> >> >> 23.07.2016, 17:43, Daniel Holth kirjoitti: >> >> Here is my attempt. The SConstruct (like a Makefile) builds the >> extension. The .toml file gives the static metadata. No need to put the two >> in the same file. >> >> https://bitbucket.org/dholth/cryptacular/src/tip/SConstruct >> >> https://bitbucket.org/dholth/cryptacular/src/tip/pyproject.toml >> >> On Sat, Jul 23, 2016 at 10:11 AM Alex Grönholm >> wrote: >> >>> 23.07.2016, 17:04, Thomas Kluyver kirjoitti: >>> > On Sat, Jul 23, 2016, at 02:32 PM, Alex Grönholm wrote: >>> >> I'm -1 on this because requirements.txt is not really the standard way >>> >> to list dependencies. >>> >> In the Python world, setup.py is the equivalent of Node's >>> package.json. >>> >> But as it is >>> >> Python code, it cannot so easily be programmatically modified. >>> > Packaging based on declarative metadata: >>> > http://flit.readthedocs.io/en/latest/ >>> > >>> > >>> > We have a bit of a divide. Specifying dependencies in setup.py (or >>> > flit.ini, or upcoming pyproject.toml) is the standard for library and >>> > tool packages that are intended to be published on PyPI and installed >>> > with pip. requirements.txt is generally used for applications which >>> will >>> > be distributed or deployed by other means. >>> > >>> > As I understand it, in the Javascript world package.json is used in >>> both >>> > cases. Is that something Python should try to emulate? Is it hard to >>> > achieve given the limitations of setup.py that you pointed out? >>> This topic has been beaten to death. There is no way to cram the >>> complexities of C extension compilation setup into purely declarative >>> metadata. Distutils2 tried and failed. Just look at the setup.py files >>> of some popular projects and imagine all that logic expressed in >>> declarative metadata. >>> > Thomas >>> > ___ >>> > Distutils-SIG maillist - Distutils-SIG@python.org >>> > https://mail.python.org/mailman/listinfo/distutils-sig >>> >>> ___ >>> Distutils-SIG maillist - Distutils-SIG@python.org >>> https://mail.python.org/mailman/listinfo/distutils-sig >>> >> >> ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Deprecating little used file types/extensions on PyPI?
+1 on the whole idea. Trying to continue to nudge the community towards more standardized approaches in packaging is always a good thing. :) I only have one data point in relation to sdists file formats. On Mon, 15 Aug 2016 at 12:09 Donald Stufft wrote: > [SNIP] > > Looking at numbers again, the current number of projects for each file ext > are: > > * .tar.gz: 444,338 > * .zip: 58,774 > * .tar.bz2: 3,265 > * .tgz: 217 > * Everything Else: 0 > One thing to remember is that Windows can't read tar files natively while it can for zip files. Now you can easily download tools on Windows to read tar files and thanks to Bash on Windows you even have it included once you turn that feature on. The other point is we have a zip importer in Python but not a .tar.gz one. I don't know how often anyone actually downloads a zip file directly from PyPI and then tack it on to their sys.path for importing, but that is currently possible. I doubt either of these points are important enough to continue to support zip files for sdists, but I just wanted to point it out. At worst this is something to think about if we ever do formalize the sdist format and come up with a custom file extension. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Deprecating little used file types/extensions on PyPI?
On Fri, 19 Aug 2016 at 12:53 Donald Stufft wrote: > > On Aug 19, 2016, at 2:00 PM, Leonardo Rochael Almeida < > leoroch...@gmail.com> wrote: > > ure, more people will be affected this way than just the folks releasing > on Windows, but given the shortcuts for setting the sdist format per > project or per home directory that Donald mentioned, I think the collective > effort in the migration will be smaller than the continuous effort of > explaining to newcomers that the reason we use a .tar.gz based format for > sdists versus a .zip based format for wheels is some historical accident, > specially if we plan to change sdists back to .zip format in the future... > > > > I don’t think I’ve ever had anyone ask me why people (generally) use > .tar.gz for sdists and why wheels use zip, though I don’t do much beginner > stuff. Do you have some sort of evidence or data to suggest that this is a > problem that people are experiencing or are you theorizing that folks > *might* get confused by this? > > I think the effort changing non-Windows platform is going to be a lot more > effort than changing Windows platform for a few reasons: > > * There are less people releasing on Windows than on non-Windows, the more > people you need to migrate to a new thing, the longer you can expect it to > take. > > * Windows does not come with Python, thus people are generally free to > upgrade their Python or setuptools installation at all [1] meanwhile Python > and setuptools tends to be a core part of non-Windows OSs where upgrading > one or the other can “void the warranty” and cause breakage to the entire > OS, combine this with things like CentOS, RHEL, Ubuntu LTS and we lengthen > the time that people are using these older tools by years, maybe even a > decade. > > * More people are using .tar.gz than .zip, which means changing from > .tar.gz is more likely to cause issues (under the assumption that any > change can cause issue, and the more people who have some sort of change > occur to them, the more likely an issue is to occur). > > Oh, and TIL that anyone who has Python 3.4+ installed has a command line > tool for extracting ``.tar.gz`` files [2] > So I think you're both right, but at different time scales. :) I think Donald is right that the short-term time scale of "now" by suggesting we just go with tar.gz since it has the numbers. But I think Leonardo's point of general alignment with zip for packaging overall is good for the "formally define sdist" time scale and we potentially introduce an .sdist file extension. IOW I think we're all starting to talk in circles and since no one has screamed "the sky is falling" against the proposal, it's probably time to start formalizing a plan. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Deprecating little used file types/extensions on PyPI?
On Tue, 23 Aug 2016 at 07:33 Donald Stufft wrote: > > > On Aug 23, 2016, at 7:25 AM, Nick Coghlan wrote: > > > > OK, cool - that gives us all the more reason to retain bdist_wininst > > and bdist_msi hosting support. However, I do think it makes sense for > > us to say up front that we'll reconsider that decision if something > > akin to homebrew gains traction amongst developers running Windows the > > way homebrew has amongst open source users running Mac OS X. > > > I still don’t think there’s a whole lot of benefit to retaining them even > now. In the last 30 days, 90% of the downloads of bdist_wininst were > generated by things that I know for a fact to be mirroring clients (almost > all entirely bandersnatch). The next highest source of downloads was coming > from setuptools, at 7%. Over 75% of the downloads from setuptools are for > coverage.py, which tells me that it’s likely being triggered by > test_requires > and would be covered by teaching setuptools how to wheel instead. > > For bdist_msi, 96% of all downloads come from things we know to be > mirroring > clients. > > For bdist_dmg, 97% of all downloads come from things we know to be > mirroring > clients. > > For bdist_egg, 80% of all downloads come from things we know to be > mirroring > clients. > > For reference: > > For sdist, 30% of all downloads come from things we know to be mirroring > clients. > > For bdist_wheel, 6% of all downloads come from things we know to be > mirroring > clients. > > It’s hard to get per project numbers for these (or at least, it takes a > more > complex query than I can manage with my head here). However, I think it’s > pretty telling that when you start looking at other formats, not only is > the > primary consumer tools that just indiscriminately download everything from > PyPI, > but almost *all* of the consumers of those files are tools that just > indiscriminately download everything. Unless there are users of those > mirrors who > follow vastly different usage patterns than what we see on PyPI itself, > the primary > purpose of bdist_wininst, bdist_msi, bdist_dmg, etc on PyPI is to consume > disk space > and bandwidth via the mirroring infrastructure. > > I’d also like to note, that the numbers above are conservative on what they > consider to be a “mirroring client”. For instance, devpi used to use the > default > requests user-agent, and we see downloads via the requests user agent, but > did not > count them as mirroring clients because it could be some other script > doing the > downloading. > I should also mention I have never come across anyone at Microsoft use the bdist_msi or bdist_winst installers (I've added Steve to this email in case my experience is wrong). Everyone I have encountered either uses conda or pip+wheels (hence why I keep poking the sdist and build API ideas as I want to give Christophe Gohlke something else to do with his time than provide wheels on Windows). ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] What is the official position on distutils?
The discussion of Sylvain's proposed changes to distutils suggests that there isn't a clear-cut agreement or position of this SIG -- and thus Python -- on changes to distutils, its future, etc. Is there an official position I'm not aware of? If not, could we get one so we know how to handle any more proposed changes to distutils? For me personally (and to start this conversation if necessary), I view distutils as the package in the stdlib that handles building extension modules in the stdlib for Python. That view means that if something doesn't service that explicit goal then it shouldn't go into distutils. Now that we are moving down the road towards making the build step configurable I'm fine with saying that distutils is not expected to work for people other than Python itself and others can use setuptools, enscons, or other projects while we continue to improve the situation to where the build system is just something you specify in pyproject.toml to generate your wheel. IOW distutils is only exposed publicly because Python doesn't hide anything, but making it useful for the general case of direct usage by people is a non-goal of the package. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] What is the official position on distutils?
On Sun, 28 Aug 2016 at 16:55 Sylvain Corlay wrote: > Distutils seems to be the de-facto standard for building extension modules > for Python and it is used for most of the third-party extensions out there. > I don’t think that it is reasonable to declare that it is now only meant > for Python itself. > Sorry, I didn't mean to suggest distutils was suddenly going to change or disappear, but I did want a direction/focus for what we want distutils to become long-term as we have this discussion of what to do with distutils any time a change is proposed. At this point distutils is practically a black box of code as no one wants to touch out of fear of breaking the world and that's not sustainable. > I actually see a contradiction in pointing out some lack of involvement in > the library to oppose well needed improvements, especially for a tool that > has so much adoption. > It's not a contradiction because new features != maintenance. And the usage of distutils makes its lack of maintenance even more scary. I guess my key point from what I said in my previous email is that we shouldn't perpetually ignore the fact that distutils is not to a level of support or quality that any of us are happy with and yet so many people rely on. For me this means we should prioritize getting our build API defined and sdists standardized so that distutils (or a replacement) can be in the stdlib to build Python itself only, and after that you get your build tool from PyPI. And if that means setuptools swallows/subsumes distutils then that's fine with me, but my key point is we are not maintaining distutils in a way I think any of us would choose to now and so I'm trying to prod this question to start thinking about how to fix the maintenance problem long-term. > > As we discussed earlier, even though it is not a concern with C, checking > for availability of a compiler flag becomes crucial when building > extensions with C++ since new flavors of the language emerge every couple > of years now. It is important to be able to output meaningful error > messages when the compiler does not support C++[11/14/17] features if they > are needed for a given extension. This is a new aspect of the landscape in > this area. > > Finally, adding this method is a very straightforward change. `has_flag` > simply comes aside `has_function` as a method of ccompiler. I don't see a > more natural place for it. It would be a very weird design decision in my > opinion to not add it there, and instead to add it to distutils.ccompiler > by monkeypatching it in setuptools. > Honestly I have no comment on your feature, Sylvain, and I'm sorry your proposal happens to be the catalyst to this discussion. I'm just trying to get a general alignment from the PyPA group as the "distutils problem" comes up and has the same points made every time with no general decision on how to handle it long-term. -Brett > > Thanks, > > Sylvain > > On Mon, Aug 29, 2016 at 12:36 AM, Paul Moore wrote: > >> On 28 August 2016 at 18:05, Brett Cannon wrote: >> > The discussion of Sylvain's proposed changes to distutils suggests that >> > there isn't a clear-cut agreement or position of this SIG -- and thus >> Python >> > -- on changes to distutils, its future, etc. Is there an official >> position >> > I'm not aware of? If not, could we get one so we know how to handle any >> more >> > proposed changes to distutils? >> > >> > For me personally (and to start this conversation if necessary), I view >> > distutils as the package in the stdlib that handles building extension >> > modules in the stdlib for Python. That view means that if something >> doesn't >> > service that explicit goal then it shouldn't go into distutils. Now >> that we >> > are moving down the road towards making the build step configurable I'm >> fine >> > with saying that distutils is not expected to work for people other than >> > Python itself and others can use setuptools, enscons, or other projects >> > while we continue to improve the situation to where the build system is >> just >> > something you specify in pyproject.toml to generate your wheel. IOW >> > distutils is only exposed publicly because Python doesn't hide >> anything, but >> > making it useful for the general case of direct usage by people is a >> > non-goal of the package. >> >> FWIW, my view is: >> >> * distutils handles building of Python packages, both in the stdlib and >> outside >> * in practice, setuptools is almost always used with distutils, and >> any proposed change to distutils could be
[Distutils] Best way to handle packaging a project that only applies to certain versions of Python
Someone has asked that I do a new release of importlib that includes a LICENSE file on PyPI: https://pypi.org/project/importlib/. Historically I have had the setup.py simply not include any Python code when built on versions of Python that include importlib in the stdlib itself: https://github.com/brettcannon/importlib/blob/master/setup.py . But now I would like to do a wheel. Is there some way I'm not thinking of to have a wheel that will leave out code or not install itself if a certain version of Python is used? Or will the user have to specify a proper Python requirement in their requirements.txt to get that kind of experience? ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Best way to handle packaging a project that only applies to certain versions of Python
All of this seems like too much worry for a project that only applies to
Python 2.6 and earlier or Python 3.0. :) Thanks for the suggestions
everyone, but I'm just going to let the folks stuck on legacy versions deal
with the lack of wheels.
On Mon, 29 Aug 2016 at 23:21 Thomas Kluyver wrote:
> There is also the 'python requires' metadata, which can now be provided
> via setuptools, but I think pip 8.2 needs to be released before that is
> respected.
>
>
> On Tue, Aug 30, 2016, at 01:08 AM, Daniel Holth wrote:
>
> Let's say you have different code for python 3.3 and python 3.4+. Tag one
> wheel py33-none-any and the second py34-none-any. The second wheel is
> preferred on python 3.4 and above, but ignored by 3.3. The py3 tag wouldn't
> work well here.
>
> Order of preference on 3.5 is: ('py35', 'none', 'any'), ('py3', 'none',
> 'any'), ('py34', 'none', 'any'), ('py33', 'none', 'any'), ('py32', 'none',
> 'any'), ('py31', 'none', 'any'), ('py30', 'none', 'any')
>
> Enscons would let you control the wheel tag directly but doesn't yet let
> you build multiple wheels with a single invocation.
>
> On Mon, Aug 29, 2016, 19:46 Donald Stufft wrote:
>
>
> On Aug 29, 2016, at 7:34 PM, Brett Cannon wrote:
>
> Someone has asked that I do a new release of importlib that includes a
> LICENSE file on PyPI: https://pypi.org/project/importlib/. Historically I
> have had the setup.py simply not include any Python code when built on
> versions of Python that include importlib in the stdlib itself:
> https://github.com/brettcannon/importlib/blob/master/setup.py .
>
> But now I would like to do a wheel. Is there some way I'm not thinking of
> to have a wheel that will leave out code or not install itself if a certain
> version of Python is used? Or will the user have to specify a proper Python
> requirement in their requirements.txt to get that kind of experience?
> ___
> Distutils-SIG maillist - Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
> If your setup.py produces different output on different versions of
> Python, you’ll need multiple wheels.
>
> —
>
> Donald Stufft
> ___
> Distutils-SIG maillist - Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
> *___*
> Distutils-SIG maillist - Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
> ___
> Distutils-SIG maillist - Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] What is the official position on distutils?
On Fri, 2 Sep 2016 at 22:06 Nick Coghlan wrote: > On 2 September 2016 at 19:28, Paul Moore wrote: > > On 2 September 2016 at 09:58, Sylvain Corlay > wrote: > >> My point here was that I don't think that the proposed feature has much > to > >> do with the concerns that were raised about distutils in general, > unless it > >> is decided that incremental improvements to the library even backward > >> compatible will not be accepted anymore. > > > > Agreed. I think your feature is only stalled for distutils by the lack > > of people sufficiently comfortable with the code to apply it. The > > suggestions to add it to setuptools are more in the way of practical > > advice on how to make the feature available, rather than expressions > > of a policy that "we don't make changes like that in the stdlib". > > However, one of the other consequences of the status quo is that if > Jason's comfortable with a change for setuptools, there's very rarely > going to be anyone that will argue with him if he also considers it a > suitable addition to the next version of distutils :) > > Since Jason's primary involvement in distutils-sig & PyPA is as the > lead setuptools maintainer, it's easy for folks to be unaware of the > fact that he's a distutils maintainer as well. > > So perhaps that's what we should adopt as the official distutils-sig > policy? Any proposed distutils changes should *always* go into > setuptools, as that way they're available for all currently supported > Python versions, and then it's up to the setuptools project to > escalate changes or change proposals for stdlib inclusion when they > consider that an appropriate step. > If Jason is up for the responsibility that seems like a reasonable approach to take. It also helps test out features in setuptools first before upstreaming it. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] When can we kill Python 2.6 support?
I think the fact that Python 2.6 is past EOL means it's definitely up for consideration. As for the 3% usage, as a trite comparison that's the amount of scientists who deny climate change. So IMO that suggests 2.6 is not used enough to burden PyPA with the maintenance and those who still want to use it can take over maintaining 2.6 compatibility. On Fri, 2 Sep 2016 at 14:06 Donald Stufft wrote: > The packaging tools generally support 2.6+ and 3.(2|3)+ and that's sort of > been > where they've been at for a while now. I would like to think about what we > need > to be to start considering Python 2.6 as "too old" to support. In pip we > generally follow a usage based deprecation/removal of supported Pythons > but we > don't have any real guidelines for when something is at a low enough usage > to > consider it no longer supported and we instead just sort of wait until > someone > makes a case that it's "low enough". > > This issue tends to impact more than just pip, because once pip drops > support > for something people tend to start dropping it across the entire ecosystem > and > use pip's no longer supporting it as justification for doing so. > > I would like to take a look at Python 2.6 and try and figure out if we're > at a > point that we can deprecate and drop it, and if not what is such a point. > > Looking at pure usage numbers for "modern" versions of pip (6, 7, and 8) > for > downloading from PyPI I see the usage is ~3% of downloads are via Python > 2.6. > The only thing lower than Python 2.6 that is still supported is Python 3.3. > > Python 2.6 itself has been EOL since 2013-10-29 which is now just about 3 > years > ago. It's SSL module is not generally secure and requires the use of > additional > installed modules to get it to be so. I believe the only place to get a > Python 2.6 that is "supported" is through the Enterprise-y Linux > Distributions > like RHEL/CentOS/etc. > > Do we think that a ~3% usage of Python 2.6 and being end-of-life'd for ~3 > years > is enough to start deprecating and dropping 2.6? If not what sort of > threshold > do we think is enough? It'd be nice to get the albatross of Python 2.6 > support > off from around our necks but I'm not sure how others feel. Obviously all > of > the existing versions of all of the tooling will still be fully functional > so > Python 2.6 users will simply need to not upgrade their tooling to continue > to > work, *but* it also means that they will be left out of new packaging > features > (and likewise, people can't rely on them if they still wish to support > 2.6). > > Thoughts? > > — > Donald Stufft > > > > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 517: Build system API
Thanks for starting this up again! My vote is for 1c (easier to add 1a later), and dashes (for some reason I just like how they look more in config files). On Wed, Nov 23, 2016, 06:36 Thomas Kluyver, wrote: > I'd like to push PEP 517 forwards again. This PEP specifies a general > build system interface so that a source tree can specify a tool (such as > flit), and pip can ask that tool to generate a wheel. This would allow > us to install from an sdist or a VCS checkout without running a setup.py > file. > > https://www.python.org/dev/peps/pep-0517/ > > Developments since the last time this was discussed (to my knowledge): > - It now uses the pyproject.toml file specified in PEP 518, which was > already accepted. 517 originally specified a JSON file; 518 explains why > TOML is preferred (basically: comments). > - I have implemented the proposed build-system API in a pull request for > Flit; this has been fairly straightforward so far. > https://github.com/takluyver/flit/pull/77 > > Questions: > 1. Editable installs. The PEP currenly specifies a hook to do an > editable install (like 'pip install -e' or 'setup.py develop') into a > given prefix. I don't think that specifying a prefix is sufficient to > cover '--user' installation, which uses a different install scheme, > especially on Windows and OSX framework builds. We can: > a. Add an extra parameter 'user' to the hook, to override the prefix and > do a user install. > b. Leave it as is, and do not support editable user installation (which > would make me sad, as I do editable user installs regularly) > c. Decide that editable installs are too fiddly to standardise, and > leave it to users to invoke a tool directly to do an editable install. > > 2. Dash vs. underscore, bikeshed reloaded! Currently, the table name > uses a dash: [build-system], but the key added by PEP 517 uses an > underscore: build_backend. This seems a bit messy. I propose that we > change build_backend to build-backend for consistency. Dashes and > underscores can both be used in a TOML key without needing quoting. > > Thanks, > Thomas > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Packaging multiple wheels in the same archive
This then ties into Kenneth's pipfile idea he's working on as it then makes sense to make a wagon/wheelhouse for a lock file. To also tie into the container aspect, if you dev on Windows but deploy to Linux, this can allow for gathering your dependencies locally for Linux on your Windows box and then deploy the set as a unit to your server (something Steve Dower and I have thought about and why we support a lock file concept). And if we use zip files with no nesting then as long as it's only Python code you could use zipimporter on the bundle directly. On Tue, Nov 22, 2016, 22:07 Nick Coghlan, wrote: > [Some folks are going to get this twice - unfortunately, Google's > mailing list mirrors are fundamentally broken, so replies to them > don't actually go to the original mailing list properly] > > (Note for context: I stumbled across Wagon recently, and commented > that we don't currently have a good target-environment-independent way > of bundling up a set of wheels as a single transferable unit) > > On 23 November 2016 at 03:44, Nir Cohen wrote: > > We came up with a tool (http://github.com/cloudify-cosmo/wagon) to do > just > > that and that's what we currently use to create and install our plugins. > > While wheel solves the problem of generating wheels, there is no single, > > standard method for taking an entire set of dependencies packaged in a > > single location and installing them in a different location. > > Where I see this being potentially valuable is in terms of having a > common "multiwheel" transfer format that can be used for cases where > the goal is essentially wheelhouse caching and transfer. The two main > cases I'm aware of where this comes up: > > - offline installation support (i.e. the Cloudify plugins use case, > where the installation environment doesn't have networked access to an > index server) > - saving and restoring the wheelhouse cache (e.g. this comes up in > container build pipelines) > > The latter problem arises from an issue with the way some container > build environments (most notable Docker's) currently work: they always > run in a clean environment, which means they can't see the host's > wheel cache. One of the solutions to this is to let container builds > specify a "cache state" which is archived by the build management > service at the end of the build process, and then restored when > starting the next incremental image build. > > This kind of cache transfer is already *possible* today, but having a > standardised way of doing it makes it easier for people to write > general purpose tooling around the concept, without requiring that the > tool used to create the archive be the same tool used to unpack it at > install time. > > Cheers, > Nick. > > -- > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] how to use python with cmd!
The best place to ask for this kind of general help is the python-list or python-tutor mailing lists. On Sat, 26 Nov 2016 at 09:41 Code Club Djelfa wrote: > hi > i am from algeria and i ask you! > how to use python with cmd in windows 7 > and How to download and install Python Packages and Modules with Pip > help me *-* *-* > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Maintaining a curated set of Python packages
If people are serious about trying to prototype this stuff then the easiest way might be coming up with shell scripts that do the prompting if it's faster to iterate that way than doing a full-blown GUI. Now that WIndows 10 has WSL/Bash it means for the first time all 3 major OSs have a common shell people can work from. You could even go as far as making the shell scripts be Cookiecutter templates such that people can experiment with things being included/left out (e.g. an instructor wants to require Python 3.5, no git, and have people work from a virtual environment and so they generate the shell script everyone is told to run to get things going/verify the student's system is set up properly). On Fri, 16 Dec 2016 at 05:52 Daniel Holth wrote: > I'm also a visual studio code fan. It is the first editor I've tried that > feels lightweight like Vim but has the power of many plugins. That, and the > text rendering is excellent. > > https://pypi.python.org/pypi/Stallion is a lovely GUI package manager. > > One possibility to consider is that virtualenv itself is a bad idea. Why > should the Python interpreter executable, rather than the program being > run, determine the set of packages that is available for import? It is > confusing and inconvenient to have to deal with environments at all. Yes, > even if you are using a helper. Maybe there can be a better way to manage > dependencies that is not completely disjoint from setup.py. > That just sounds like node_modules/ and I personally don't want to go down that route. If you view the interpreter as another component of an app then the disconnect doesn't seem so nutty (at least in my head; at that point it's just another /usr/local to me). -Brett > > On Fri, Dec 16, 2016 at 8:07 AM Nick Coghlan wrote: > > On 16 December 2016 at 20:57, Glyph Lefkowitz > wrote: > > > Anyhow, Xcode is far from perfect - many of the places it touches the UNIX > pipeline are extremely sharp edges you can easily impale yourself on (and > don't get me started about codesigning) - but it nevertheless points at a > different potential direction. For example; why expose the concept of a > "virtual environment" directly at all? "New Project" could just create a > requirements.txt and a setup.py for you, alongside a git repo and a > virtualenv for that project. Or, the UI could be geared towards setting up > a tox.ini rather than a virtualenv, and run everything through tox so it's > in an isolated environment with defined requirements. This is a best > practice anyway so why not make it easier to start early? > > This might all be way too much work, but I think it's important to > remember it's possible. > > > Yeah, I think we agree more than we disagree here. The main thing is that > one of the key ways newcomer-friendly environments make themselves more > approachable is to *constrain choice*. > > XCode usability benefits from being Apple-centric. Ditto for Visual Studio > and MS. > > Linux and Python, by contrast, were both born out of a DIY culture where > folks being free to choose their own tools was initially perceived solely > as a highly desirable feature, rather than as a potential barrier to entry > for newcomers. > > That means there's an argument to be made that something like YHat's Rodeo > [1] might be a better starting point for data analytics in Python than > jumping straight to Jupyter Notebook, and it's also why the Mu editor [2] > exists as a dedicated tool for folks learning Python by way of the > micro:bit project. > > [1] http://rodeo.yhat.com/docs/ > [2] http://codewith.mu/ > > However, the reason I brought up the Curse and Firefox GUI examples was to > emphasise the problems they hide from the default rich client experience: > > - their default focus is on managing one environment per device > > > In the analogous Python tool, one could replace "per device" with "per > project" - and perhaps have a "default project" so something useful could > happen even before you've decided what you're doing... > > > But we've immediately bumped the complexity level up in doing so, and it's > a level of complexity that many people initially spending all of their > development time on a single project may not need. > > I thought this thread was already interminable, I look forward to reading > the never-ending rest of it now that you've raised the grim spectre of the > PyPI user-ratings feature from the dead :). > > > All the arguments against integrating user ratings into a service that's > focused on lowering barriers to publication still hold, so I'm really just > noting that that decision to create a friendlier publishing environment > *does* introduce some additional constraints elsewhere in the distribution > pipeline. > > User-curated package sets strikes me as the _lowest_ priority feature out > of all of those, if we are ordering by priority to deliver a good user > experience. I know "steam curators" have been brought up before - but > we're talking ab
Re: [Distutils] Can't upload sdist: "File already exists"
Because you already uploaded a wheel for version 0.1.2 you can't upload any other files for that version, else people could accidentally upload e.g. an sdist with code different from what was already uploaded in the wheel. If you want an sdist then I would do another release as version 0.1.2post1 with the wheel and sdist (or whatever the proper post release version format is; on my phone so a pain to look up right now). On Wed, Dec 21, 2016, 12:30 Nick Timkovich, wrote: > I have a little package "huffman" where I build an sdist and wheel (python > setup.py sdist bdist_wheel) and both seem to get built and can install > fine. I can't seem to upload both to PyPI because the "File already exists": > > $ twine upload dist/* > Uploading distributions to https://upload.pypi.org/legacy/ > Uploading huffman-0.1.2-py2.py3-none-any.whl > Uploading huffman-0.1.2.tar.gz > HTTPError: 400 Client Error: File already exists. for url: > https://upload.pypi.org/legacy/ > > Subsequent call to upload *just* the tarball fails the same way. I can't > see an sdist anywhere, and uploading it via the website or twine just tells > me it's already there...somehow. Asking pip to try to give it to me fails > though (the binary works, however): > > $ pip download --no-cache-dir --only-binary :all: huffman==0.1.2 > Collecting huffman==0.1.2 > Downloading huffman-0.1.2-py2.py3-none-any.whl > Saved ./huffman-0.1.2-py2.py3-none-any.whl > Successfully downloaded huffman > > > $ pip download --no-cache-dir --no-binary :all: huffman==0.1.2 > Collecting huffman==0.1.2 > Could not find a version that satisfies the requirement huffman==0.1.2 > (from versions: ) > No matching distribution found for huffman==0.1.2 > > Am I missing something? I am as sure as I can be that I didn't upload it > twice; I bumped my version up one because I figured that may have been it. > "twine register" that some guides mention just gets shot down with an HTTP > "410 [...] simply upload the file" > > Cheers, > Nick > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Announcement: TLSv1.2 will become mandatory in the future
On Tue, 10 Jan 2017 at 12:51 Donald Stufft wrote: > [SNIP] > > > It would be really nice if we could deprecate `ssl` (which has a bunch of > OpenSSL specific stuff in it) and add a new `tls` module that served as an > implementation agnostic library that would use OpenSSL on *nix, > SecureTransport on macOS, and SChannel on Windows. However, in the mean > time there are some folks poking to see about making something pip suitable > that will enable us to use SecureTransport at least. > I know both Cory Benfield and Christian Heimes brought this up briefly at the PyCon US 2016 language summit at the end of their SSL discussion, but I don't think it went anywhere because there was some other discussion that dominated the end of their talk (I've now tweeted at them about this discussion). I know Steve has also said he would love to see a agnostic TLS library so that Windows' built-in libraries for this stuff could be directly used. With the predicament this is going to put us in I think it makes it very prudent to create a tls module for the stdlib. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Has anyone looked at bug 29225?
It's probably a combination of people being busy and the fact that it involves distutils which not very many people feel comfortable reviewing patches for and changing its semantics. Regardless of what happens to your patch, though, thanks for taking the time to report the problem you ran into and trying to fix it! On Wed, 18 Jan 2017 at 23:21 Elvis Stansvik wrote: > Hi distutils folks, > > Just wanted to ask if anyone has had time to look at my > > http://bugs.python.org/issue29225 > > ? It's my first bug report _and_ first patch for Python, so it's quite > possible I screwed something up. Or people are just busy :) > > Cheers, > Elvis > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Has anyone looked at bug 29225?
Oh, over the 26 years of Python's lifetime you would be surprised what people come to rely on (including broken semantics which they have written fixes for themselves which then break when a proper fix is introduced). On Thu, 19 Jan 2017 at 10:01 Elvis Stansvik wrote: > 2017-01-19 18:52 GMT+01:00 Brett Cannon : > > It's probably a combination of people being busy and the fact that it > involves distutils which not very many people feel comfortable reviewing > patches for and changing its semantics. > > > Alright, that's understandable and sort of what I suspected. > > I hope (and believe) my patch preserves the semantics for all cases except > the specific one it fixes. The wrong behavior which it fixes is so > obviously wrong that I can't imagine someone relying on it somehow :) > > Elvis > > > > > Regardless of what happens to your patch, though, thanks for taking the > time to report the problem you ran into and trying to fix it! > > On Wed, 18 Jan 2017 at 23:21 Elvis Stansvik > wrote: > > Hi distutils folks, > > Just wanted to ask if anyone has had time to look at my > > http://bugs.python.org/issue29225 > > ? It's my first bug report _and_ first patch for Python, so it's quite > possible I screwed something up. Or people are just busy :) > > Cheers, > Elvis > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Enquire for python
Instructions can be found at https://packaging.python.org/installing/ On Wed, 25 Jan 2017 at 03:42 ali refaee wrote: > Dear Sir/Madam > I’m beginner in python, how can I get python modules? > when run some function > such as (import numpy as np) > message > ImportError: No module named numpy > > another ex > import matplotlib.pyplot as plt > ImportError: No module named matplotlib.pyplot > > how we install modules? > where are the modules? > can I get modules to run python? > > Sent from Windows Mail > > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python installation not working
This actually isn't the right place to ask for installation help, Chitra (this list is about how to package up Python projects). For general support questions you should email python-list. On Wed, 15 Feb 2017 at 05:11 Chitra Dewan via Distutils-SIG < distutils-sig@python.org> wrote: > Hello, > > I am *beginner in Python* > I am facing problems in installing Python 3.5 on my windows vista x32 > machine. > I downloaded python-3.5.2.exe from Python.org. It is downloaded as an > exe. When I try to install it via "Run as administrator" , nothing > happens. Same behavior with 3.6 version > > kindly advise > > > Regards & Thanks, Chitra Dewan > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 426 moved back to Draft status
On Fri, 10 Mar 2017 at 07:56 Nick Coghlan wrote: > On 11 March 2017 at 00:52, Nathaniel Smith wrote: > > On Fri, Mar 10, 2017 at 1:26 AM, Nick Coghlan wrote: > > Hi folks, > > > > After a few years of dormancy, I've finally moved the metadata 2.0 > > specification back to Draft status: > > > https://github.com/python/peps/commit/8ae8b612d4ea8b3bf5d8a7b795ae8aec48bbb7a3 > > We have lots of metadata files in the wild that already claim to be > version 2.0. If you're reviving this I think you might need to change > the version number? > > > They're mostly in metadata.json files, though. That said, version numbers > are cheap, so I'm happy to skip straight to 3.0 if folks think it makes > more sense. > +1 on jumping. > > > > Based on our last round of discussion, I've culled a lot of the > complexity > > around dependency declarations, cutting it back to just 4 pre-declared > > extras (dev, doc, build, test), > > I think we can drop 'build' in favor of pyproject.toml? > > > No, as that's a human edited input file, not an output file from the sdist > generation process. > > > Actually all of the pre-declared extras are really relevant for sdists > rather than wheels. Maybe they should all move into pyproject.toml? > > > Think "static release metadata in an API response from PyPI" for this > particular specification, rather than something you'd necessarily check > into source control. > Or "stuff PyPI has to parse, not you". ;) > That's actually one of the big benefits of doing this post pyproject.toml > - with that taking care of the build system bootstrapping problem, it > frees up pydist.json to be entirely an artifact of the sdist generation > process (and then copying it along to the wheel archives and the installed > package as well). > > That said, that's actually an important open question: is pydist.json > always preserved unmodified through the sdist->wheel->install and > sdist->install process? > Is there a reason not to? > > There's a lot to be said for treating the file as immutable, and instead > adding *other* metadata files as a component moves through the distribution > process. If so, then it may actually be more appropriate to call the > rendered file "pysdist.json", since it contains the sdist metadata > specifically, rather than arbitrary distribution metadata. > Since this is meant for tool consumption and not human consumption, breaking the steps into individual files so that they are considered immutable by tools farther down the toolchain makes sense to me. > > > > and some reserved extras that can be used to > > say "don't install this, even though you normally would" (self, runtime). > > Hmm. While it's not the most urgent problem we face, I really think in > the long run we need to move the extras system to something like: > > > https://mail.python.org/pipermail/distutils-sig/2015-October/027364.html > > The current extras system is inherently broken with respect to > upgrades, and reified extras would solve this, along with several > other intractable problems (e.g. numpy ABI tracking). > > So from that perspective, I'm wary of adding new special case "magic" > to the extras system. Adding conventional names for things like > test-dependencies is fine, that doesn't pose any new obstacles to a > future migration. But adding complexity to the "extras language" like > "*", "self", "runtime", etc. does make it harder to change how extras > work in the future. > > > Technically the only part of that which the PEP really locks in is barring > the use of "self" and "runtime" as extras names (which needs to be > validated by a check against currently published metadata to see if anyone > is already using them). > Do you have something planned for these names? > > '*' is already illegal due to the naming rules, and the '-extra' syntax is > also an illegal name, so neither of those actually impacts the metadata > format, only what installation tools allow. The main purpose of having them > in the PEP is to disallow using those spellings for anything else and > instead reserve them for the purposes described in the PEP. > > I'd also be fairly strongly opposed to converting extras from an optional > dependency management system to a "let multiple PyPI packages target the > same site-packages subdirectory" because we already know that's a nightmare > from the Linux distro experience (having a clear "main" package that owns > the parent directory with optional subpackages solves *some* of the > problems, but my main reaction is still "Run awaaay"). > > It especially isn't needed just to solve the "pip forgets what extras it > installed" problem - that technically doesn't even need a PEP to resolve, > it just needs pip to drop a pip specific file into the PEP 376 dist-info > directory that says what extras to request when doing future upgrades. > Similarly, the import system offers so much flexibility in checking for > optional packages at startup and lying about where imports a
Re: [Distutils] reproducible builds
On Tue, 21 Mar 2017 at 04:54 Marius Gedminas wrote: > On Mon, Mar 20, 2017 at 11:30:59AM +, Robin Becker wrote: > > thanks for this; it seems the emphasis is on security. If the intent is > that > > reportlab should be able to reliably reproduce the same binary output > then I > > think I need to do more than just fix a couple of dates. We use many > > dictionary like objects to produce PDF and I am not sure all are sorted > by > > key during output. > > I'm sure the reproducible builds folks will send you patches if they > find any spots that you missed. ;-) > > > Is there a way to excite dictionary ordering changes? I believe there was > > some way to modify the hashing introduced when the dos dictionary attacks > > were an issue. Would it be sufficient to generate documents with say > Python > > 2.7 and check against 3.6? > > Python 3.6 changed the dict implementation so the ordering is always stable > (and matches insertion order). > Do realize that is an implementation detail and not guaranteed by the language specification, so it won't necessarily hold in the future or for other interpreters. -Brett ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Is the SourceForge repo still used? (was: package ownership transfer: kobo
On Mon, 27 Mar 2017 at 05:02 Ken Dreyer wrote: > [SNIP] > [1] https://sourceforge.net/p/pypi/support-requests/632/ I didn't even know this repo existed until I noticed that the sidebar on pypi.python.org. It isn't mentioned anywhere on https://pypi.org/help/ . Should people still be using that project or the Google Form? Or do we need a GitHub repo to track things like package ownership transfers? -Brett ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] bug with get-pip.py?
How old is the version of setuptools being used? I vaguely remember this error from a while back and it being fixed in a later release of setuptools. On Thu, 27 Apr 2017 at 13:22 Randy Syring wrote: > I've installed Python 3.6 on Ubuntu 14.04 through the dead snakes PPA and > I'm trying to get pip installed. > > $ python3.6 ~/tmp/get-pip.py > Traceback (most recent call last): > File "/home/rsyring/tmp/get-pip.py", line 20061, in > main() > File "/home/rsyring/tmp/get-pip.py", line 194, in main > bootstrap(tmpdir=tmpdir) > File "/home/rsyring/tmp/get-pip.py", line 119, in bootstrap > import setuptools # noqa > File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 12, > in > from setuptools.extension import Extension > File "/usr/lib/python3/dist-packages/setuptools/extension.py", line 7, > in > from setuptools.dist import _get_unpatched > File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 16, in > > import pkg_resources > File "/usr/lib/python3/dist-packages/pkg_resources.py", line 1479, in > > register_loader_type(importlib_bootstrap.SourceFileLoader, > DefaultProvider) > AttributeError: module 'importlib._bootstrap' has no attribute > 'SourceFileLoader' > > Any ideas? > > *Randy Syring* > Husband | Father | Redeemed Sinner > > > *"For what does it profit a man to gain the whole world and forfeit his > soul?" (Mark 8:36 ESV)* > > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python modules -reg.,
This list is for discussing the development of how to install packages in Python. For suggestions of what to install, I would ask the python-list mailing list. On Tue, 9 May 2017 at 02:20 Kamalini wrote: > Dear sir, > > I am Nalini working as Assistant Professor in Velammal Institute of > Technology,chennai. > > I am working in python 3.4.4. I would like to import excel for research > purpose. > > Kindly guide me to install supporting packages. > > Thanks, > > S. Nalini > 9894812725 <(989)%20481-2725> > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 517 - specifying build system in pyproject.toml
On Fri, May 19, 2017, 09:20 Thomas Kluyver, wrote: > On Fri, May 19, 2017, at 05:17 PM, Paul Moore wrote: > > On 19 May 2017 at 16:53, Daniel Holth wrote: > > > Congrats on getting 518 in. > > > > Agreed, by the way. That's a big step! > > Thanks both. It does feel like an achievement. :-) > As it should! Thanks for bringing the PEP to life! -brett ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface
Just to make sure I'm following this correctly, Donald is asking for: - A way for pip to ask back-ends what files should be in an sdist from a source checkout or to make an actual sdist - Because sdists are a thing and so we should support them properly - To make it so building wheels interact with just the files from an sdist instead of skipping that and going straight from a source checkout - Have this be a part of PEP 517 or at least a requirement for back-ends to support so it doesn't get left out Am I missing anything? On Tue, 30 May 2017 at 09:48 Donald Stufft wrote: > > On May 30, 2017, at 12:29 PM, Thomas Kluyver wrote: > > What about saying that the copying step, if necessary, is part of the > build backend's responsibilities? I.e. pip doesn't copy the whole directory > to a temporary build location, but the build backend may decide to do that > at its discretion when it's asked to build a wheel. pip would continue to > handle this for setup.py builds. > > > > That still leaves the other use cases for building sdists unsatisfied. In > addition it’ll likely be pip that gets the bug reports when some backend > inevitably doesn’t copy those files and then leaves random debris laying > about (including things like read only file systems where random crap > *can’t* be written to the ``.`` directory or mounting the same package in > multiple docker containers that would cause different things to be pooped > into the ``.`` directory). > > — > > Donald Stufft > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface
On Tue, 30 May 2017 at 11:40 Donald Stufft wrote: > > On May 30, 2017, at 2:17 PM, Brett Cannon wrote: > > Just to make sure I'm following this correctly, Donald is asking for: > >- A way for pip to ask back-ends what files should be in an sdist from >a source checkout or to make an actual sdist > - Because sdists are a thing and so we should support them properly > - To make it so building wheels interact with just the files from > an sdist instead of skipping that and going straight from a source > checkout >- Have this be a part of PEP 517 or at least a requirement for >back-ends to support so it doesn't get left out > > Am I missing anything? > > > > More or less. If I had my druthers we’d just add another mandatory method > to the API, something like: > > def build_sdist_tree(sdist_directory, config_settings) -> None: > … > > All this does is assemble the would be sdist tree into sdist_directory, > handling any sdist creation time steps WITHOUT actually tar+gz’ing up this > tree. My tests show that this is basically as fast as the copytree option > in the simple case (which makes sense, that’s basically all it is) and is > way faster than actually assembling the entire sdist tarball and all. > So the back-ends then need to provide a way for users to specify what files to go into the sdist (e.g. the MANIFEST.in case). Obviously a fallback is to just copy everything or just what is used to build the wheel (overly board or overly selective, respectively), but I doubt tools will want to do either in general and so they will need to come up with some solution to specify extra files like READMEs and stuff. > > To handle creation, we can either have ``twine sdist`` or something like > that which just calls the above API and then runs ``shutil.make_archive()`` > on the final directory. > Do we want to say that sdist creation is the front-end's job? If so that is different from PEP 517 where the back-end packages zip up the wheel versus simply providing all the data and files that make up a wheel as seems to be suggested in the sdist case here. > > When building a wheel, pip can then just inside of this directory, call > into the wheel metadata/wheel build API and ultimately install that wheel. > It may even make sense for the build_wheel API to produce an “unpacked > wheel” as well and again let something like ``twine wheel`` handle running > ``shutil.make_archive()`` on it. Similar to the sdist case, this would > further reduce the install time by avoiding the need to zip and then > immediately unzip inside of pip. > If we're going to go down a route where back-ends simply provide the files and the front-ends do the packaging then it makes sense to do that for wheels as well in PEP 517, separating the back-ends to create the components of the final artifact while the front-ends handle the bundling up for all of it into its final form. This makes the back-ends purely a compiler-and-metadata thing and the front-ends a manage-final-files thing. -Brett > > One unrelated thing I just noticed, I don’t think the PEP states how pip > is supposed to communicate to this API what it’s actually building. Is it > just assumed to be ``os.curdir()``? Ideally I think we’d add another > parameter to all of these functions that is the directory containing the > “input” to each of these. > > — > > Donald Stufft > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface
On Wed, 31 May 2017 at 14:14 Thomas Kluyver wrote: > On Wed, May 31, 2017, at 09:16 PM, Donald Stufft wrote: > > How you build the release-quality sdist isn’t really of concern of PEP 517 > any more than building a release quality wheel is, it’s up to the build > tool to implement that as it makes sense for them. > > > But if we have a hook for building something called an sdist, we need to > define what an sdist is. The definition you're referring to is a functional > definition sufficient for a tool that only wants to install it, but it > doesn't cover what an sdist means or how it fits into workflows. > > I could see this as an argument that the PEP should have *both* a > build_sdist and a prepare_build_files hook, if you don’t think that the > build_sdist hook is suitable on it’s own. > > > I would prefer that to the the current status of one hook that tries to > cover both use cases. > > I'd still rather specify prepare_build_files in this PEP, and leave the > sdist hook to a later PEP. I don't see a great need for a standard > interface to building an sdist: the developer doing that by calling their > build tool directly seems adequate for the majority of use cases. > So it sounds like the list_build_files() part of the API is still useful for isolated builds versus in-place builds, correct? As for Thomas' idea that "calling a build tool directly seems adequate" is somewhat interesting and not something I thought about. Let's look at Donald's list of current ways to get from something to installation (which I know we want to scale back): 1) VCS Checkout -> Installed 2) VCS Checkout -> Sdist -> Installed 3) VCS Checkout -> Wheel -> Installed 4) VCS Checkout -> Sdist -> Wheel -> Installed 5) VCS Checkout -> Editable Install 6) VCS Checkout -> Sdist -> Editable Install OK, so what Thomas is suggesting is this isn't all necessarily directly under pip's purview. So if you take that list and break out those steps into "back-end responsibility" and "front-end responsibility" then you end up with: Back-end (e.g. flit): 1) VCS Checkout -> wheel (driven by pip) 2) VCS Checkout -> sdist 2) sdist -> wheel (driven by pip) Front-end (e.g. pip): 1) wheel -> installed You can then generate the non-editable install steps above by combining these roles. And I think the point Thomas is trying to make is if you look at back-end#2 you can simply leave pip out of it in a post-PEP 517 world if you view an sdist as a trimmed down VCS checkout that pip just needs to know how to unpack. But thinking from a more fundamental perspective, why does pip even need to be involved with any part of PEP 517? If pip is meant to be viewed as a package manager then isn't its key focus on fetching and installing appropriate things? Well, it's because pip may have to build a wheel from a post-517 sdist that it downloaded from PyPI when a project doesn't provide e.g. a Windows wheel. That's why pip needs to be PEP 517-aware at all. Otherwise pip could just work in a wheel-only world and not even care about building wheels to begin with. But where does building sdists come into play? You need to be able to make them, and you need to be able to get them up to PyPI. But beyond pip needing to know how to unpack an sdist to make the wheel it wants to work with, pip doesn't actually *need* to produce an sdist. But I think *twine* is the tool that needs a way to specify how to produce an sdist. If we want to view twine as the tool to upload artifacts to PyPI then we need twine to know how to produce sdists and wheels in a PEP 517 world, not pip. Maybe I'm missing something fundamental here about pip and all of this is wrong, but from where I'm sitting it seems the key thing pip needs from PEP 517 is how to go from a bunch of source to a wheel so that it can install that wheel. Now pip needs to know how to *consume* an sdist that it gets from PyPI that has a pyproject.toml, but it technically doesn't need to know how to *produce* one if in the end it really just wants a wheel. Twine, OTOH, does need a way to produce an sdist as well as a wheel so it can upload those to PyPI. And so I think in a very wordy way, I just said we need to stop saying "pip needs a standardized way to produce an sdist" and instead start saying "twine needs a way to produce an sdist". And that leads to the question about whether PEP 517 must cover sdist *production* for twine because we want to have that solved before we have the *consumption* side in pip in place. Or put another way, are we okay with pip consuming things that twine simply can't make from a pyproject.toml file (yet)? A "yes" means PEP 517 shouldn't be held up, while a "no" means we need a solution for twine. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PyQT python module .
So this list is about discussing the development of the tools for
packaging, not individual packages themselves (e.g. we have nothing to do
with PyAudio). If you need help installing a package then I would ask on
the python-list or tutor mailing lists.
On Wed, 7 Jun 2017 at 07:54 Catalin George Festila via Distutils-SIG <
distutils-sig@python.org> wrote:
> Dear python team .
>
> I used python 2.7 and 3.4 version and I saw your simple way Installing
> Python Modules — Python 3.6.1 documentation
>
> Installing Python Modules — Python 3.6.1 documentation
>
> Hope this mail will help you and your changes will help python users.
>
> I think is need to fix some python modules, like: PyQt 4 and 5 with SIP.
> The main reason is the hard way to install this python modules.
> Also some python package PyAudio can be install into python 2.7 version
> but not into 3.4.
> This python module ( PyAudio ) is need by SpeechRecognition python module.
>
> I try also the wheel way , but nothing .
> For windows I used this :
>
> pip-review.exe --auto --verbose
>
> and working well with some packages but I got some errors:
>
> Rolling back uninstall of mysql-connector
> Command "C:\Python34\python.exe -u -c "import setuptools,
> tokenize;__file__='C:\\Users\\mythcat\\AppData\\Local\\Temp\\pip-build-21khqzin\\mysql-connector\\setup.py';f=getattr(tokenize,
> 'open', open)(__file__);code=f.read().replace('\r\n',
> '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record
> C:\Users\mythcat\AppData\Local\Temp\pip-80abjmph-record\install-record.txt
> --single-version-externally-managed --compile" failed with error code 1 in
> C:\Users\mythcat\AppData\Local\Temp\pip-build-21khqzin\mysql-connector\
>
>
> Thank you.
> Best regards.
>
> Cătălin George Feștilă
>
>
> https://python-catalin.blogspot.ro/
> ___
> Distutils-SIG maillist - Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 517: Open questions around artifact export directories
On Thu, 15 Jun 2017 at 08:25 Donald Stufft wrote: > > On Jun 15, 2017, at 10:10 AM, C Anthony Risinger > wrote: > > On Tue, Jun 13, 2017 at 8:53 PM, Nick Coghlan wrote: > >> On 13 June 2017 at 19:44, Thomas Kluyver wrote: >> > On Tue, Jun 13, 2017, at 02:27 AM, Nick Coghlan wrote: >> >> > I've updated the PR to specify zip archives for build_wheel and .tar.gz >> > archives for build_sdist. >> >> +1 >> >> I've added one suggestion, which is to explicitly require PAX_FORMAT >> for the sdist tarballs produced this way (that's a POSIX format >> standardised in 2001 and supported by both 2.7 and 3.x that >> specifically requires that the paths be encoded as UTF-8). While the >> standard library does still default to GNU_FORMAT in general, the >> stated rationale for doing so (it being more widely supported than >> PAX_FORMAT) was last updated more than 10 years ago, and I don't think >> it applies here. > > > I'm not trying to open a bikeshedding opportunity here -- and I tried to > ignore it, honest! -- but why are tarballs preferable to zipfiles for > sdists? > > I looked around the 517 threads to see if it had been covered already, and > all I found was that zipfiles have additional PKG-INFO expectations in > existing implementations, and other honorable mentions of their features > over tarballs. > > I've never understood the anti-affinity towards zip because the format > itself seems superior in many ways, such as the ability to easily append or > replace-via-append (which might actually help perf when being used as an > interchange format, with a repack/prune at the end), compress individual > files, and the brilliance of placing the central directory/manifest at the > end, allowing it to be appended to binaries, etc. and allowing rapid > indexing of files. Tarballs are a black box. > > Just seems a little odd/arbitrary to me that wheel is zip, python supports > zip importing, sdists are often zip, and Windows is zip-central, but we'd > decide to codify tar.gz. It doesn't affect me personally because I'm Linux > all the way down and barely remember how to use Windows, but with all the > existing zip usage, and technical superiority(?), if we are going to pick > something, why not that? At that point Python is all-zip and no-tar. > > Yeah, the inconsistency bugs me as well and I was about to email about this until this started up. :) > > It's not a strong opinion really, but since the PEP does attempt to limit > what's currently possible, can we add some verbiage as to why tar.gz is > preferred? Or consider it with more scrutiny? > > > > > Basically it’s the least disruptive option, the vast bulk of sdists are > using ``.tar.gz`` already, multiple downstream redistributors need to do > extra work to consume a .zip rather than a .tar.gz, and the technical > benefits of wheel don’t really matter much in the context of a sdist. Zip > isn’t a flat out win technical wise either, for instance .tar.gz can > compress smaller than a .zip file because it’s compression will act over > the entire set of files and not on a per file basis. > Then shouldn't we be pushing for .tar.xz instead? (The Rust community is actually moving to .tar.xz for distributing Rust itself: https://users.rust-lang.org/t/rustup-1-4-0-released/11268 ; I don't know what their plans are for crates) > > But mostly it’s just that most sdists are .tar.gz, and most Pythons except > older ones on Windows default to producing .tar.gz. > Well, I've been actively overriding that default and uploading only .zip files since it's so much easier to work with zip files on Windows and UNIX than tar.gz which are only easy on UNIX. :) And it is rather ironic that historically projects lack a Windows wheel and thus need the sdist and yet it's typically in a format that's the most painful to work with on Windows. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 517: editable installs
https://github.com/python/pythondotorg for website stuff. (and the most recent version should be at https://www.python.org/dev/peps/pep-0517/). On Fri, 16 Jun 2017 at 14:06 Nathaniel Smith wrote: > Oh, yeah, that's totally an ancient version of the PEP. That sucks. I > thought the legacy.python.org URLs were redirecting to python.org now. > Probably we should file a bug with the python.org infra team, but I'm not > sure where their tracker is. > > On Jun 16, 2017 1:48 PM, "Daniel Holth" wrote: > >> Probably this older version from >> http://legacy.python.org/dev/peps/pep-0517/ >> >> On Fri, Jun 16, 2017 at 4:42 PM Nathaniel Smith wrote: >> >>> On Jun 16, 2017 11:59 AM, "Daniel Holth" wrote: >>> >>> I noticed PEP 517 uses a prefix for editable installs. >>> >>> >>> Which part of the pep are you looking at? All I see about editable >>> installs is the note saying that they're deferred to a future pep. >>> >>> -n >>> >> ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Provide separate development and documentation URLs in PyPI metadata?
When you go to PyPI.org for a project you will find a link to the "homepage". Now for some projects that's their development site, e.g. GitHub URL. For others it's their documentation site, e.g. Read the Docs. And not all projects link to both from their PyPI page (e.g. yesterday I noticed flit didn't directly link to its doc site, although Thomas fixed this when I pointed it out). So my question/idea is if it would make sense to have separate, explicit development and documentation URLs in the PyPI metadata? For me that would make a project's PyPI page a true homepage as I would know that no matter what I could find where it's developed or where the docs are by going to PyPI. This compares to now where either I gamble and go to PyPI in hopes the developer provided the link there or hope I craft the right search on Google (which based on my search yesterday for [Sphinx makefile] shows I don't always succeed at). Anyway, just an idea I had based on my flit experience yesterday plus a tweet sent to me. (And if PyPI already supports this somehow then Thomas should brace for the feature request from me 😉.) ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Provide separate development and documentation URLs in PyPI metadata?
On Sat, Jun 24, 2017, 10:49 Thomas Kluyver, wrote: > On Sat, Jun 24, 2017, at 06:34 PM, Brett Cannon wrote: > > Anyway, just an idea I had based on my flit experience yesterday plus a > tweet sent to me. (And if PyPI already supports this somehow then Thomas > should brace for the feature request from me 😉.) > > > This prompted me to go and look at the metadata PEPs. I thought the URL > fields were only 'Home-Page' and 'Download-URL', but in fact PEP 345 added > a multi-use 'Project-URL' field: > > https://www.python.org/dev/peps/pep-0345/#project-url-multiple-use > > I quite like this idea - rather than prescribing a couple of specific > kinds of URL, it lets the project author list as many as make sense. > Perhaps we should recommend a set of common labels for URLs in this field, > e.g. 'Documentation', 'Bug tracker', 'Source repository'. > In flit's case it could have some reasonable set of defaults supported in the metadata section and then have a more general "URLs" section that's more free-form. -brett > This does leave open the question of which one to put in the mandatory > 'Home-Page' field (I usually use the address of the repo), and whether to > duplicate it in a 'Project-URL'. > > Thomas > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Provide separate development and documentation URLs in PyPI metadata?
On Sat, Jun 24, 2017, 10:51 Donald Stufft, wrote: > It's only kind of mandatory. The spec says it is but nothing fails IIRC if > you omit it. Perhaps we should just deprecate it and move everything to > project urls. > That sounds reasonable toe if the flexible, general case is going to be supported. -Brett > Sent from my iPhone > > > On Jun 24, 2017, at 1:48 PM, Thomas Kluyver > wrote: > > > > This does leave open the question of which one to put in the mandatory > 'Home-Page' field (I usually use the address of the repo), and whether to > duplicate it in a 'Project-URL'. > > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Provide separate development and documentation URLs in PyPI metadata?
On Tue, 27 Jun 2017 at 10:06 Thomas Kluyver wrote: > On Tue, Jun 27, 2017, at 04:58 PM, Barry Warsaw wrote: > > On Jun 25, 2017, at 11:33 AM, Nick Coghlan wrote: > > >I'd favour "Participate" over any variant of "Contribute", as without > > >context, "Contribute" makes me think of financial support in the > > >crowdfunding/tip jar sense. > > > > "Participate" may mean two different things. > > > > * Here's the development home, with a repo and issue tracker, > > contributions > > welcome! > > > > * Here's the mailing list or other forum where we discuss the future of > > Guido's Magical Mystery Time Machine. > > Perhaps this points to labelling URLs with nouns rather than verbs: > things like 'mailing list', 'source code' or 'issue tracker' seem less > ambiguous than 'participate' or 'contribute'. > I agree with Thomas on this one. Seeing a link that says "Participate" just feels like it's missing an exclamation point and subtext saying I could earn $2,000/week from it. ;) Since this has turned into bikeshedding over names when we have a general metadata solution, I've filed https://github.com/takluyver/flit/issues/116 and consider my question answered. :) ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] status check on PEP 517
On Thu, 20 Jul 2017 at 03:56 Nathaniel Smith wrote: > On Thu, Jul 20, 2017 at 3:22 AM, Paul Moore wrote: > > On 20 July 2017 at 10:46, Nathaniel Smith wrote: > >> To make this concrete: I'm *pretty* sure (?) that at this point all > >> the basic elements in my "simplified" rewrite are things that we now > >> have consensus are needed in some form, so maybe we can use that as a > >> kind of "minimal core" reference point: > >> > >> > https://github.com/njsmith/peps/commits/517-refactor-streamline/pep-0517.txt > > > > I don't have time this week to look at this, sorry, but a couple of > points: > > > > 1) This links to a commit in your repo, not to the actual document. > > Ugh, sorry! Pasted the wrong link. This is the correct one: > > > https://github.com/njsmith/peps/blob/517-refactor-streamline/pep-0517.txt > > > Also, I'd *really* prefer a rendered version if I'm going to review > > this. > > Annoyingly, the PEPs repo uses a .txt extension for .rst files, thus > cleverly preventing github from doing anything useful by default... > We will happily accept help in converting plaintext PEPs into reST ones so we can universally change the file extensions to .rst (see https://github.com/python/peps/issues/1) :) -Brett ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] status check on PEP 517
I'm also willing to bet that Thomas is busy prepping for JupyterCon. On Tue, 15 Aug 2017 at 03:14 Nick Coghlan wrote: > On 15 August 2017 at 15:11, 12345 67890 wrote: > > Do you have any update on when the PEP will be completed? > > From my perspective, the main item from the last round of discussions > that has yet to be explicitly added to the PEP is the ability for > build_sdist and build_wheel to raise NotImplementedError to indicate > that an operation has failed for aniticipated reasons and to provide > an error message explaining the problem. > > There were a couple of other items where the PEP authors weren't > necessarily sure that what was in the PEP was exactly what they wanted > to propose implementing, but my own view on those aspects is that > what's currently in the PEP is fine, as the last round of discussions > cleaned up some key problems where the way the explicitly out-of-tree > build support was designed in earlier versions of the API didn't align > with the way most build systems actually work (i.e. they expect to be > given the target directory as a build setting, allowing the build > system itself to decide exactly how to handle that). > > Cheers, > Nick. > > -- > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Documentation link on PyPI.org
If you search the archive of this mailing list you will notice I asked this exact question about a month or 2 ago (I think). The answer I got was it is used on PyPI.org, but I don't know how to set it with setuptools (flit will probably add support after PEP 517). On Sun, Aug 27, 2017, 13:01 Ronald Oussoren wrote: > On 27 Aug 2017, at 17:33, Wes Turner wrote: > > You can add a URL in the long_description (e.g. to > https://read-the-docs.readthedocs.org/ ) > > > I know, but that adds the URL to the project description and not the > sidebar. PEP 345 appears to specify metadata that could be used for this, > and warehouse seems to contain to make use of this (or at least a database > model that allows for specifying more links than just the default one). > > Adding links to the sidebar would be nicer with the new design on PyPI.org, > but there doesn’t seem to be a clean way to add “Project-URL” metadata as > described in PEP 345 to a wheel file. I could tweak my setup.py files to > add Project-URL metadata, but before I do that I’d like to confirm that (a) > that metadata is actually used by warehouse and (b) there is no cleaner way > to add this metadata using setuptools. > > Ronald > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 517 again
On Mon, 28 Aug 2017 at 16:29 Nathaniel Smith wrote: > On Mon, Aug 28, 2017 at 1:27 PM, Thomas Kluyver > wrote: > > On Mon, Aug 28, 2017, at 09:13 PM, Daniel Holth wrote: > > > > > Then end the debate by letting the PEP authors decide the return type, > and > > > write a paragraph explaining why the other options were rejected. It > is not > > > going to make a big difference. > > > > > > Will that work now? Are we all so tired of this endless war that people > will > > sign a peace treaty written by the people whose names are on the PEP > > (Nathaniel & me)? > > > > If so, let the trumpets sound, and the heralds declare that "return > > NotImplemented" is the way to do it. (I hope I've remembered Nathaniel's > > preference right ;-) > > Fine with me, though if it turns out Donald and Nick prefer the > version where the backend has to export an exception class then I'm > fine with that too. (I'm basing this on -- Donald has said he likes > it, and Nick hasn't commented yet but AFAICT it does address his > concerns with NotImplemented, so it seem like a plausible outcome.) > I loathe to weigh in on this and add yet another voice in this discussion, but the exported exception seems like the best solution for everyone involved from my lurking perspective. For me, using NotImplemented is a misuse of the singleton since I know what it's meant to be used for (and so I cringe every time I hear it brought up as a solution). And I was fine with NotImplementedError but if people want something more specific and None is out due to worries of accidental bare returns, then the exported exception comes the closest to making everyone happy (it does tick the EIBTI box :) . ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 517 - is it ready?
Are we waiting on anything then? Just Nick officially accepting the PEP? Or did I miss something while catching up on over 2 weeks of emails? I ask not only to help close this long chapter of packaging behind us, but because I'm leading a PyLadies workshop next month on packaging and it would be awesome to say "this is the future" (or "this is the bleeding edge" if Thomas gets flit updated before the workshop) vs. "this is *probably* be the future". :) On Mon, 11 Sep 2017 at 03:43 Paul Moore wrote: > Go for it. > > On 11 September 2017 at 11:36, Thomas Kluyver > wrote: > > Discussion on PEP 517 seems to have settled down, and I believe that > > Nick said he was about ready to accept it. Is everyone involved > > satisfied with the current state? Or is there anything else you think > > should be considered before accepting it? > > > > https://www.python.org/dev/peps/pep-0517/ > > ___ > > Distutils-SIG maillist - Distutils-SIG@python.org > > https://mail.python.org/mailman/listinfo/distutils-sig > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] how to install
The tutorial on how to install packages can be found at https://packaging.python.org/tutorials/installing-packages/. On Mon, 25 Sep 2017 at 05:01 Minesh Parikh wrote: > how to install setup.py file on my website > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Extracting distutils into setuptools
Actually, you can't remove any module in the stdlib that exists in both
Python 2.7 and 3.5 until after Python 2.7 is no longer supported:
https://www.python.org/dev/peps/pep-0004/#for-modules-existing-in-both-python-2-7-and-python-3-5
.
On Thu, 28 Sep 2017 at 23:31 Nick Coghlan wrote:
> On 29 September 2017 at 07:42, Matthias Klose wrote:
> > On 27.09.2017 11:37, Nick Coghlan wrote:
> >> On 27 September 2017 at 12:30, xoviat wrote:
> In short, I think your proposal is a good one, but how can we allocate
> manpower?
> >>>
> >>> (issue31595 on bugs.python.org)
> >>>
> >>> So what do others think of this? My sense of things is that people are
> open
> >>> to the idea, but there isn't a plan to make it happen.
> >>
> >> As long as Jason is amenable to the idea on the setuptools side, and
> >> pip's trick of injecting setuptools into a process to change the
> >> behaviour of distutils imports continues to work, then this seems like
> >> a reasonable idea to me.
> >>
> >> There will still be folks that want to patch distutils itself, but I
> >> don't see that as being substantially different from folks still
> >> patching the standard library Tcl/Tk bindings, even though there are a
> >> number of popular non-stdlib GUI toolkits.
> >
> > In this case, we should deprecate distutils in python3.7 and remove it
> in 3.8.
>
> Aye, that seems reasonable to me as well (especially if 3.7+ also
> offers an "ensuresetuptools" module), and, looking at the recent(-ish)
> commit history at
> https://github.com/python/cpython/commits/master/Lib/distutils, I'd
> suggest we ask Steve Dower if he'd be willing to be BDFL-Delegate for
> the related PEP.
>
> This is one of those changes where the longer term process improvement
> benefits are already reasonably clear to the folks involved in
> maintaining the software (i.e. we think it will provide a better end
> user experience overall if distutils switches to being updated and
> versioned based on the setuptools release cycle rather than the
> reference interpreter & standard library one), so the main thing the
> PEP process will need to ensure is that we're providing a sufficiently
> non-disruptive transition plan for getting from the current state to
> the more desirable state.
>
> Between Steve ("Don't break Windows"), you ("Don't break Debian (et
> al)"), me ("Don't break Fedora (et al)"), Donald ("Don't break pip"),
> and Jason ("Don't break setuptools"/"Don't lumber setuptools with an
> unreasonable ongoing maintenance burden"), along with everyone else on
> distutils-sig, python-ideas, and python-dev, I think we'll be able to
> make a reasonable assessment of what counts as "too disruptive".
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
> ___
> Distutils-SIG maillist - Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 517 - is it ready?
Spelling mistake fixed! And a huge thank you to everyone for trudging forward with this and reaching this point. I think this will be a real boon for the community. On Fri, 29 Sep 2017 at 01:08 Ben Finney wrote: > Nick Coghlan writes: > > > I've heard back off-list from the folks I was waiting to hear from, > > and I'm very happy to say that I'm provisionally accepting PEP 517, as > > defined in > > https://www.pypa.io/en/latest/specifications/#provisional-acceptance > > Good news. You might want to update the PEP document again, though; > https://www.python.org/dev/peps/pep-0517/#rrovisional-acceptance> > has the title “Rrovisional acceptance” :-) > > -- > \ “I know you believe you understood what you think I said, but I | > `\ am not sure you realize that what you heard is not what I | > _o__) meant.” —Robert J. McCloskey | > Ben Finney > > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Entry points: specifying and caching
On Wed, 18 Oct 2017 at 17:54 Nick Coghlan wrote: > On 19 October 2017 at 04:18, Alex Grönholm > wrote: > >> Daniel Holth kirjoitti 18.10.2017 klo 21:06: >> >> >> http://setuptools.readthedocs.io/en/latest/formats.html?highlight=entry_points.txt#entry-points-txt-entry-point-plugin-metadata >> >> >> http://setuptools.readthedocs.io/en/latest/pkg_resources.html?highlight=pkg_resources#creating-and-parsing >> >> It is not very complicated. It looks like the characters are mostly >> 'python identifier' rules with a little bit of 'package name' rules. >> >> I am also concerned about the amount of parsing on startup. A hard >> problem for certain, since no one likes outdated cache problems either. It >> is also unpleasant to have too much code with a runtime dependency on >> 'packaging'. >> >> Wasn't someone working on implementing pkg_resources in the standard >> library at some point? >> > > The idea has been raised, but we've been hesitant for the same reason > we're inclined to take distutils out: packaging APIs need to be free to > evolve in line with packaging interoperability standards, rather than with > the Python language definition. > > Barry Warsaw & Brett Cannon recently mentioned something to me about > working on a potential runtime alternative to pkg_resources that could be > installed without also installing setuptools, but I don't know any of the > specifics (and I'm not sure either of them follows distutils-sig). > I've been following distutils-sig for a couple of years now. :) And what Barry and I are working on is only a subset of pkg_resources, specifically the reading of data files included in a package. We aren't touching any other aspect of pkg_resources. Heck, until this discussion, "entry points" == "console scripts" for me so I don't really know what y'all are talking about standardizing when it comes to plug-in systems and metadata. Having said that, I do understand why Donald doesn't want to just go ahead and standardize something by giving it the level of a spec on packaging.python.org just because it's out there. But since entry points seem to be used widely enough, having them written down appropriately also seems reasonable. As a compromise, could entry points be documented as Thomas is suggesting, but have a note at the top saying something along the lines of "entry points are considered a setuptools-specific feature, but their wide spread use warrants a clear understanding of how they function for other packaging tools choose on their own to also support them"? Basically acknowledge there are ad-hoc, folk standards in the community that a decent chunk of people rely on and thus docs would be helpful, but don't need to be promoted to full-on, everyone-implements standard. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Announcement: Pip 10 is coming, and will move all internal APIs
On Sat, 21 Oct 2017 at 11:03 xoviat wrote: > Nick: > > That's generally a good idea, but one significant problem that can occur > is that the Python import system will cache certain libraries, people will > run "pip install," and then they will expect such libraries to be > available. I don't even know exactly how the caching for the import system > works, so I don't want to go and make claims about it that may be incorrect > (maybe you do?). However, it is important to keep that in mind when > considering an API. > Nick does know how the import system works, but to specifically address this point, as long as the module isn't already imported it's fine. If it has, though, then you will need to do a reload instead of an import, and even then that doesn't necessarily work the way some people want it to. -Brett > > 2017-10-21 6:15 GMT-05:00 Nick Coghlan : > >> On 21 October 2017 at 20:03, Paul Moore wrote: >> >>> Likely the biggest problems will be for people who call into the pip >>> resolver and build APIs, as I don't know of any alternatives out >>> there. But they were *definitely* breaking anyway, as we've made major >>> changes to that code (and will be making more). >>> >>> Also, I should note that we didn't take this decision lightly. We >>> don't have any particular objection in principle to having a supported >>> stable pip API, it's just that we don't have anything even close to >>> the resources needed to define a supported API, maintain it with >>> acceptable backward compatibility guarantees, and support users who >>> will inevitably be using it in unexpected and creative ways (we don't >>> even have the resources to support the limited use of pip's internals >>> that we see today). Also, pip was never designed originally as a >>> library, so we *would* have to design that API from scratch. As I >>> alluded to above, the existing internals code makes some strong >>> assumptions about how it's called - assumptions that would be >>> unacceptable in library code, but are fine in an application's >>> internal code. >>> >> >> (Note: this is entirely speculative, and I have no idea how hard it would >> be, so please read it as the question it's intended to be) >> >> Do you know if there any key APIs (like installation) that could be >> turned into wrappers around pip CLI calls in order to mitigate some of the >> impact? >> >> The reason I ask is because it's unlikely anyone else is going to >> understand how to emulate the previous functionality better than the pip >> devs would, and if there's an API for those particular invocations, than >> they can be covered directly by pip's test suite. >> >> Plus if there are previous API capabilities that *can't* currently be >> emulated via the CLI, then the pip devs are the folks in the best position >> to add the necessary CLI enhancements (such as the ones Noah asked about >> for doing a more selective `pip list`). >> >> If that's an approach you might be amenable to, then a 10.0 pre-release >> could be a good time to solicit PRs from folks that were using particular >> APIs and would be prepared to invest time in defining comparable CLI >> wrappers for them. >> >> Cheers, >> Nick. >> >> -- >> Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia >> >> ___ >> Distutils-SIG maillist - Distutils-SIG@python.org >> https://mail.python.org/mailman/listinfo/distutils-sig >> >> > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Announcement: Pip 10 is coming, and will move all internal APIs
On Sat, 21 Oct 2017 at 11:26 Donald Stufft wrote: > > > On Oct 21, 2017, at 2:15 PM, Brett Cannon wrote: > > as long as the module isn't already imported it's fine. > > > Negative imports get cached too don’t they? > Yes, that too. :) (aside: don't reply to technical emails while you're waiting to head out the door for a weekend away, you will forget details ;) . -Brett ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
