Re: Using a ChangeLog as a canonical source of package metadata
Ben Finney writes: > The idea is to parse from the Changelog the version metadata, and > record it in Setuptools metadata. Then the ‘pkg_resources’ module of > Setuptools allows programmatic access to that metadata. One tricky aspect is: at what specific point should the Changelog be parsed and the version metadata recorded in Setuptools package metadata? At first I thought it should be done immediately on starting ‘setup.py’, in order to have values to supply to the ‘setup()’ call. So I imported the ‘version’ module which itself imports ‘docutils’ to have the reST parsing available; then ‘setup.py’ continues by feeding the Changelog to a function which parses it and emits values which are used to supply parameters to ‘setup()’. As was revealed in a recent version of the code base, though, this causes a circular dependency. Docutils is a third-party library, which needs to be declared as a dependency and satsified before it can be imported. But that dependency can't be declared until ‘setup()’ has run to specify what the dependencies are! So in a later release I've had to break that circle by introducing an initial “unknown” state for the version information, in order to allow ‘setup()’ to run and get the dependencies installed. Then when the ‘setup.py egg_info’ command is run, the Changelog is parsed and the version info metadata file is injected into the Setuptools metadata for the distribution. Rather more complicated than I would like, because of the need to have ‘setup()’ as a top-level call in ‘setup.py’. This would be more straightforward if we could assume the commonly-deployed existence of a *declarative* build system (such as Make); but in Python we're stuck with ‘setup.py’ and its limitations for now. Fortunately, the hard work of many people have made those much better in recent years. I'm interested to know what people writing Python distributions think of this approach. Again, the example I'm discussing is in ‘python-daemon’'s code base, at https://alioth.debian.org/projects/python-daemon/>. Eventually I might propose this to the Distutils folks as a possible improvement, but I'd like to refine it more in the face of actual usage. -- \ “… a Microsoft Certified System Engineer is to information | `\ technology as a McDonalds Certified Food Specialist is to the | _o__) culinary arts.” —Michael Bacarella | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: Using a ChangeLog as a canonical source of package metadata
On Tue, 13 Jan 2015 00:09:04 -0800, wxjmfauth wrote: > Le mardi 13 janvier 2015 08:00:06 UTC+1, Steven D'Aprano a écrit : >> On Mon, 12 Jan 2015 05:24:00 -0800, wxjmfauth wrote: >> >> > To tell you the truth, I'm unable to >> > put your product to work. >> >> >> If you follow the instructions in the README, and it still doesn't >> work, that's a bug and I will be happy to fix it. >> >> If you insist on doing things your own way, and breaking the package, >> then I cannot help you. >> >> > No, I do not claim, your package is buggy. I just wanted to toy with it > without installing it in the XXX\pythonXX directory structure. It is a pure Python package. Copy the package directory out of the tar ball and put it where Python can see it, and it will work. If you need to read more about packages, start here: https://docs.python.org/2/tutorial/modules.html#packages -- Steve -- https://mail.python.org/mailman/listinfo/python-list
Re: Using a ChangeLog as a canonical source of package metadata
On Mon, 12 Jan 2015 05:24:00 -0800, wxjmfauth wrote: > To tell you the truth, I'm unable to > put your product to work. If you follow the instructions in the README, and it still doesn't work, that's a bug and I will be happy to fix it. If you insist on doing things your own way, and breaking the package, then I cannot help you. -- Steve -- https://mail.python.org/mailman/listinfo/python-list
Re: Using a ChangeLog as a canonical source of package metadata
On Sun, 11 Jan 2015 12:37:03 -0800, wxjmfauth wrote: > 1) I downloaded pyprimes-0.2.1a.tar.gz > 2) I extracted the relevant part, > the py files, the pyprimes subdirectory, > awful.py, compat23.py, factors.py, test.py, .. and put in > d:\junk That is not the way packages work. pyprimes is a package, which means it has to stay together in a single directory called "pyprimes", with a file called "__init__.py" inside it. It is not a set of independent modules. Please read the README file, it explains the correct way to install pyprimes. You should use the setup.py installer script. If you insist on manually doing this, you can copy the *entire* pyprimes package directory. That is, unpack the tar.gz file to something like this: pyprimes/ +-- CHANGES.txt +-- LICENCE.txt +-- MANIFEST +-- README.txt +-- setup.py +-- src/ +-- pyprimes/ +-- __init__.py +-- awful.py +-- factors.py +-- test.py etc. Copy the *entire* folder pyprimes/src/pyprimes/ and move that somewhere into your PYTHONPATH. Or instead you can: cd pyprimes/src # ***NOT*** pyprimes/src/pyprimes !!! and run python from there. -- Steve -- https://mail.python.org/mailman/listinfo/python-list
Re: Using a ChangeLog as a canonical source of package metadata
Steven D'Aprano writes: > I currently read this metadata from the Python code itself. The > advantages of putting the metadata into the source code include: > > - the source code is the definitive source of information about itself; The Changelog document should be in the same source tree and maintained by the same people, so I don't count that as a difference between the two approaches. > - even if the user deletes the README and CHANGELOG files, they can > still find the metadata; Perhaps I didn't emphasise it, but: The approach I'm advocating (as inspired by Debian packaging tools) has the version metadata *canonically* recorded in the Changelog, but not *only* stored there. The packaging tools parse the version metadata from the Changelog, then duplicate whatever metadata is needed in various other places — such as for programmatic access by the package's own code. > - if your application wants to report a version number (as many apps > do) then it is easy for them to do so. The idea is to parse from the Changelog the version metadata, and record it in Setuptools metadata. Then the ‘pkg_resources’ module of Setuptools allows programmatic access to that metadata. Thanks for enumerating those points, I think they are adequately addressed by this approach. > > I've now produced a small Python library which knows how to > > transform a reST Changelog to package metadata; and how to get that > > package metadata into and out of a Python distribution with > > Distutils. > > Sounds interesting. Where can we see this? It is the approach used in ‘python-daemon’ version 2. Find the source at https://alioth.debian.org/projects/python-daemon/>. The ‘ChangeLog’, ‘setup.py’, ‘version.py’, ‘daemon/_metadata.py’ files are the relevant ones to examine. Since this is a new approach I'm trying, I flubbed a couple of versions: you'll need version 2.0.1 or later (I omitted the ‘version’ module entirely from the earlier distribution), and you'll need ‘docutils’ already installed (a future ‘python-daemon’ distribution will declare this dependency). I'll be interested to see how Python developers like this. -- \ “The process by which banks create money is so simple that the | `\ mind is repelled.” —John Kenneth Galbraith, _Money: Whence It | _o__) Came, Where It Went_, 1975 | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: Using a ChangeLog as a canonical source of package metadata (was: Announce: PyPrimes 0.2.1a)
Ben Finney wrote: [...] > The perils of duplicate sources of information: a Changelog makes claims > about which version is latest, but the packaging metadata comes from > somewhere else. > > This problem is addressed quite well, in my opinion, by the Debian > packaging tools. The tools by default read the following information: > > * release version string > * maintainer name > * maintainer email address > * target suite (a conceptual “which Debian version should this go into”) > > from the Changelog document, automatically. This means that the > Changelog document, as well as being directly useful to the recipient as > a text document, becomes the canonical place to record all those fields > for the packaging tools to read. No need to maintain duplicate records. A very interesting way to handle this, and one which deserves further configuration. I currently read this metadata from the Python code itself. The advantages of putting the metadata into the source code include: - the source code is the definitive source of information about itself; - even if the user deletes the README and CHANGELOG files, they can still find the metadata; - if your application wants to report a version number (as many apps do) then it is easy for them to do so. But the disadvantages include: - it's a bit sad to have to put such metadata into the source code; - the risk of the changelog getting out of date. > I would like Python's packaging tools to have the same capability. > > This requires recording the Changelog document in a machine-parseable > format, with specified locations for each of the fields to be read for > each version. I wonder whether it would be better to have setup.py check the CHANGELOG and halt if it isn't up to date? > I have recently gone through a process of converting the Changelog for a > package I maintain (‘python-daemon’) to reStructuredText [0]. This has > all the above features: machine-parseable, a well-defined way to attach > fields to a section, etc. > > [0] reStructuredText: http://docutils.sourceforge.net/rst.html> > > I've now produced a small Python library which knows how to transform a > reST Changelog to package metadata; and how to get that package metadata > into and out of a Python distribution with Distutils. Sounds interesting. Where can we see this? > The result is that I will never again upload the package with a mismatch > between what the Changelog claims is the latest version, and what the > package metadata says. > > > This may be generally useful to others. I'm interested to know how > people would expect to use this. As an extension to Distutils? As a > third-party library? Something else? Offering it as a third-party library is a good start, but perhaps you could discuss this on the packaging mailing list and see whether they have any interest in it? https://mail.python.org/mailman/listinfo/distutils-sig/ -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Using a ChangeLog as a canonical source of package metadata (was: Announce: PyPrimes 0.2.1a)
On Fri, Jan 9, 2015 at 11:06 AM, Ben Finney wrote: > I've now produced a small Python library which knows how to transform a > reST Changelog to package metadata; and how to get that package metadata > into and out of a Python distribution with Distutils. > > The result is that I will never again upload the package with a mismatch > between what the Changelog claims is the latest version, and what the > package metadata says. > > > This may be generally useful to others. I'm interested to know how > people would expect to use this. As an extension to Distutils? As a > third-party library? Something else? I can't speak as a package maintainer (because I'm not one), but speaking as an end user, I'm in favour of anything that guarantees consistency like that. It's only occasionally an issue, but for instance, if I clone someone's source code repository and then install a package from there, it's not necessarily obvious from 'pip freeze' that I have something that can't so easily be downloaded. If the Changelog were guaranteed to show that, then I could easily see what's going on (eg if a new entry is created, immediately after the version release, adding an alpha version tag). ChrisA -- https://mail.python.org/mailman/listinfo/python-list
