On 19 April 2015 at 04:42, Nick Coghlan <[email protected]> wrote: > On 18 April 2015 at 21:31, Daniel Holth <[email protected]> wrote: >> Remember when distutils-sig didn't believe that Windows users actually >> lacked compilers?
:-) I remember the days before we had package_data, when projects had to abuse the data file support in setuptools to create $prefix/share, or $prefix/docs. Projects like moin and cx_Oracle did this, for example. It looked a mess on Windows installations :-) >> graphite-web and carbon are two examples of Python applications with >> lots of data files, that want to install config files into an obvious >> place so you can edit them and get the applications up and running. >> Because Python packaging has such terrible support for applications >> that are not libraries, it's unnecessarily hard to get these packages >> up and running -- especially if you want to put them in a virtualenv. Thanks. If I get the time, I'll have a look at those packages and see how their requirements might translate to Windows. > It's a fairly standard practice in Fedora/RHEL/CentOS to use setup.py > to define the build process in the RPM spec file, even if the package > itself is never distributed via the upstream package index (e.g. > beaker-project.org is built that way, as is pulpproject.org). > > Fedora's packaging policy for redistribution of upstream Python > projects also switched last year to favour using "pip install" in the > RPM build process over invoking "setup.py install" directly. > > Historically, all the "extra bits" needed to comply with FHS have > lived in the spec file, independently of the upstream packaging > system, requiring changes in two places for certain kinds of packaging > modifications, and frequently rendering the projects undeployable in a > virtual environment for no good reason. > > The benefit of Daniel's proposal is that it should make it feasible to > modify many of these projects to be virtualenv friendly, and then > *automate* the process of generating FHS policy compliant downstream > packages. That will be a big step towards "package for PyPI, get your > conda/Nix/Debian/Fedora packaging for free", so it feeds directly into > my own interests in streamlining the redistribution pipeline in > Fedora. OK, thanks. That clarifies why this is needed, but it does read to me as being something that projects should use sparingly, and only when they are facing the sort of Linux packaging issues that it's designed to solve. That's fine - my worry is that it'll be easy for projects to see it as the "normal" way to structure their data, and as a result we'll see a move away from package_data (which was introduced precisely to solve the sorts of problems that the Unix model of dedicated directories introduced in the context of Windows, zipimport and virtualenv). > From a Windows perspective, I believe this change mostly has the > potential to make services that were previously Linux-only solely for > packaging related reasons available on Windows as well. However, there > may also be an opportunity to better automate the process of > generating wix-based installers from PyPI packages (see > http://wixtoolset.org/documentation/manual/v3/howtos/files_and_registry/add_a_file.html) > rather than generating Windows installers directly (if I understand > the tooling correctly, introducing wix into that process should offer > the same kind of potential for better platform integration that > integrating with distro package managers offers on Linux). The biggest problem I see from a Windows POV is that installing files outside of the package data structure is actively hostile to tools like py2exe, and cx_Freeze, which are widely used for bundling standalone Windows applications written in Python. Such tools won't be able to find data installed via the proposed file category mechanism. As a possible compromise, how about an approach where on Linux system installs (or more accurately "those install schemes that are relevant to the distribution packaging issue you described") the file categories are installed into dedicated directories, as described. But on other installs (virtualenvs, Windows, maybe OSX) the file categories map to locations within package_data, so that they can be accessed via normal mechanisms like loader.get_data. Application code would need some support code to locate and read the files, but that's true whether we go for this proposal or an "outside of site-packages" scheme. Also, some things may even be better designated as "don't install" in certain schemes (man files, for example, are pretty much useless on Windows). Beyond this issue, I do have some concerns over the specific locations proposed, but they are better addressed as part of the PEP review process, once we have a specific proposal to review. Paul _______________________________________________ Distutils-SIG maillist - [email protected] https://mail.python.org/mailman/listinfo/distutils-sig
