On 06/12/2016 10:45 PM, Monty Taylor wrote: > On 06/11/2016 10:13 PM, Robert Collins wrote: >> On 12 June 2016 at 11:31, Thomas Goirand <z...@debian.org> wrote: >>> On 06/11/2016 05:02 AM, Robert Collins wrote: >> ... >>> Then keystone.conf ends up installed in /usr/etc/keystone when setup.py >>> install is used. With this configuration, it would be installed >>> correctly within a venv. >> >> Except that noone uses 'setup.py install' - *except* for rpm and dpkg >> packagers. Everyone else uses pip, and pip won't pass new options. >> Worse, pip creates wheels and then installs the wheels, and until PEP >> 491 [1] is finalised and implemented, wheels won't change their >> behaviour. >> >>> If instead we have: >>> [files] >>> data_files = >>> /etc/keystone/keystone.conf = etc/keystone.conf.sample >>> >>> [ notice the leading / for the destination... ] >>> >>> Then keystone.conf gets installed on the root of the filesystem instead >>> of properly be installed within the venv. Though it's installed properly >>> for distributions in /etc/keystone. >>> >>> To address this problem, Julien Danjou attempted to add a --sysconfdir >>> option in PBR. Though it was rejected by Robert, who wrote in the code >>> review that it should be addressed within setuptools or disutils. >> >> Not should. Must be. A local change to pbr won't solve the problem, >> except for the very narrow case of package creators (who already have >> excellent tools - its a single one-line file to fix it on Debian, and >> I presume roughly the same on RPM distributions). It certainly won't >> solve it for virtual envs. >> >>> If I got the above facts wrong, feel free to let everyone know. Now, my >>> reply to Robert... >>> === End of context === >>> >>> Robert, >>> >>> I do not agree that we *must* wait until politics are discussed and a >> >> I'm going to quote what I said back in February - 4 months ago: "Ok, >> so here's where I got to: >> >> - wheel doesn't support or understand /etc per se >> - setuptools and distutils disagree on the meaning of data files paths >> - wheel takes the distutils value today >> - pep 491, which seems moribund, is an attempt to explicitly support >> sysconf dir's like /etc, and thats what is needed in general. >> >> I think the way forward to get this available as a sensible, >> supportable feature is to discuss it on the distutils-sig list, and >> that community can help figure out a design, including the backwards >> compat implications." >> >> I don't -2 lightly, and I did considerable research into this patch >> before leaving the -2 in place. >> >> With regards to politics and consensus in the upstream packaging >> community - well, it would move faster if folk who are distro folk and >> thus want to see the mess sorted were collaborating there: its been 4 >> months since Julien's original patch and no discussion has taken >> place, nor has anyone volunteered in that community to do the work. >> Its not super hard, and I'd be happy to guide anyone interested in >> reviving PEP 491 (or drafting an alternative). What I'm not interested >> in doing is adding an ineffective hack to pbr which we'll have to >> maintain for an extended period, which isn't actually in pbr's domain >> (pbr exists to prevent duplication amongst OpenStack projects and >> provide solid and sane policy, not to replace wheel and setuptools). >> I'm sorry if that sounds like a hard line, but we had an ongoing >> series of bugs in pbr - the gift that keeps on giving - from the >> previous attempts to work around defects upstream of pbr rather than >> fixing those defects directly, which is a much better strategy : less >> cost, less technical debt, benefits the entire community, gets better >> outcomes. > > I'm going to have to second what Robert says here. > > Also - I would like to add the following. > > Having pip or setup.py install install a config file is at best a > strange thing to want, as it does not have the mechanisms to deal with > templating in values, or dealing with conflicts with locally changed > file values on upgrades. > > Consider the case where a person does "pip install keystone" to install > keystone. > > If that command installs keystone.conf into /etc/keystone - there is > _NO_ way that that config file will have any useful values. The user > will then edit the file to add the values into the file that need to be > added. Then, the user will, at a later point in time do "pip install -U > keystone" to upgrade the keystone installation. > > What does pip do? Does it overwrite /etc/keystone/keystone.conf? What > about the user's values? Does it NOT overwrite > /etc/keystone/keystone.conf? Why not? > > This is, btw, why dpkg and rpm and friends exist. It's their job. > > Now, you might argue, there are files, such as the paste config files > which are configs that the software needs to install to be useful. > > I will call hogwash on that. That we install files such as that which > are actually code has been a deployer nightmare since day one, and the > solution is not to make pip smarter, it's to stop installing such files. > > NOW - if it's quite literally a file that just exists better as a file > and not direct python code - that's fine - just put it into the package > itself and then load it either by relative paths from the package or via > pkg_resources. For instance, one could put paste.conf into > /usr/local/lib/python2.7/site-packages/keystone/paste.ini instead of > into etc. > > Which is my way of saying: > > - If it's a file that one expects the user to edit, having pip/setup.py > install it is a bad idea > - If it's a file that one does NOT expect the user to edit, put it into > the package. > > There is, of course, a class of files that meet neither condition: > manpages. Yup. Manpage handling is bong. But we already have as much > special casing in pbr as I think is reasonable to do the best we can > with manpages. That best is crappy anyway, which brings us back to > Robert's point about fixing things in distutils/setuptools. > > But seriously - figuring out _why_ we want pip to be able to deal with > config files I think would be much better served by reducing the amount > of boilerplate config we expect users to plop down. No amount of code in > pbr is going to help that problem. > > Monty
I proposed to use the --install-layout=deb parameter, so that *ONLY* distros would be affected. But this was also rejected. Could you explain why it would be a bad idea? Also, I fail to understand why it's ok to have "python setup.py install" to install things in etc/ in the case of a virtualenv, but you think it's forbidden to fix the silly /usr/etc path when called by packaging. Here, I'm not trying to add a feature, just address a bug. Because that's really what we are dealing with here (unless someone is insane enough to believe that installing stuff in /usr/etc is ok, somehow...). Also, Monty, I read you writing that nobody should ever attempt to use pbr to install config files in etc, not even using virtualenv. However, it's already a common practice in OpenStack. So like it or not, we should deal with it. As for the answer of Robert, yes, please explain to me how I can attempt to fix it in setuptools / pip / whatever. In the long run, I agree it should be nice, and that's the way to go. Though probably you're maybe being misslead when writing: On 06/12/2016 04:13 AM, Robert Collins wrote: > Except that noone uses 'setup.py install' - *except* for rpm and dpkg > packagers. Everyone else uses pip, and pip won't pass new options. In terms of actual code having to deal with this issue, the folks using pip count as 1 type of setup. Package maintainers count as 5 currently (Debian, Ubuntu, RDO, SuSE and Gento). It doesn't mater how many *users* these groups of packaging system have. So IMO, trying to fix the work of 5 type of programming vs a single one (even if that single one has more developers using it) isn't that silly. If on top, we're protecting it with --install-layout=deb, or reading anything else that would work for other distro (an env var?) then I'm sure it's going to be safe for everyone, and I do believe that this is the pragmatic thing to do, until upper level of tooling can be fixed. Your thoughts? Cheers, Thomas Goirand (zigo) __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev