Bug#311188: Draft proposal for handling configuration file manipulations in Debian blends
[re-adding bugreport to list of recipients] On 12-01-31 at 10:17pm, Mike Gabriel wrote: > Hi Andreas, hi Jonas, hi all, > > thanks for the reply!!! > > On Fr 27 Jan 2012 11:44:02 CET Andreas Tille wrote: > > >On Thu, Jan 26, 2012 at 04:27:40PM +0100, Jonas Smedegaard wrote: > > >>[...] > > >[...] > > >I also would like to add the following: Mike's suggestion needs some > >dpkg/apt/whatever coding first. It does not help to make good > >suggestions if you will not come up with patches which you tested for > >some time and than make the maintainers of this core functionality > >accepting these patches. This is not an easy job and according to my > >experience this takes ages. > > I am aware of this. Before starting to code comes enrolling people, > discussing possibilities, listening to what's already there, > listening to other people's ideas, etc. It does not really make > sense to start coding into the dark and finding out that it is not > at all the way to go. > > >I'm comparing with how long it took to make > >apt aware about description translations - and translations is a feature > >about 50% of all Debian users might really *want*. Unfortunately we > >need to realise that Blends is - like it or not - a quite unknown topic > >in the Debian universe even if I try my best to talk about it at any > >DebConf and other events. I like to quote Peter Eisentraut: "You are > >talking about something which does not exist." Well, it is not that > >drastical, but changing the Debian infrastructure on behalf of the needs > >of Blends is at current state not realistic. > > ACK. > > >However, if we are talking about #311188 I think what we could try to > >approach is making config issues of Blends RC critical and thus making > >the bugs we filed against those packages RC critical which in turn would > >enable us NMUing packages of maintainers which are not willing to help > >us otherwise. I know that's also not very nice but would solve the > >problem we are facing and is way more realistic to be solved until June > >(suggested freeze time). > > :-) /me likes this... However, I am rather not thinking about > wheezy, this is a short period. For wheezy the Debian Edu goal has > to be to release D-E wheezy with the first or second point release > of D-E wheezy. Apart from that I hear voices that want to change > over to using Git for D-E development (I am one of that voices). > > >>I am pretty sure that anyone interested in blending would be excited if > >>you invent/refine needed mechanisms for above two needs. ...if done > >>Policy compliant - which does *not* mean weaken Policy but (understand > >>reasons for and) obey Policy. > >> > >>I am less sure that anyone else will volunteer to do the work for you, > >>if that's what you are asking. Personally I would not, because I cannot > >>imagine such work bear fruit - i.e. become Debian Policy compliant. > > > >Perfectly correct. You just will not manage to convince somebody else > >to do the work for you. That's why I would suggest to find a way were > >you can do the work yourself more easy (just do an NMU). > > Should we not address this approach (blend bugs = RC critical bugs > -> make NMUing possible) on debian-devel ML? If you mean raise severity of bug#311188, then that bugreport belongs to Skolelinux, so if feel representative for Skolelinux and you consider the issue more severe than currently tagged then go ahead and change it. I've argued strongly in the past that is was more severe, but have been overruled by Debian release managers, and Petter Reinholdtsen also disagrees (see approx. 30min. into Debconf11 Skolelinux meeting video). If you mean raise severity of bugs underlying bug#311188 then feel free to try, but beware that they package maintainers of those packages have the last say. If you want to force raise severity despite the judgement of respective owners of bugreports, then you should contact the Technical Committee, not raise it at d-devel@ (but going down that route is not recommended). Having said all that, you need not raise severity in order to make NMUs. - Jonas -- * Jonas Smedegaard - idealist & Internet-arkitekt * Tlf.: +45 40843136 Website: http://dr.jones.dk/ [x] quote me freely [ ] ask before reusing [ ] keep private signature.asc Description: Digital signature
Bug#311188: Draft proposal for handling configuration file manipulations in Debian blends
On Donnerstag, 26. Januar 2012, Mike Gabriel wrote: > To address #311188 the latest approach that has been discussed is > enrolling the maintainers of all affected packages (and that can be > many) to add blend-customized debconf values to their packages so that > a clean preseeding of the package is possible. yes, filing bug against the "affected" packages, best with patches, that's the way to go. we know this for years. nobody is doing it, that's the problem. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#311188: Draft proposal for handling configuration file manipulations in Debian blends
On Thu, Jan 26, 2012 at 04:27:40PM +0100, Jonas Smedegaard wrote: > On 12-01-26 at 12:24pm, Mike Gabriel wrote: > > I am talking about each single Debian package. Not the whole system. > > And every package in this model can be in non-blended mode or in blend > > mode for _one_ blend. > > Thanks for the clarification. I just work down my backlog after traveling to Southport where we have the second Debian Med sprint and followed your interesting discussion. > > Example: if we install d-e-c, we have to tweak apache2 configuration. > > For this we would set apache2 into ,,edu'' blend mode. If apache2 is > > in ,,edu'' blend mode it cannot be set into ,,gis'' blend mode etc. > > You can unblend apache2 and blend it with some other blend only then. My opinion about having two (or more) Blends cooexisting on one machine we need to differentiate between practical application and theoretical implementation. I doubt that there is any *existing* machine which has a need to have a real need to install two or more Blends and its configuration files. This doubt is based on the fact that currently only Debian Edu provides specific configuration at all (which probably will change in the future). However, I would really love to see the chance to enable the peaceful coexistence if possible at all so in other words I think we should care for an implementation which enables the theoretical chance to install more than one Blend on one machine. > So if "edu" wants to tweak a single option and "gis" wants to tweak a > single _other_ option, those two tweaks cannot both be applied if "edu" > and "gis" both use your proposed mechanism? > > Seems bad design to make blends mutually exclusive at the core of the > blend mechanism. Sure it might be sensible for one blend to conflict > with another, but some blends might go well together, so should not > technically be hindered in doing so by the underlying mechanisms IMO. BTW, it might perfectly be the case that two Blends need the very same change in the config and can not coexist just because we found a mechanism which excludes two Blends on one machine. This would be an unnecessary restriction. > Makes sense for me that a blend maintains _what_ should be tweaked, but > _how_ it is tweaked is better maintained by the package maintainer than > blend maintainers or dpkg maintainers IMHO. ACK > When a blend includes full config files or scripted config tweaks then > the package _maintainers_ are kept out of the loop of _maintaining_ > those config choices, and the config options are not offered > individually by Debian, e.g. for tools like piuparts to regression test > in automated ways. > > I highly recommend to instead help make it easier for package > maintainers to implement and _maintain_ config handling. ACK > I believe that if it was easier to implement and maintain debconf > interfaces to config files, we would have less of a problem convincing > them to offer config options for the tweaks we need for various blends. > > Specifically I believe that work to integrate debconf with Config::Model > could help improve the situation. > > More on Config::Model here: http://wiki.debian.org/PackageConfigUpgrade I also would like to add the following: Mike's suggestion needs some dpkg/apt/whatever coding first. It does not help to make good suggestions if you will not come up with patches which you tested for some time and than make the maintainers of this core functionality accepting these patches. This is not an easy job and according to my experience this takes ages. I'm comparing with how long it took to make apt aware about description translations - and translations is a feature about 50% of all Debian users might really *want*. Unfortunately we need to realise that Blends is - like it or not - a quite unknown topic in the Debian universe even if I try my best to talk about it at any DebConf and other events. I like to quote Peter Eisentraut: "You are talking about something which does not exist." Well, it is not that drastical, but changing the Debian infrastructure on behalf of the needs of Blends is at current state not realistic. However, if we are talking about #311188 I think what we could try to approach is making config issues of Blends RC critical and thus making the bugs we filed against those packages RC critical which in turn would enable us NMUing packages of maintainers which are not willing to help us otherwise. I know that's also not very nice but would solve the problem we are facing and is way more realistic to be solved until June (suggested freeze time). > I am pretty sure that anyone interested in blending would be excited if > you invent/refine needed mechanisms for above two needs. ...if done > Policy compliant - which does *not* mean weaken Policy but (understand > reasons for and) obey Policy. > > I am less sure that anyone else will volunteer to do the work for you, > if that's what you are ask
Bug#311188: Draft proposal for handling configuration file manipulations in Debian blends
Hi, NB! Please do not cc me - I am subscribed to both d-blends@ and the bugreport. More info at http://www.debian.org/MailingLists/#codeofconduct On 12-01-26 at 12:24pm, Mike Gabriel wrote: > I am talking about each single Debian package. Not the whole system. > And every package in this model can be in non-blended mode or in blend > mode for _one_ blend. Thanks for the clarification. > Example: if we install d-e-c, we have to tweak apache2 configuration. > For this we would set apache2 into ,,edu'' blend mode. If apache2 is > in ,,edu'' blend mode it cannot be set into ,,gis'' blend mode etc. > You can unblend apache2 and blend it with some other blend only then. So if "edu" wants to tweak a single option and "gis" wants to tweak a single _other_ option, those two tweaks cannot both be applied if "edu" and "gis" both use your proposed mechanism? Seems bad design to make blends mutually exclusive at the core of the blend mechanism. Sure it might be sensible for one blend to conflict with another, but some blends might go well together, so should not technically be hindered in doing so by the underlying mechanisms IMO. Debian is about structuring choices implemented in upstream code, and blending is in a sense about reducing choices to a sensible minimum. Makes sense for me that a blend maintains _what_ should be tweaked, but _how_ it is tweaked is better maintained by the package maintainer than blend maintainers or dpkg maintainers IMHO. When a blend includes full config files or scripted config tweaks then the package _maintainers_ are kept out of the loop of _maintaining_ those config choices, and the config options are not offered individually by Debian, e.g. for tools like piuparts to regression test in automated ways. I highly recommend to instead help make it easier for package maintainers to implement and _maintain_ config handling. I believe that if it was easier to implement and maintain debconf interfaces to config files, we would have less of a problem convincing them to offer config options for the tweaks we need for various blends. Specifically I believe that work to integrate debconf with Config::Model could help improve the situation. More on Config::Model here: http://wiki.debian.org/PackageConfigUpgrade > >> * This blending process may do the following... > >> **of course, the below has to become a legal part of Debian now...** > >> - create copies of existing configuration file(s) .d folders in > >> > >> > >> /etc// -> /etc//.dpkg-native > >> /etc//.d -> /etc//.d.dpkg-native > >> > >> - divert the original configuration file and .d folder names to the > >> corresponding files/folders in the /etc/blend/edu namespace. > >> - tag the affected package (maybe in /var/lib/dpkg/info) as blended > > > >Above seems like the central piece of where we are stalled at the moment > >regarding nedding-different-config-than-package-offers: The way forward > >is not to legalize mechanisms currently violates Policy, but to work on > >refining said mechanisms to a point where the Debian community is > >convinced that it is sane to do, and _then_ document the fact in Policy. > > This sounds like a senible way of progressing on this. However, what > exactly do _you_ mean by ,,refining said mechanisms'' (not sure what > the said mechanisms are and how to refine those). I mean tweaking mechanisms like config.d and dpkg-divert listed above. > >I believe dpkg does not reliably support diverting conffiles. That > >particular piece can be worked on (or at least investigated and > >documented more clearly) independently from this grand plan. > > > The above is what you refer to as refining, I guess? So that would > be dpkg development then. Correct. dpkg-divert is a dpkg mechanism so most likely needs development to happen in close collaboration with maintainers of dpkg. > In Debian Edu we currently follow two strategies: > > 1) provide our (D-E) own+complete config file for some Debian package > 2) apply cfengine of D-E to some non-D-E config files in some Debian > package > > Both ways of tweaking config files should be managable with a proposed > model. Actually the first is pretty much alike to dpkg-divert and it > probably can be a wrapper around dpkg-divert that handles the blending > and unblending process. > > The second approach is rather creating a > ready-for-blending-with-cfengine-copy of the orginal config files, > move the original's out of the way (.dpkg-native), replace the copied > files by symlinks and then tweak the copied files with cfengine (or > puppet or ...). > > My basic question at this point becomes this: does such an approach > have a chance to be supported and refined by the people being > interested in blends, or do people who have to deal with blends deny > such an approach right away for reasons A-B-C-etc. It does not make > sense enrolling people outside the group with defini
Bug#311188: Draft proposal for handling configuration file manipulations in Debian blends
Hi Jonas, hi all, On Do 26 Jan 2012 11:58:55 CET Jonas Smedegaard wrote: So, my opinion is that without implementing the blending mechanism within Debian policy itself (and that is also: within dpkg itself), we may possibly stall here for longer. It is the other way around: Debian Policy only reflects what is already consensus in Debian, so changing policy requires to first create consensus and then - after the fact - document it in Debian Policy. Good point, thanks for giving this more light. So, my proposal is: * let Debian blends become a real element of the Debian package system * that is: implement in dpkg three options: - Option 1: --blend= - Option 2: --unblend - Option 3: --init-blend (or --native2blend or similar) What does the above mean? Is it flags tied to a source package or to a binary package or to a system? If the latter then I suspect that you may really mean APT, not DPKG. In other words, does it imply that only a single blend can be applied? I am talking about each single Debian package. Not the whole system. And every package in this model can be in non-blended mode or in blend mode for _one_ blend. Example: if we install d-e-c, we have to tweak apache2 configuration. For this we would set apache2 into ,,edu'' blend mode. If apache2 is in ,,edu'' blend mode it cannot be set into ,,gis'' blend mode etc. You can unblend apache2 and blend it with some other blend only then. If really you are trying to document debathena rebranded as blends then please say so. If so it also seems sensible to involve the developers of debathena - either by discussing with them first to understand why their package(s) live only outside of Debian, not (tried to become) official part of it, or invite them to this very discussion directly. The idea proposed here was first. Then I re-read #311188 and stumbled over Marius's posting and took a closer look. I was astonished by some similarities. And it was also clear that they are doing it outside of Debian and that they do the same stuff as d-e-c. The install package-A and then config-package-X and config-package X overrides files in package-A. * This blending process may do the following... **of course, the below has to become a legal part of Debian now...** - create copies of existing configuration file(s) .d folders in /etc// -> /etc//.dpkg-native /etc//.d -> /etc//.d.dpkg-native - divert the original configuration file and .d folder names to the corresponding files/folders in the /etc/blend/edu namespace. - tag the affected package (maybe in /var/lib/dpkg/info) as blended Above seems like the central piece of where we are stalled at the moment regarding nedding-different-config-than-package-offers: The way forward is not to legalize mechanisms currently violates Policy, but to work on refining said mechanisms to a point where the Debian community is convinced that it is sane to do, and _then_ document the fact in Policy. This sounds like a senible way of progressing on this. However, what exactly do _you_ mean by ,,refining said mechanisms'' (not sure what the said mechanisms are and how to refine those). I believe dpkg does not reliably support diverting conffiles. That particular piece can be worked on (or at least investigated and documented more clearly) independently from this grand plan. The above is what you refer to as refining, I guess? So that would be dpkg development then. * Alternatively, if the configuration files of a package shall not be replaced by d-e-c then we also find a dpkg --init-blend command call useful (or --native2blend or --clone-native2blend or ...). -> install a copy of the original package's config files from /etc/ -> /etc/blend/edu/ After this, configuration files provided by the package maintainer can be manipulated with cfengine (within /etc/blend/edu/, of course. Above seems to me as a reinvention of dpkg-divert. If you feel that is a sensible way forward (I don't) then it seems to me that you need not reach concensus for the whole grand plan in order to improve this piece: you can discuss that with dpkg developers directly. In Debian Edu we currently follow two strategies: 1) provide our (D-E) own+complete config file for some Debian package 2) apply cfengine of D-E to some non-D-E config files in some Debian package Both ways of tweaking config files should be managable with a proposed model. Actually the first is pretty much alike to dpkg-divert and it probably can be a wrapper around dpkg-divert that handles the blending and unblending process. The second approach is rather creating a ready-for-blending-with-cfengine-copy of the orginal config files, move the original's out of the way (.dpkg-native), replace the copied files by symlinks and then tweak the copied files with cfengine (or puppet or ...). My basic question at this point becomes th
Bug#311188: Draft proposal for handling configuration file manipulations in Debian blends
Hi Mike, On 12-01-26 at 11:13am, Mike Gabriel wrote: > To address #311188 the latest approach that has been discussed is > enrolling the maintainers of all affected packages (and that can be > many) to add blend-customized debconf values to their packages so that > a clean preseeding of the package is possible. Correct. Thanks for summarizing. [debathena comment snipped] > So, my opinion is that without implementing the blending mechanism > within Debian policy itself (and that is also: within dpkg itself), > we may possibly stall here for longer. It is the other way around: Debian Policy only reflects what is already consensus in Debian, so changing policy requires to first create consensus and then - after the fact - document it in Debian Policy. > So, my proposal is: > > * let Debian blends become a real element of the Debian package >system > > * that is: implement in dpkg three options: >- Option 1: --blend= >- Option 2: --unblend >- Option 3: --init-blend (or --native2blend or similar) What does the above mean? Is it flags tied to a source package or to a binary package or to a system? If the latter then I suspect that you may really mean APT, not DPKG. In other words, does it imply that only a single blend can be applied? If really you are trying to document debathena rebranded as blends then please say so. If so it also seems sensible to involve the developers of debathena - either by discussing with them first to understand why their package(s) live only outside of Debian, not (tried to become) official part of it, or invite them to this very discussion directly. > * This blending process may do the following... > **of course, the below has to become a legal part of Debian now...** >- create copies of existing configuration file(s) .d folders in > > > /etc// -> /etc//.dpkg-native > /etc//.d -> /etc//.d.dpkg-native > >- divert the original configuration file and .d folder names to the > corresponding files/folders in the /etc/blend/edu namespace. >- tag the affected package (maybe in /var/lib/dpkg/info) as blended Above seems like the central piece of where we are stalled at the moment regarding nedding-different-config-than-package-offers: The way forward is not to legalize mechanisms currently violates Policy, but to work on refining said mechanisms to a point where the Debian community is convinced that it is sane to do, and _then_ document the fact in Policy. I believe dpkg does not reliably support diverting conffiles. That particular piece can be worked on (or at least investigated and documented more clearly) independently from this grand plan. > * Alternatively, if the configuration files of a package shall not >be replaced by d-e-c then we also find a dpkg --init-blend > command call useful (or --native2blend or >--clone-native2blend or ...). -> install a copy of the original >package's config files from /etc/ -> >/etc/blend/edu/ After this, configuration files >provided by the package maintainer can be manipulated with cfengine >(within /etc/blend/edu/, of course. Above seems to me as a reinvention of dpkg-divert. If you feel that is a sensible way forward (I don't) then it seems to me that you need not reach concensus for the whole grand plan in order to improve this piece: you can discuss that with dpkg developers directly. Regards, - Jonas -- * Jonas Smedegaard - idealist & Internet-arkitekt * Tlf.: +45 40843136 Website: http://dr.jones.dk/ [x] quote me freely [ ] ask before reusing [ ] keep private signature.asc Description: Digital signature
Bug#311188: Draft proposal for handling configuration file manipulations in Debian blends
Hi all, after having worked with the Debian Edu team for a couple of months now I would like to make a proposal on how to address configuration file manipulation within Debian blends. For further reading on configuration file manipulation and Debian policy violation refer to bug #311188 in BTS: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=311188 To address #311188 the latest approach that has been discussed is enrolling the maintainers of all affected packages (and that can be many) to add blend-customized debconf values to their packages so that a clean preseeding of the package is possible. Only a short time ago Marius has asked for debathena as a means to legalize what blend packages like debian-edu-config are doing to config files of other packages. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=311188#294 I agree with Jonas that debathena also fiddles with other packages' config files and that this causes the same violation of the same Debian policy 10.7.4 as described in #311188. So, my opinion is that without implementing the blending mechanism within Debian policy itself (and that is also: within dpkg itself), we may possibly stall here for longer. So, my proposal is: * let Debian blends become a real element of the Debian package system * that is: implement in dpkg three options: - Option 1: --blend= - Option 2: --unblend - Option 3: --init-blend (or --native2blend or similar) * in FHS we provide/define blend namespaces in /etc - e.g. /etc/blend/edu - or /etc/blend/gis - ... * blend packages with configuration files (like debian-edu-config) will put their blended config files into /etc/blend/edu/ So on installation of a blend config package the following might happen. I will use the example of debian-edu-config (d-e-c) from here on... * every package that gets manipulated by d-e-c has to be in Pre-Depends of d-e-c. (I am aware of DDs not liking this and trying to avoid that whereever possible, maybe we find another approach here) * d-e-c installs its files targeted for /etc/* into /etc/blend/edu/* * on postinst every native Debian package that is affected by the d-e-c configuration override gets prepared/tagged by a - dpkg --blend=edu * This blending process may do the following... **of course, the below has to become a legal part of Debian now...** - create copies of existing configuration file(s) .d folders in /etc// -> /etc//.dpkg-native /etc//.d -> /etc//.d.dpkg-native - divert the original configuration file and .d folder names to the corresponding files/folders in the /etc/blend/edu namespace. - tag the affected package (maybe in /var/lib/dpkg/info) as blended * Alternatively, if the configuration files of a package shall not be replaced by d-e-c then we also find a dpkg --init-blend command call useful (or --native2blend or --clone-native2blend or ...). -> install a copy of the original package's config files from /etc/ -> /etc/blend/edu/ After this, configuration files provided by the package maintainer can be manipulated with cfengine (within /etc/blend/edu/, of course. * On package upgrades the dpkg system has to recognize if a package is in blend state or not. If it is in blend state, the dpkg system has to install new configuration files with .dpkg-native file suffix. * With such a mechanism the system can also easily be unblended (reverted to a vanilla/native Debian installation). -> dpkg --unblend Happy for feedback, Mike -- DAS-NETZWERKTEAM mike gabriel, dorfstr. 27, 24245 barmissen fon: +49 (4302) 281418, fax: +49 (4302) 281419 GnuPG Key ID 0xB588399B mail: mike.gabr...@das-netzwerkteam.de, http://das-netzwerkteam.de freeBusy: https://mail.das-netzwerkteam.de/freebusy/m.gabriel%40das-netzwerkteam.de.xfb pgpYUxdkjkRxo.pgp Description: Digitale PGP-Unterschrift
