Re: Proposal to improve package configuration upgrades
Stefano Zacchiroli z...@debian.org writes: ... now it is only the two of us which needs to stop talking and start proposing patches as needed :-) Before patches, I've created a page on Debian wiki to better articulate the proposal: http://wiki.debian.org/PackageConfigUpgrade I'll update this page regularly. I'll follow the rough plan mentioned in this page. I'll send patches on this list as soon as I have something ready. All the best -- Dominique Dumont Delivering successful solutions requires giving people what they need, not what they want. Kurt Bittner irc: domidumont at irc.freenode.net ddumont at irc.debian.org -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Proposal to improve package configuration upgrades
Harald Braumann ha...@unheit.net writes: Agreed. But VCS solution is a 80% success/20% silent failure. Config::Model is a 80% success/20% abort. The latter should be easier to deal with for average user. But you don't need to silently merge (and thus silently fail in some cases). You can still ask the user about confirmation. Agreed. Some user will think before confirming, others will not. All the best -- Dominique Dumont Delivering successful solutions requires giving people what they need, not what they want. Kurt Bittner irc: domidumont at irc.freenode.net ddumont at irc.debian.org -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Proposal to improve package configuration upgrades
On Thu, Feb 26 2009, Harald Braumann wrote: But there are 3 possible situations here: 1. value has been changed between the last and the new maintainer version 2. value was modified locally 3. both of the above Well, a complete analysis of the situations ucf faces are at [0] and [1], and that discusses more initial states than 3 (the local file removed by user offers an interesting set of states). [2] is an essay into functional programming for ucf, but I wonder about the practicality. manoj 0: http://www.golden-gryphon.com/blog/manoj/blog/2009/02/24/Rethinking_ucf/ 1: http://www.golden-gryphon.com/blog/manoj/blog/2009/03/01/Rethinking_ucf_redux/ 2: http://www.golden-gryphon.com/blog/manoj/miscellaneous/functional_ucf/ -- Sex is hereditary. If your parents never had it, chances are you wont either. Joseph Fischer Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Proposal to improve package configuration upgrades
Stefano Zacchiroli z...@debian.org writes: I can agree, at least in theory. But as we all known, due to how source code tends to work, in 90% of the cases automatic merges do the right thing. Well, of course I cannot prove that number, but my personal feelings is that with a high confidence automatic merges do the right thing. I think your numbers are right. The main problem I see is that the automatic merge will not be able to inform the user whether the merge is correct or not. In case of merge failure, the application will exit on error and leave the average user in the dark. Even 10% of this kinf of failure will be badly perceived. You know, in the general case it is an undecidable problem, so I seriously doubt Config::Model can be the silver bullet. It's not as I already know that Config::Model cannot address *all* config files. Possibly you can get a good coverage of most of the files we have under /etc which have a trivial structure (hence the questions raised by other people: how many of those files in a typical installation you can cover?). Potentially, I'd say 90% of the files (very ballpark figure). But the configuration files need to be created. Config::Model is designed to reduce the work (and maintenance) work as the model are specifed in a data structure. This data structure can be created and maintained with a GUI (Config::Model::Itself). But then we are back at the issue of a 80-20 problem, and I see the VCS solution as more flexible and more readily available. Agreed. But VCS solution is a 80% success/20% silent failure. Config::Model is a 80% success/20% abort. The latter should be easier to deal with for average user. But again, it looks to me that the two approaches can coexist. Absolutely: Something like try Config::Model, if it fails (missing or incomplete model) may be VCS merge with mandatory user interaction or usual ucf question. ... now it is only the two of us which needs to stop talking and start proposing patches as needed :-) :-) For this I need a candidate package with a package maintainer willing to experiment the patch I might send... All the best -- Dominique Dumont Delivering successful solutions requires giving people what they need, not what they want. Kurt Bittner irc: domidumont at irc.freenode.net ddumont at irc.debian.org -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Proposal to improve package configuration upgrades
On Fri, Feb 27, 2009 at 01:35:56PM +0100, Dominique Dumont wrote:
I think your numbers are right. The main problem I see is that the
automatic merge will not be able to inform the user whether the
merge is correct or not. In case of merge failure, the application
will exit on error and leave the average user in the dark. Even 10%
of this kinf of failure will be badly perceived.
I agree with this argument, but I contend that it would be no
regression: currently, would the user know that after having chosen to
preserve the installed version of a conffile (instead of the
maintainer version) that configuration can be wrong wrt the new
version of the package being installed?
She wouldn't know, where is the difference?
(Yes, I know you are proposing an improvement over that situation, but
it is not for free, since it requires a per-conffile-kind
development. Mine is conffile-kind-independent and I believe it wont
introduce any regression.)
But then we are back at the issue of a 80-20 problem, and I see
the VCS solution as more flexible and more readily available.
Agreed. But VCS solution is a 80% success/20% silent
failure. Config::Model is a 80% success/20% abort. The latter should
be easier to deal with for average user.
With my former argument, that 20% of silent failures is possibly very
comparable with the silent failures enabled by the current status. I
stop before the temptation of repeating:
But again, it looks to me that the two approaches can coexist.
Absolutely: Something like try Config::Model, if it fails (missing
or incomplete model) may be VCS merge with mandatory user
interaction or usual ucf question.
:-)
Cheers.
--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
z...@{upsilon.cc,pps.jussieu.fr,debian.org} -- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...| ..: | Je dis tu à tous ceux que j'aime
signature.asc
Description: Digital signature
Re: Proposal to improve package configuration upgrades
On Fri, 27 Feb 2009 13:35:56 +0100 Dominique Dumont dominique.dum...@hp.com wrote: Stefano Zacchiroli z...@debian.org writes: But then we are back at the issue of a 80-20 problem, and I see the VCS solution as more flexible and more readily available. Agreed. But VCS solution is a 80% success/20% silent failure. Config::Model is a 80% success/20% abort. The latter should be easier to deal with for average user. But you don't need to silently merge (and thus silently fail in some cases). You can still ask the user about confirmation. harry signature.asc Description: PGP signature
Re: Proposal to improve package configuration upgrades
On Wed, 25 Feb 2009 17:32:08 +0100 Dominique Dumont dominique.dum...@hp.com wrote: Harald Braumann ha...@unheit.net writes: I don't really know Config::Model. But the main problem I have with the current system is, that I only see diffs between the currently installed version and the new package version. With ucf, you see a diff between current file (i.e. installed version with your modifications) and the new package version. That I also see without ucf. No what I really would like to see is the diff between the last version I've merged and the new package version. I fail to see the differences (no pun intented) between the last version I've merged and the current file ... I mean the last maintainer version I merged into the installed version, not the result of the last merge. So changes can easily be seen (changes in defaults, new/removed parameters or just white-space changes?) and merging would work without a conflict in most cases. With Config::Model: - you would not see white space or any other formatting difference - your customized values would be merged into the new package file (more accurately, loaded in Config::Model configuration tree using the new package *model*). Thus you would see the delta between your new file and the new default value. See this example of a screenshot [1] where the config values different from default are shown with a green arrow But there are 3 possible situations here: 1. value has been changed between the last and the new maintainer version 2. value was modified locally 3. both of the above In case 1 the modification could be merged silently, in case 2 the modification should be left alone and in case 3 I'd have to decide what to do. Config::Model on its own couldn't tell the difference. - yes, merging would work mostly without conflict Similar to like SCMs work. The main difference between a SCM and Config::Model is that a SCM works purely at text level without knowledge of the semantic of the file under control. OTOH, Config::Model uses the semantic knowledge provided by the model to perform the upgrade. You could apply the way merges work in an SCM to structured information. Config::Model could be useful in addition, but would it support such a work-flow? Provided I've understood correctly your work-flow, I'd say mostly yes... Having the diff between the last and the new maintainer version is not an alternative to Config::Model. The former can tell me what has changed in what way and the latter can present this information enriched with its semantic knowledge of the changes. Both are things I find useful. So my question is, if Config::Model can also deal with the additional information of what has changed how, so the two things could be combined? Cheers, harry signature.asc Description: PGP signature
Re: Proposal to improve package configuration upgrades
On Wed, 25 Feb 2009 12:08:00 -0600 Manoj Srivastava sriva...@debian.org wrote: Well. If the maintainer so desires, ucf does have this to say: ,[ Manual page ucf(1) ] | --three-way I thought I remembered seeing smth. like this. Seems like this is what is desired; Yes, this is exactly what is desired. however, the reason this is not on by default is that some configuration files can be huge, and ucf did not want to stash away _all_ the configuration files handled by it into /var. The exectation was that most developers with smallish configuration files would use --three-way, but that expectation was unfounded. Could this be made configurable locally? So the maintainer could still set whatever default he finds useful but it could be overridden by the admin? So everyone is happy. Cheers, harry signature.asc Description: PGP signature
Re: Proposal to improve package configuration upgrades
On Wed, Feb 25, 2009 at 06:03:03PM +0100, Dominique Dumont wrote:
Stefano Zacchiroli z...@debian.org writes:
Actually, this is something I've been pondering about for a
while. Having /etc under some VCS (as many of us, I presume, already
have by the means of etckeeper and similar tools), diff file merging
can be seriously improved.
I tend to disagree.
Well, it depends on how dpkg currently handles merges. My impression
(as a user, never looked at the actual code) is that it not even tries
to merge, it simply discovers that the local file is not pristine and
then asks the user. On the contrary, every VCS I'm aware of at least
_tries_ a merge, succeeding when changes do not affect the same
patch hunk.
Of course that would mean that dpkg should be made aware of the
difference between the last pristine configuration file installed on
the machine, and the configuration file the package being installed is
shipping.
Also, there is of course no guaranteed that no conflicting changes
lead to a configuration file semantically sound, but AFAIU you have no
guarantee of that with Config::Model either. They are both about
syntax only.
From a user point of view, you will get the same diff output whether a
SCM performs the diff or ucf performs the diff.
So your proposal will probably not help my mother-in-law to upgrade
the packages on her system ;-)
I disagree. It will not help your mother-in-law once she is faced with
the diff, but it will decrease dramatically the number of times she
will be faced with that. ... so maybe we should strive for both?
Cheers.
--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
z...@{upsilon.cc,pps.jussieu.fr,debian.org} -- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...| ..: | Je dis tu à tous ceux que j'aime
signature.asc
Description: Digital signature
Re: Proposal to improve package configuration upgrades
Stefano Zacchiroli z...@debian.org writes: Well, it depends on how dpkg currently handles merges. My impression (as a user, never looked at the actual code) is that it not even tries to merge, it simply discovers that the local file is not pristine and then asks the user. On the contrary, every VCS I'm aware of at least _tries_ a merge, succeeding when changes do not affect the same patch hunk. Agreed. Also, there is of course no guaranteed that no conflicting changes lead to a configuration file semantically sound, That's the main problem I see with VCS like merge. The main problem is that the merge result *should* be reviewed by the user. Will the user always have the skills to spot the places where the merge was wrong ? but AFAIU you have no guarantee of that with Config::Model either. They are both about syntax only. No Config::Model is all about checking the *semantic* of a configuration file. So you will have the guarantee that the resulting file is correct from the application point of view. From a user point of view, you will get the same diff output whether a SCM performs the diff or ucf performs the diff. So your proposal will probably not help my mother-in-law to upgrade the packages on her system ;-) I disagree. It will not help your mother-in-law once she is faced with the diff, but it will decrease dramatically the number of times she will be faced with that. ... so maybe we should strive for both? There's may be cases where the merge completes automatically but the end result is wrong. That would be really bad. All the best -- Dominique Dumont Delivering successful solutions requires giving people what they need, not what they want. Kurt Bittner irc: domidumont at irc.freenode.net ddumont at irc.debian.org -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Proposal to improve package configuration upgrades
Manoj Srivastava sriva...@debian.org writes: /etc/kernel-pkg.conf, for example, is in Perl. You may define functions, variables, closures (given enough make-kpkg-fu) and have it all work. Agreed. But this is valid for power user that would not really need the safe merge capability provided by Config::Model. I dare you to try one for sendmail.cf. (and yes, often don't use the new fangled m4 stuff) A complete model of sendmail.cf would also be required only for power users like you. For the mother-in-law use case, it would be more useful to validate that upgrades will work correctly for Config::Model enabled packages. Later on, the more packages use Config::Model, the easier will be the system maintenance of most common users. All the best -- Dominique Dumont Delivering successful solutions requires giving people what they need, not what they want. Kurt Bittner irc: domidumont at irc.freenode.net ddumont at irc.debian.org -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Proposal to improve package configuration upgrades
On Thu, Feb 26, 2009 at 01:25:33PM +0100, Dominique Dumont wrote:
Also, there is of course no guaranteed that no conflicting changes
lead to a configuration file semantically sound,
That's the main problem I see with VCS like merge. The main problem is
that the merge result *should* be reviewed by the user. Will the user
always have the skills to spot the places where the merge was wrong ?
I can agree, at least in theory. But as we all known, due to how
source code tends to work, in 90% of the cases automatic merges do the
right thing. Well, of course I cannot prove that number, but my
personal feelings is that with a high confidence automatic merges do
the right thing.
but AFAIU you have no guarantee of that with Config::Model
either. They are both about syntax only.
No Config::Model is all about checking the *semantic* of a
configuration file. So you will have the guarantee that the
resulting file is correct from the application point of view.
You know, in the general case it is an undecidable problem, so I
seriously doubt Config::Model can be the silver bullet. Possibly you
can get a good coverage of most of the files we have under /etc which
have a trivial structure (hence the questions raised by other people:
how many of those files in a typical installation you can cover?). But
then we are back at the issue of a 80-20 problem, and I see the VCS
solution as more flexible and more readily available. But again, it
looks to me that the two approaches can coexist.
... now it is only the two of us which needs to stop talking and start
proposing patches as needed :-)
Cheers.
--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
z...@{upsilon.cc,pps.jussieu.fr,debian.org} -- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...| ..: | Je dis tu à tous ceux que j'aime
signature.asc
Description: Digital signature
Re: Proposal to improve package configuration upgrades
Harald Braumann ha...@unheit.net writes: I fail to see the differences (no pun intented) between the last version I've merged and the current file ... I mean the last maintainer version I merged into the installed version, not the result of the last merge. ok But there are 3 possible situations here: 1. value has been changed between the last and the new maintainer version 2. value was modified locally 3. both of the above In case 1 the modification could be merged silently, in case 2 the modification should be left alone and in case 3 I'd have to decide what to do. Config::Model on its own couldn't tell the difference. Currently a maintainer set some value in a package config file to reflect his decisions or debian policy or whatever. With Config::Model such decision would not be encoded in the delivered config file but in the model of the configuration file. This would be specified in the configuration model as a default value. Here's what will happen during an upgrade: a. Config::Model will dump customized value somewhere (dump values which are different from maintainer's model version N. i.e. this will keep track of values from case 2 and 3 *if* they are different from the default value specified in the old model version N ). b. application model is upgraded from N to N+1 (case: 1 and 3: some default config values are changed) c. Config::Model performs the merge: * case 1: no customized value stored in step a. Config::Model will write the default value (specified in the new model version N+1) in the config file * case 2: customized value stored in step a. is loaded, checked and will be written in configuration file * case 3: just like case 2 I hope this replies to your concerns. If not feel free to ask more questions. - yes, merging would work mostly without conflict Similar to like SCMs work. The main difference between a SCM and Config::Model is that a SCM works purely at text level without knowledge of the semantic of the file under control. OTOH, Config::Model uses the semantic knowledge provided by the model to perform the upgrade. You could apply the way merges work in an SCM to structured information. More often than not, the order of the configuration data is not important. For SCM merge, this order *is* important. Having the diff between the last and the new maintainer version is not an alternative to Config::Model. The former can tell me what has changed in what way and the latter can present this information enriched with its semantic knowledge of the changes. Both are things I find useful. So my question is, if Config::Model can also deal with the additional information of what has changed how, so the two things could be combined? Config::Model can only show the diff between the current state (current file or file + modifications) and the default values from the configuration model. There's no notion of history or diff with previous version of a configuration file. Well, not yet. I'll have to think about this... All the best -- Dominique Dumont Delivering successful solutions requires giving people what they need, not what they want. Kurt Bittner irc: domidumont at irc.freenode.net ddumont at irc.debian.org -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Proposal to improve package configuration upgrades
hiya zack, On Thu, Feb 26, 2009 at 09:14:23AM +0100, Stefano Zacchiroli wrote: Well, it depends on how dpkg currently handles merges. My impression (as a user, never looked at the actual code) is that it not even tries to merge, it simply discovers that the local file is not pristine and then asks the user. On the contrary, every VCS I'm aware of at least _tries_ a merge, succeeding when changes do not affect the same patch hunk. it can't do a merge currently, because it doesn't have the common ancestor available to do a 3-way diff. Of course that would mean that dpkg should be made aware of the difference between the last pristine configuration file installed on the machine, and the configuration file the package being installed is shipping. if you go about a year back or so in the dpkg-mailing lists/bts (or look in the debian wiki, there's references there somewhere) you'll see some stuff i proposed--including a working patch. unfortunately this never seemed to amount to much and didn't keep the interest of the relevant dpkg maintainer. sean -- signature.asc Description: Digital signature
Re: Proposal to improve package configuration upgrades
On Thu, Feb 26, 2009 at 07:07:30PM +0100, sean finney wrote:
On Thu, Feb 26, 2009 at 09:14:23AM +0100, Stefano Zacchiroli wrote:
Well, it depends on how dpkg currently handles merges. My impression
(as a user, never looked at the actual code) is that it not even tries
to merge, it simply discovers that the local file is not pristine and
then asks the user. On the contrary, every VCS I'm aware of at least
_tries_ a merge, succeeding when changes do not affect the same
patch hunk.
it can't do a merge currently, because it doesn't have the common
ancestor available to do a 3-way diff.
Yes, that's clear. In fact my idea (actually, more of a flux of
thoughts) was to externalize that information, as in:
- dpkg discovers a change (and it can do that using plain old
checksums as it currently does)
- it checks whether there is an hook registered to do that
- if so, it invokes it by passing (API totally approximated) the
path of the current conffile, the version of the package owning it
which is being removed, the path of the new file
- it is now up to the hook to be able to retrieve the old file, which
a VCS in someway can be able to do [*]
then you have different ways for the hook to return, such as:
- 0: yay, everything merged - continue without bothering the user
- 1: don't know what to do (e.g., I cannot retrieve the old file)
- fallback to the current behavior or try some other hook
- 2: merge failure - offer the change to revert and fallback, or
maybe to examine the conflict as we usually do with VCSs
[*] Now that you make me think about it however, it would be perfectly
plausible to request dpkg to store under /var/lib/dpkg/info/ all
pristine copies of configuration files, instead of only their
checksums (I doubt the space consumption would be significant). If
that is acceptable, you can pass the hook the actual path to the
old pristine conffile without requesting the hook to be able to
retrieve it. YMMV.
Of course that would mean that dpkg should be made aware of the
difference between the last pristine configuration file installed
on the machine, and the configuration file the package being
installed is shipping.
if you go about a year back or so in the dpkg-mailing lists/bts (or
look in the debian wiki, there's references there somewhere) you'll
see some stuff i proposed--including a working patch. unfortunately
this never seemed to amount to much and didn't keep the interest of
the relevant dpkg maintainer.
Can you please give some tiny teeny details about what strategy did
you implement? In this thread, and from your mail, I don't get yet
which approach you could possibly have taken ..., but thanks for the
heads up!
Cheers.
--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
z...@{upsilon.cc,pps.jussieu.fr,debian.org} -- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...| ..: | Je dis tu à tous ceux que j'aime
signature.asc
Description: Digital signature
Re: Proposal to improve package configuration upgrades
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Wed, Feb 25, 2009 at 09:28:52AM +0100, Dominique Dumont wrote: Of course, there's no miracle. For the merge to work automatically and the result to be valid, the semantic of the configuration file must be known by Config::Model. This is done by describing the structure and constraints of the configuration file in a model (hence the Config::Model name). Do Config::Model support migration from one model to another? Example: CUPS 2.x has boolean option foo, which is changed in CUPS 3.x to numeric option foobar. - Jonas P.S. Please cc me on responses, I am not subscribed to -devel - -- * Jonas Smedegaard - idealist og Internet-arkitekt * Tlf.: +45 40843136 Website: http://dr.jones.dk/ [x] quote me freely [ ] ask before reusing [ ] keep private -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAkmlHtcACgkQn7DbMsAkQLjcSACfSHHm6+wMFVanffFsDWr9brsl 1gkAoJRQWvyvmGEuRxA7aC3iqhkHfIsc =V7EK -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Proposal to improve package configuration upgrades
Jonas Smedegaard d...@jones.dk writes: Do Config::Model support migration from one model to another? Yes. In fact model version n can include specific attribute to deal with migration from n-1 to n. Example: CUPS 2.x has boolean option foo, which is changed in CUPS 3.x to numeric option foobar. In such a case, CUPS3.x model would need to speficy that: - boolean option foo is status deprecated (which means the value can be managed, but the option foo is hidden from the user in the interfaces) - the default value of foobar can be computed from foo value (using compute attribute of Value object. See [1] for gory details) More complex value migration scenarios are possible. All the best [1] http://search.cpan.org/dist/Config-Model/lib/Config/Model/ValueComputer.pm -- Dominique Dumont Delivering successful solutions requires giving people what they need, not what they want. Kurt Bittner irc: domidumont at irc.freenode.net ddumont at irc.debian.org -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Proposal to improve package configuration upgrades
On Wed, 25 Feb 2009 09:28:52 +0100 Dominique Dumont dominique.dum...@hp.com wrote: Of course, there's no miracle. For the merge to work automatically and the result to be valid, the semantic of the configuration file must be known by Config::Model. This is done by describing the structure and constraints of the configuration file in a model (hence the Config::Model name). What do you think about this ? I don't really know Config::Model. But the main problem I have with the current system is, that I only see diffs between the currently installed version and the new package version. No what I really would like to see is the diff between the last version I've merged and the new package version. So changes can easily be seen (changes in defaults, new/removed parameters or just white-space changes?) and merging would work without a conflict in most cases. Similar to like SCMs work. Config::Model could be useful in addition, but would it support such a work-flow? Cheers, harry signature.asc Description: PGP signature
Re: Proposal to improve package configuration upgrades
Harald Braumann ha...@unheit.net writes: I don't really know Config::Model. But the main problem I have with the current system is, that I only see diffs between the currently installed version and the new package version. With ucf, you see a diff between current file (i.e. installed version with your modifications) and the new package version. No what I really would like to see is the diff between the last version I've merged and the new package version. I fail to see the differences (no pun intented) between the last version I've merged and the current file ... So changes can easily be seen (changes in defaults, new/removed parameters or just white-space changes?) and merging would work without a conflict in most cases. With Config::Model: - you would not see white space or any other formatting difference - your customized values would be merged into the new package file (more accurately, loaded in Config::Model configuration tree using the new package *model*). Thus you would see the delta between your new file and the new default value. See this example of a screenshot [1] where the config values different from default are shown with a green arrow - yes, merging would work mostly without conflict Similar to like SCMs work. The main difference between a SCM and Config::Model is that a SCM works purely at text level without knowledge of the semantic of the file under control. OTOH, Config::Model uses the semantic knowledge provided by the model to perform the upgrade. Config::Model could be useful in addition, but would it support such a work-flow? Provided I've understood correctly your work-flow, I'd say mostly yes... All the best [1] http://freshmeat.net/screenshots/69123/74589/ -- Dominique Dumont Delivering successful solutions requires giving people what they need, not what they want. Kurt Bittner irc: domidumont at irc.freenode.net ddumont at irc.debian.org -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Proposal to improve package configuration upgrades
On Wed, Feb 25, 2009 at 05:15:52PM +0100, Harald Braumann wrote:
No what I really would like to see is the diff between the last version
I've merged and the new package version. So changes can easily be seen
(changes in defaults, new/removed parameters or just white-space
changes?) and merging would work without a conflict in most cases.
Similar to like SCMs work.
Actually, this is something I've been pondering about for a
while. Having /etc under some VCS (as many of us, I presume, already
have by the means of etckeeper and similar tools), diff file merging
can be seriously improved.
The point is that we of course do not want neither to add the
dependency on some VCS into dpkg, nor to have dpkg making the choice
of which VCS.
So, it looks like that what we need is a pluggable mechanism into
dpkg, which dpkg will invoke when configuration file change conflicts
are encountered. A usual /etc/*.d/ place where to stick some script
with a well-defined API or something like that ...
Would something like that be thinkable of or am I totally on crack?
Cheers.
--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
z...@{upsilon.cc,pps.jussieu.fr,debian.org} -- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...| ..: | Je dis tu à tous ceux que j'aime
signature.asc
Description: Digital signature
Re: Proposal to improve package configuration upgrades
On Wed, Feb 25 2009, Dominique Dumont wrote: So I was thinking that this is a typical case where the upgrade could be smoothly handled by Config::Model. Of course, there's no miracle. For the merge to work automatically and the result to be valid, the semantic of the configuration file must be known by Config::Model. This is done by describing the structure and constraints of the configuration file in a model (hence the Config::Model name). Do we have an idea of how many configuration files can be described in terms of such a model? (I generally tend to code configuration files in a scripting language if the code is written in a scripting language). While I suspect a large number of our configuration files are simple, I fear that a significan chunk of them are fairly complex; and possibly not amenable to being described in terms of a non-trivial model. manoj -- Drop the vase and it will become a Ming of the past. The Adventurer Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Proposal to improve package configuration upgrades
Stefano Zacchiroli z...@debian.org writes: Actually, this is something I've been pondering about for a while. Having /etc under some VCS (as many of us, I presume, already have by the means of etckeeper and similar tools), diff file merging can be seriously improved. I tend to disagree. From a user point of view, you will get the same diff output whether a SCM performs the diff or ucf performs the diff. So your proposal will probably not help my mother-in-law to upgrade the packages on her system ;-) For seasoned sys admin, storing /etc/ files under a SCM will give a much better roll-back capability if a config is screwed up by a manual merge. All the best -- Dominique Dumont Delivering successful solutions requires giving people what they need, not what they want. Kurt Bittner irc: domidumont at irc.freenode.net ddumont at irc.debian.org -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Proposal to improve package configuration upgrades
Manoj Srivastava sriva...@debian.org writes: Do we have an idea of how many configuration files can be described in terms of such a model? I do not know how many. I'd say most of the files that do not use variables. For instance exim config is out. I do not know for Apache config files. So far, I've created models for OpenSsh [1] (quite easy) and Xorg [2] (more challenging and the model is still not complete) You will be able to try yourself OpenSsh editor as soon as libconfig-model-openssh-perl is accepted by ftp-masters. (I generally tend to code configuration files in a scripting language if the code is written in a scripting language). Uh ? While I suspect a large number of our configuration files are simple, I fear that a significan chunk of them are fairly complex; and possibly not amenable to being described in terms of a non-trivial model. Agreed. We may need to use hybrid solution for the most complex configuration files. Something like exim-like template + Config::Model for the template variables. All the best [1] http://cpansearch.perl.org/src/DDUMONT/Config-Model-OpenSsh-1.203/lib/Config/Model/models/ [2] http://cpansearch.perl.org/src/DDUMONT/Config-Model-Xorg-0.513/lib/Config/Model/models/ -- Dominique Dumont Delivering successful solutions requires giving people what they need, not what they want. Kurt Bittner irc: domidumont at irc.freenode.net ddumont at irc.debian.org -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Proposal to improve package configuration upgrades
On Wed, Feb 25 2009, Dominique Dumont wrote: Harald Braumann ha...@unheit.net writes: I don't really know Config::Model. But the main problem I have with the current system is, that I only see diffs between the currently installed version and the new package version. With ucf, you see a diff between current file (i.e. installed version with your modifications) and the new package version. No what I really would like to see is the diff between the last version I've merged and the new package version. I fail to see the differences (no pun intented) between the last version I've merged and the current file ... Well. If the maintainer so desires, ucf does have this to say: ,[ Manual page ucf(1) ] | --three-way |This turns on the option, during installation, for the user to |be offered a chance to see a merge of the changes between old |maintainer version and the new maintainer version into the |local copy of the configuration file. If the user likes what |they see, they can ask to have these changes merged in. This |allows one to get new upstream changes merged in even while |retaining local modifications to the configuration file. This |is accomplished by taking the configuration file and stashing |it in a cache area during registration, and using diff3 during |the install (the stashed file name is a munged version of the |full path of the configuration file to avoid name space |clashes). Note This option appeared in Version 0.8 of ucf, |which was the first version released into unstable and |ultimately Sarge. The version of ucf in woody does not contain |this option. ` Seems like this is what is desired; however, the reason this is not on by default is that some configuration files can be huge, and ucf did not want to stash away _all_ the configuration files handled by it into /var. The exectation was that most developers with smallish configuration files would use --three-way, but that expectation was unfounded. manoj -- In the stairway of life, you'd best take the elevator. Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Proposal to improve package configuration upgrades
On Wed, Feb 25 2009, Dominique Dumont wrote: Manoj Srivastava sriva...@debian.org writes: (I generally tend to code configuration files in a scripting language if the code is written in a scripting language). Uh ? /etc/kernel-pkg.conf, for example, is in Perl. You may define functions, variables, closures (given enough make-kpkg-fu) and have it all work. While I suspect a large number of our configuration files are simple, I fear that a significan chunk of them are fairly complex; and possibly not amenable to being described in terms of a non-trivial model. Agreed. We may need to use hybrid solution for the most complex configuration files. Something like exim-like template + Config::Model for the template variables. I dare you to try one for sendmail.cf. (and yes, often don't use the new fangled m4 stuff) manoj -- No one can forbid us the future. Inscription on the base of Paris's monument to Leon Gambetta Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
