Re: Configuration file handling (Re: [desktop] Unix configuration nightmare)
So I got to prototyping this, just to play around with it and see what worked. Then I was reading a thread on debian-mentors about Apache configuration, and I remembered how much of a pain that is currently. It occurred to me that a merging system could perhaps be used to make this sane. I'd like to think about this a bit and see if it affects any design decisions, so that this could be supported in the future at least. The basic idea would be for the package to modify a copy of the saved config file, rather than basing its modifications on the installed file. Example: - Package P wants to add a stanza to /etc/shared.conf - P asks for a copy of /etc/shared.conf to modify; we give it the installed copy - P makes its modifications and provides the resulting config file - We present the result as a proposed config file merge, as in the simple case discussed in previous messages in this thread - We store /etc/shared.conf as the last known revision for P - we apply the changes ...time passes... - A new version of package P comes along and wants to apply the same changes to /etc/shared.conf, not knowing if they are already present - P asks for a copy of /etc/shared.conf to modify, we give it the last known revision - P finds its changes already present, and no changes are made ...time passes, the admin decides he needs to change something that was added by P, and does so... - A new version of package P is installed. the above scenario repeats; it knows that it has already added its changes, without regard for whether they are actually present in the current /etc/shared.conf. The admin's changes are preserved, and disaster averted. ...time passes, and package P's needs change... - A new version of package P needs to make a different change to /etc/shared.conf, perhaps even one overlapping with the old change - P asks for the last known revision of /etc/shared.conf, applies its changes - We attempt to merge those changes into /etc/shared.conf. If they do not overlap with the admin's changes, then we present the result as a merge. If they do overlap, then a conflict results, and we resolve it as for the usual case. - We apply the changes requested by P to the last known revision, indicating that they have been resolved. Future releases of P find their changes already applied, and are satisfied. Similar actions would take place independently for each package which modifies the configuration file. Each package would be modifying its own view of the config file as it last saw it, and the changes merged into the installed version. Each new change made by the package only needs to be resolved once This scheme would provide for shared configuration files with the same user experience as configuration files which are owned by a single package. The only implementation detail is that we keep track of a separate revision of the config file for each package which modifies it. This does not seem like a high price to pay. This leads into a continuation of the question of how to store these files. Warn me now if using RCS in this application would bother you, because it's starting to look like it might be a good fit. -- - mdz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Configuration file handling (Re: [desktop] Unix configuration nightmare)
So I got to prototyping this, just to play around with it and see what worked. Then I was reading a thread on debian-mentors about Apache configuration, and I remembered how much of a pain that is currently. It occurred to me that a merging system could perhaps be used to make this sane. I'd like to think about this a bit and see if it affects any design decisions, so that this could be supported in the future at least. The basic idea would be for the package to modify a copy of the saved config file, rather than basing its modifications on the installed file. Example: - Package P wants to add a stanza to /etc/shared.conf - P asks for a copy of /etc/shared.conf to modify; we give it the installed copy - P makes its modifications and provides the resulting config file - We present the result as a proposed config file merge, as in the simple case discussed in previous messages in this thread - We store /etc/shared.conf as the last known revision for P - we apply the changes ...time passes... - A new version of package P comes along and wants to apply the same changes to /etc/shared.conf, not knowing if they are already present - P asks for a copy of /etc/shared.conf to modify, we give it the last known revision - P finds its changes already present, and no changes are made ...time passes, the admin decides he needs to change something that was added by P, and does so... - A new version of package P is installed. the above scenario repeats; it knows that it has already added its changes, without regard for whether they are actually present in the current /etc/shared.conf. The admin's changes are preserved, and disaster averted. ...time passes, and package P's needs change... - A new version of package P needs to make a different change to /etc/shared.conf, perhaps even one overlapping with the old change - P asks for the last known revision of /etc/shared.conf, applies its changes - We attempt to merge those changes into /etc/shared.conf. If they do not overlap with the admin's changes, then we present the result as a merge. If they do overlap, then a conflict results, and we resolve it as for the usual case. - We apply the changes requested by P to the last known revision, indicating that they have been resolved. Future releases of P find their changes already applied, and are satisfied. Similar actions would take place independently for each package which modifies the configuration file. Each package would be modifying its own view of the config file as it last saw it, and the changes merged into the installed version. Each new change made by the package only needs to be resolved once This scheme would provide for shared configuration files with the same user experience as configuration files which are owned by a single package. The only implementation detail is that we keep track of a separate revision of the config file for each package which modifies it. This does not seem like a high price to pay. This leads into a continuation of the question of how to store these files. Warn me now if using RCS in this application would bother you, because it's starting to look like it might be a good fit. -- - mdz
Re: Configuration file handling (Re: [desktop] Unix configuration nightmare)
On Mon, Oct 28, 2002 at 11:54:54PM -0500, Branden Robinson wrote: On Wed, Oct 23, 2002 at 08:10:43PM -0400, Matt Zimmerman wrote: I don't think that existing .config handling necessarily needs to change at this point, unless we want to provide a standard way to suppress all attempts at automatic configuration for a particular config file. Well, Joey Hess is of the opinion that manage this config file with debconf-style questions are Evil and Wrong. So, assuming I want to get with the Debconf orthodoxy, I would be changing my config scripts to eliminate this sort of prompting. I can't speak for joeyh, but I think his beef is that maintainers use this kind of question to be lazy about preserving changes. I'm guilty of this myself, because I do not want to parse some arbitrary config format using shell commands to seed debconf with the appropriate responses. We're all about preserving changes here, so the light of the One True Way should guide us to salvation. I was thinking of things more similar to --force-confnew/--force-confold than to manage this config file with debconf questions. In other maintainer scripts, we need to be able to say I have generated a configuration file /tmp/blah as a possible replacement for /etc/foo/bar. At that point, the maintainer script is done with the file, and passes control to us. Here we run into a problem: * For most packages, the other maintainer script is going to be the postinst. In fact it's difficult for me to imagine a scenario when anything but the postinst would be generating a config file.[1] From my perspective, the tool doesn't care from which maintainer script it's called. postinst would seem to be the only useful place to call it, I agree. * The postinst, by definition, runs during the configuration phase. Your proposal is going to pull us farther away from the utopia of being able to handle all interactivity before packages are even unpacked, a la dpkg-preconfigure. While dpkg's conffile prompts already violate this, they *can* be replaced with pre-unpack prompting, because dpkg-preconfigure can suck new conffiles out of a package just as well as it can config and templates files. Non-conffile config files cannot enjoy this luxury, because they don't exist within the package. Right. As we discussed on IRC, there seems to be a fundamental conflict between preconfiguration and generated configuration files because the package is by definition not configured yet (for the new version) at preconfiguration time. Packages with simple requirements could theoretically generate a configuration in .config, but that is not the right place for it, and packages with simple requirements can use conffiles. I had not considered that one day dpkg could prompt about conffiles ahead of time. That weakens my no worse than conffiles argument considerably. Preconfiguration should remain unchanged in (IMO) the two most important cases: initial installation, and novice user. In the former case, we will never need to prompt in postinst, and in the latter, we should have a reasonable default for them. * On the other hand, if we're doing an upgrade instead of an install, the tool(s) we use to generate the config file may already be on the system at pre-configure time. However, if those tools change, and a package's .config script needs to be able to use the config-generation tool that's in the corresponding version of the package, you'd need to have a way of declaring this requirement so that config file generation could be deferred to package configuration. Not to mention the fact that the runtime dependencies of the tools used to generate the configuration files would need to be present at pre-configure time. Oy vey. Yes, this would be a mess. Upgrades from one Debian release to the next would be fragile and difficult to maintain and test effectively. We check again whether the file has been modified since last time by comparing it to a saved copy or checksum (the copy is optional, but gives much more flexibility than storing only a checksum). Why not just a checksum? Do you have a specific application in mind, or do you think the copy is a good idea for the sake of people cleverer than we? :) The copy makes it possible to calculate the merge. Since I am pretty confident that we will want merges, I think we should go ahead and use the copy for the comparison as well. It might be marginally more efficient to cache checksums for the comparison, but I doubt it will be worth worrying about. [prompts for various cases] In the common cases, this should be possible with a single prompt, though it could be split into two phases or selected from either a simple or advanced method, or even suppressed entirely for novice users if a sane default action sequence could be decided (always preserve? merge, and if that fails, preserve? warn?). As you
Re: Configuration file handling (Re: [desktop] Unix configuration nightmare)
On Mon, Oct 28, 2002 at 11:54:54PM -0500, Branden Robinson wrote: On Wed, Oct 23, 2002 at 08:10:43PM -0400, Matt Zimmerman wrote: I don't think that existing .config handling necessarily needs to change at this point, unless we want to provide a standard way to suppress all attempts at automatic configuration for a particular config file. Well, Joey Hess is of the opinion that manage this config file with debconf-style questions are Evil and Wrong. So, assuming I want to get with the Debconf orthodoxy, I would be changing my config scripts to eliminate this sort of prompting. I can't speak for joeyh, but I think his beef is that maintainers use this kind of question to be lazy about preserving changes. I'm guilty of this myself, because I do not want to parse some arbitrary config format using shell commands to seed debconf with the appropriate responses. We're all about preserving changes here, so the light of the One True Way should guide us to salvation. I was thinking of things more similar to --force-confnew/--force-confold than to manage this config file with debconf questions. In other maintainer scripts, we need to be able to say I have generated a configuration file /tmp/blah as a possible replacement for /etc/foo/bar. At that point, the maintainer script is done with the file, and passes control to us. Here we run into a problem: * For most packages, the other maintainer script is going to be the postinst. In fact it's difficult for me to imagine a scenario when anything but the postinst would be generating a config file.[1] From my perspective, the tool doesn't care from which maintainer script it's called. postinst would seem to be the only useful place to call it, I agree. * The postinst, by definition, runs during the configuration phase. Your proposal is going to pull us farther away from the utopia of being able to handle all interactivity before packages are even unpacked, a la dpkg-preconfigure. While dpkg's conffile prompts already violate this, they *can* be replaced with pre-unpack prompting, because dpkg-preconfigure can suck new conffiles out of a package just as well as it can config and templates files. Non-conffile config files cannot enjoy this luxury, because they don't exist within the package. Right. As we discussed on IRC, there seems to be a fundamental conflict between preconfiguration and generated configuration files because the package is by definition not configured yet (for the new version) at preconfiguration time. Packages with simple requirements could theoretically generate a configuration in .config, but that is not the right place for it, and packages with simple requirements can use conffiles. I had not considered that one day dpkg could prompt about conffiles ahead of time. That weakens my no worse than conffiles argument considerably. Preconfiguration should remain unchanged in (IMO) the two most important cases: initial installation, and novice user. In the former case, we will never need to prompt in postinst, and in the latter, we should have a reasonable default for them. * On the other hand, if we're doing an upgrade instead of an install, the tool(s) we use to generate the config file may already be on the system at pre-configure time. However, if those tools change, and a package's .config script needs to be able to use the config-generation tool that's in the corresponding version of the package, you'd need to have a way of declaring this requirement so that config file generation could be deferred to package configuration. Not to mention the fact that the runtime dependencies of the tools used to generate the configuration files would need to be present at pre-configure time. Oy vey. Yes, this would be a mess. Upgrades from one Debian release to the next would be fragile and difficult to maintain and test effectively. We check again whether the file has been modified since last time by comparing it to a saved copy or checksum (the copy is optional, but gives much more flexibility than storing only a checksum). Why not just a checksum? Do you have a specific application in mind, or do you think the copy is a good idea for the sake of people cleverer than we? :) The copy makes it possible to calculate the merge. Since I am pretty confident that we will want merges, I think we should go ahead and use the copy for the comparison as well. It might be marginally more efficient to cache checksums for the comparison, but I doubt it will be worth worrying about. [prompts for various cases] In the common cases, this should be possible with a single prompt, though it could be split into two phases or selected from either a simple or advanced method, or even suppressed entirely for novice users if a sane default action sequence could be decided (always preserve? merge, and if that fails, preserve? warn?). As you
Re: [desktop] Unix configuration nightmare
Branden Robinson [EMAIL PROTECTED] wrote: [...] * hates and fears text config files and will not touch them * will read and edit a text config file But there is such a third group: * will edit a text config file, but only given very precise and explicit instructions -- will not read manpages, will not read the config file itself, has tunnel vision, will only take action on suggestions like right after the line that says Driver mga in your XFree config file[sic], add a line that says Option WhizzBang and you'll be able to use the special Tomb Raider hack that let you play Lara Croft without any clothes on, and adds lots of mirrors to the level maps ROTFL This one not only sent me rolling on the floor but managed to amuse my sister, too, although she'snot interested in computer stuff. Please keep it up. So, like I said, it's my fault. I didn't think many such people existed. They do, and because they are numerous it's my responsibility as a Debian package maintainer to accomodate their needs. What you want to do will still be possible, but you'll have to use interdiff or something. An easy thing will be made hard -- or at least tedious -- so that a different easy thing can be made even easier than it was, because even easy was too hard for some people. We had the same problem with debconfization of exim v4 but could not use your original workaround to allow user-modifications, because you cannot have sections two times in exim.conf. That's when I came up with the scheme of saving the results of the debconf-questions to a simple parseable variable=value file, and automatically generating the final configuration-file in /var/lib/exim4 from this file and the dpkg-conffile /etc/exim4/exim4.conf.template with a simple script update-exim4.conf(8). The template file is a normal exim.conf file with additional magic-strings that are replaced by the update script with the values from the debconf-results file. | hostlist relay_from_hosts = 127.0.0.1 : 1 : DEBCONFrelay_netsDEBCONF | qualify_domain = DEBCONFvisiblenameDEBCONF | acl_smtp_rcpt = acl_check_rcpt | never_users = root Of course the user can put a file without the magic-strings in /etc/exim4/exim4.conf.template and it'll be used unmodified. This is a variation of your original system with some added value: - correct debconf support, the file that is managed with debconf is simply parseable, so user modifications outside debconf are easy to preserve. - /etc/exim4/exim4.conf.template has dpkg conffile managment, user modifications are easy to preserve. - everthing in /etc/ can be edited. Of course this scheme is bound to fail if our little tomb-raider friend starts editing /var/lib/exim4/exim4.conf directly but I hope that he simply won't find it because it is hidden in a non-standard location. And I am not sure if this might work X too, XF86Config has a very complex structure. cu andreas -- Hey, da ist ein Ballonautomat auf der Toilette! Unofficial _Debian-packages_ of latest unstable _tin_ http://www.logic.univie.ac.at/~ametzler/debian/tin-snapshot/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: [desktop] Unix configuration nightmare
Branden Robinson [EMAIL PROTECTED] wrote: [...] * hates and fears text config files and will not touch them * will read and edit a text config file But there is such a third group: * will edit a text config file, but only given very precise and explicit instructions -- will not read manpages, will not read the config file itself, has tunnel vision, will only take action on suggestions like right after the line that says Driver mga in your XFree config file[sic], add a line that says Option WhizzBang and you'll be able to use the special Tomb Raider hack that let you play Lara Croft without any clothes on, and adds lots of mirrors to the level maps ROTFL This one not only sent me rolling on the floor but managed to amuse my sister, too, although she'snot interested in computer stuff. Please keep it up. So, like I said, it's my fault. I didn't think many such people existed. They do, and because they are numerous it's my responsibility as a Debian package maintainer to accomodate their needs. What you want to do will still be possible, but you'll have to use interdiff or something. An easy thing will be made hard -- or at least tedious -- so that a different easy thing can be made even easier than it was, because even easy was too hard for some people. We had the same problem with debconfization of exim v4 but could not use your original workaround to allow user-modifications, because you cannot have sections two times in exim.conf. That's when I came up with the scheme of saving the results of the debconf-questions to a simple parseable variable=value file, and automatically generating the final configuration-file in /var/lib/exim4 from this file and the dpkg-conffile /etc/exim4/exim4.conf.template with a simple script update-exim4.conf(8). The template file is a normal exim.conf file with additional magic-strings that are replaced by the update script with the values from the debconf-results file. | hostlist relay_from_hosts = 127.0.0.1 : 1 : DEBCONFrelay_netsDEBCONF | qualify_domain = DEBCONFvisiblenameDEBCONF | acl_smtp_rcpt = acl_check_rcpt | never_users = root Of course the user can put a file without the magic-strings in /etc/exim4/exim4.conf.template and it'll be used unmodified. This is a variation of your original system with some added value: - correct debconf support, the file that is managed with debconf is simply parseable, so user modifications outside debconf are easy to preserve. - /etc/exim4/exim4.conf.template has dpkg conffile managment, user modifications are easy to preserve. - everthing in /etc/ can be edited. Of course this scheme is bound to fail if our little tomb-raider friend starts editing /var/lib/exim4/exim4.conf directly but I hope that he simply won't find it because it is hidden in a non-standard location. And I am not sure if this might work X too, XF86Config has a very complex structure. cu andreas -- Hey, da ist ein Ballonautomat auf der Toilette! Unofficial _Debian-packages_ of latest unstable _tin_ http://www.logic.univie.ac.at/~ametzler/debian/tin-snapshot/
Re: Configuration file handling (Re: [desktop] Unix configuration nightmare)
On Wed, Oct 23, 2002 at 08:10:43PM -0400, Matt Zimmerman wrote: On Wed, Oct 23, 2002 at 02:02:12PM -0500, Branden Robinson wrote: Matt, I'd be more than happy to use xfree86 in unstable as a testbed for your proposal. If you agree, let's migrate this subthread over to debian-x. OK, let's spec things out a bit, then. I don't think that existing .config handling necessarily needs to change at this point, unless we want to provide a standard way to suppress all attempts at automatic configuration for a particular config file. Well, Joey Hess is of the opinion that manage this config file with debconf-style questions are Evil and Wrong. So, assuming I want to get with the Debconf orthodoxy, I would be changing my config scripts to eliminate this sort of prompting. In other maintainer scripts, we need to be able to say I have generated a configuration file /tmp/blah as a possible replacement for /etc/foo/bar. At that point, the maintainer script is done with the file, and passes control to us. Here we run into a problem: * For most packages, the other maintainer script is going to be the postinst. In fact it's difficult for me to imagine a scenario when anything but the postinst would be generating a config file.[1] * The postinst, by definition, runs during the configuration phase. Your proposal is going to pull us farther away from the utopia of being able to handle all interactivity before packages are even unpacked, a la dpkg-preconfigure. While dpkg's conffile prompts already violate this, they *can* be replaced with pre-unpack prompting, because dpkg-preconfigure can suck new conffiles out of a package just as well as it can config and templates files. Non-conffile config files cannot enjoy this luxury, because they don't exist within the package. * On the other hand, if we're doing an upgrade instead of an install, the tool(s) we use to generate the config file may already be on the system at pre-configure time. However, if those tools change, and a package's .config script needs to be able to use the config-generation tool that's in the corresponding version of the package, you'd need to have a way of declaring this requirement so that config file generation could be deferred to package configuration. Not to mention the fact that the runtime dependencies of the tools used to generate the configuration files would need to be present at pre-configure time. Oy vey. [Time passes.] Well, after a brief discussion on IRC with Matt and Adam Heath, the latter of the Debian Dpkg Development Team, I guess we might as well forge ahead with file update prompting in the postinst. Dpkg may or may not implement technologies to make our job easier. We'll find out debconf prompting as postinst time is felt by the users of XFree86 to be too high a price to pay for conffile-style handling of generated configuration files. We check again whether the file has been modified since last time by comparing it to a saved copy or checksum (the copy is optional, but gives much more flexibility than storing only a checksum). Why not just a checksum? Do you have a specific application in mind, or do you think the copy is a good idea for the sake of people cleverer than we? :) If it has not been modified, we overwrite the existing config with the new one, and update the saved copy to match. If the generated configuration file is identical to the existing one, then by some miracle, the user has made changes identical to what we would have made, and we update our saved copy of the config file and exit. If the files are different, then we attempt a merge, check whether it was successful, and interact with the user via debconf, explaining the situation and the result of the merge attempt (displaying a diff if the user cares about such things). At the end of the interaction, we should have decided on one of these courses of action: - Throw away the admin's changes to the file and replace it with the new one entirely (conffile 'y') - Ignore the package maintainer's changes to the file and keep the existing one (conffile 'n') - Merge the package maintainer's changes and give the user a chance to fix things up manually before continuing - (this option is only available if the merge was successful) Merge the package maintainer's changes and continue without further interaction In the common cases, this should be possible with a single prompt, though it could be split into two phases or selected from either a simple or advanced method, or even suppressed entirely for novice users if a sane default action sequence could be decided (always preserve? merge, and if that fails, preserve? warn?). As you said, this is really two different questions: 1) generated file only / existing file only / merge 2) if merge: prompt for review on problems / always prompt for review I think the latter is a prime candidate
Re: Configuration file handling (Re: [desktop] Unix configuration nightmare)
On Wed, Oct 23, 2002 at 08:10:43PM -0400, Matt Zimmerman wrote: On Wed, Oct 23, 2002 at 02:02:12PM -0500, Branden Robinson wrote: Matt, I'd be more than happy to use xfree86 in unstable as a testbed for your proposal. If you agree, let's migrate this subthread over to debian-x. OK, let's spec things out a bit, then. I don't think that existing .config handling necessarily needs to change at this point, unless we want to provide a standard way to suppress all attempts at automatic configuration for a particular config file. Well, Joey Hess is of the opinion that manage this config file with debconf-style questions are Evil and Wrong. So, assuming I want to get with the Debconf orthodoxy, I would be changing my config scripts to eliminate this sort of prompting. In other maintainer scripts, we need to be able to say I have generated a configuration file /tmp/blah as a possible replacement for /etc/foo/bar. At that point, the maintainer script is done with the file, and passes control to us. Here we run into a problem: * For most packages, the other maintainer script is going to be the postinst. In fact it's difficult for me to imagine a scenario when anything but the postinst would be generating a config file.[1] * The postinst, by definition, runs during the configuration phase. Your proposal is going to pull us farther away from the utopia of being able to handle all interactivity before packages are even unpacked, a la dpkg-preconfigure. While dpkg's conffile prompts already violate this, they *can* be replaced with pre-unpack prompting, because dpkg-preconfigure can suck new conffiles out of a package just as well as it can config and templates files. Non-conffile config files cannot enjoy this luxury, because they don't exist within the package. * On the other hand, if we're doing an upgrade instead of an install, the tool(s) we use to generate the config file may already be on the system at pre-configure time. However, if those tools change, and a package's .config script needs to be able to use the config-generation tool that's in the corresponding version of the package, you'd need to have a way of declaring this requirement so that config file generation could be deferred to package configuration. Not to mention the fact that the runtime dependencies of the tools used to generate the configuration files would need to be present at pre-configure time. Oy vey. [Time passes.] Well, after a brief discussion on IRC with Matt and Adam Heath, the latter of the Debian Dpkg Development Team, I guess we might as well forge ahead with file update prompting in the postinst. Dpkg may or may not implement technologies to make our job easier. We'll find out debconf prompting as postinst time is felt by the users of XFree86 to be too high a price to pay for conffile-style handling of generated configuration files. We check again whether the file has been modified since last time by comparing it to a saved copy or checksum (the copy is optional, but gives much more flexibility than storing only a checksum). Why not just a checksum? Do you have a specific application in mind, or do you think the copy is a good idea for the sake of people cleverer than we? :) If it has not been modified, we overwrite the existing config with the new one, and update the saved copy to match. If the generated configuration file is identical to the existing one, then by some miracle, the user has made changes identical to what we would have made, and we update our saved copy of the config file and exit. If the files are different, then we attempt a merge, check whether it was successful, and interact with the user via debconf, explaining the situation and the result of the merge attempt (displaying a diff if the user cares about such things). At the end of the interaction, we should have decided on one of these courses of action: - Throw away the admin's changes to the file and replace it with the new one entirely (conffile 'y') - Ignore the package maintainer's changes to the file and keep the existing one (conffile 'n') - Merge the package maintainer's changes and give the user a chance to fix things up manually before continuing - (this option is only available if the merge was successful) Merge the package maintainer's changes and continue without further interaction In the common cases, this should be possible with a single prompt, though it could be split into two phases or selected from either a simple or advanced method, or even suppressed entirely for novice users if a sane default action sequence could be decided (always preserve? merge, and if that fails, preserve? warn?). As you said, this is really two different questions: 1) generated file only / existing file only / merge 2) if merge: prompt for review on problems / always prompt for review I think the latter is a prime candidate
Re: [desktop] Unix configuration nightmare
They. Will. Not. Read. Maybe there will be somewhat less complains from those who don't read, if debconf stuff will calculate and store a checksum of autogenerated part of the file, and later don't touch it (or give extra warnings before touching) if the checksum does not match.
Re: [desktop] Unix configuration nightmare
On Thu, Oct 24, 2002 at 09:26:48AM -0400, Susan Kleinmann wrote: Hello, On Wed, Oct 23, 2002 at 05:16:23PM -0500, Branden Robinson wrote: You can tell people to put their shit outside the debconf area, but they won't. Here are two peripherally related suggestions that might help: 1. There is at least one other package that has a do-not-configure-here component to its file: grub. I wonder if it might be possible to find There is another one: masqmail (mine). I used xfree86 as an example. The main conf has these lines on top, copied from /etc/X11/XF86Config-4: ### BEGIN DEBCONF SECTION # Do not edit within this region if you want your changes to be preserved by # debconf. Instead, make changes after the ### END DEBCONF SECTION line. I have no idea how someone can miss those lines... some commonalities between the way grub tells users to keep their mits off of various sections, and the way the XF86Config file does it? Just using the same words or the same case or the same placement in the file for the warning might be a start. My point is that if there is a common set of characteristics in several programs that tell users to keep out, then a broader group of users might come to expect and know to look for do-not-touch sections. It would also help users if the utilities that manipulated files that were only partially user-changeable had common names, or some other common features. 2. It might be interesting to develop either colorized vim or emacs modes that were invoked when editing only-partially-editable files, such that the hands-off part appeared, say, in some distinct color. That's an idea. Greetings, Oliver -- DAM approval waiting time: 127 days. See http://nm.debian.org/nmstatus.php?email=oku%40masqmail.cx pgpqaV8z7ZRcS.pgp Description: PGP signature
Re: [desktop] Unix configuration nightmare
Hi, Branden == Branden Robinson [EMAIL PROTECTED] writes: Branden On Wed, Oct 23, 2002 at 10:09:58PM +0200, Andreas Metzler wrote: Time for a third opinion: I think your setup circumvents the problem (parsing XF86Config) _very_ nicely with little overhead. It allows me to customize any section I want while still letting debconf handle the rest. Basically I just have to copy it and move it outside of the debconf part. Branden Well, that was the use case I had in mind when I wrote Branden XFree86's debconf support, but judging by the dozens of Branden config files I've seen, that's not the use case in widest Branden deployment. Branden Cutting and pasting a block of text is Too Hard. Perhaps you should add that to the comment: that the unit to copy is a section; and any section may be copied at will. Branden The scenario you enjoy will die, because People Will Not Branden Read. It's also arguably a violation of way Debconf is Branden supposed to work (there's not supposed to be any such thing Branden as a debconf area, and for files that aren't as Branden potentially insanely complex as XF86Config, I agree), so I'm Branden not getting any support from the Orthodox Church of Debconf, Branden either. Perhaps the solution is to grab the area between the debconf markers, store a md5sum of that area, and act like dpkg does with conffiles: allow the user to accept, refuse, view diffs, of the section inside the debconf area. Once I get ucf cleaned up and insert the diff3 functionality in there, you can use ucf to really preserve user changes. manoj -- People will do tomorrow what they did today because that is what they did yesterday. Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/%7Esrivasta/ 1024R/C7261095 print CB D9 F4 12 68 07 E4 05 CC 2D 27 12 1D F5 E8 6E 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C
Re: [desktop] Unix configuration nightmare
They. Will. Not. Read. Maybe there will be somewhat less complains from those who don't read, if debconf stuff will calculate and store a checksum of autogenerated part of the file, and later don't touch it (or give extra warnings before touching) if the checksum does not match. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: [desktop] Unix configuration nightmare
Hi, On Thu, Oct 24, 2002 at 12:19:20PM -0500, Branden Robinson wrote: 2. It might be interesting to develop either colorized vim or emacs modes that were invoked when editing only-partially-editable files, such that the hands-off part appeared, say, in some distinct color. A nice idea, but my guess is that a lot of the people who struggle with concepts like put your changes before ### BEGIN DEBCONF AREA or after ### END DEBCONF AREA also aren't using Vim or Emacs. Well, as one of those who edit config files but don't grok the ### HANDS OFF HERE concept, maybe I should explain why I did it (and yes, now I've seen the error of my ways and lo and behold... it works just like Branden promises!). The reason is... it feels bad. You duplicate one of the sections in the debconf area, change it a little and put it up front. Two identical section headers in config files just go against every ingrained belief in fighting redundancy. Also, I was not so sure where to put my updated section.. before the DEBCONF area? After it? When would *my* section win over the debconf area section, and when would it lose? (Sure it's documented. But it's not... intuitive.) Taking a look at the other side, I must confess that just deactivating debconf support for the X config file feels bad, too (like an evil, though supported, hack). Luckily, I was bright enough not to complain about that weird ### BEGIN DEBCONF AREA stuff and almost settled with re-updating my X config file after every X update. Until I got a clue. You may now officially brand me as stupid, but maybe this relevation(sp?) helps design a better system/a bigger warning message/whatever. -Malte #8-) (Maybe a debianplanet vote on Did you know you can put your own config changes into /etc/X11/XF86Config-4? - No. What's this? No. I do this manually. No, it felt forbidden Yes. I'm Branden. could shed some additional light on this important issue... :)) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: [desktop] Unix configuration nightmare
On Thu, Oct 24, 2002 at 09:26:48AM -0400, Susan Kleinmann wrote: Here are two peripherally related suggestions that might help: 1. There is at least one other package that has a do-not-configure-here component to its file: grub. I wonder if it might be possible to find some commonalities between the way grub tells users to keep their mits off of various sections, and the way the XF86Config file does it? Just using the same words or the same case or the same placement in the file for the warning might be a start. My point is that if there is a common set of characteristics in several programs that tell users to keep out, then a broader group of users might come to expect and know to look for do-not-touch sections. It would also help users if the utilities that manipulated files that were only partially user-changeable had common names, or some other common features. Well, Matt Zimmerman has a proposal for some standardized debconfage that would basically duplicate the functionality of dpkg's conffile-update prompting, but for configuration files that aren't conffiles -- like XF86Config-4. His proposal is generalized, so I imagine it could be applied to any package that works this way. We'll pilot-test it with xserver-xfree86 and see how it goes. If, eventually, we can teach dpkg to use it for real conffiles as well, the last big hurdle to eliminating dpkg's dependency on an interactive terminal will have been removed, as I understand it. 2. It might be interesting to develop either colorized vim or emacs modes that were invoked when editing only-partially-editable files, such that the hands-off part appeared, say, in some distinct color. A nice idea, but my guess is that a lot of the people who struggle with concepts like put your changes before ### BEGIN DEBCONF AREA or after ### END DEBCONF AREA also aren't using Vim or Emacs. -- G. Branden Robinson|Any man who does not realize that Debian GNU/Linux |he is half an animal is only half a [EMAIL PROTECTED] |man. http://people.debian.org/~branden/ |-- Thornton Wilder msg04293/pgp0.pgp Description: PGP signature
Re: [desktop] Unix configuration nightmare
Hi, Branden == Branden Robinson [EMAIL PROTECTED] writes: Branden On Wed, Oct 23, 2002 at 10:09:58PM +0200, Andreas Metzler wrote: Time for a third opinion: I think your setup circumvents the problem (parsing XF86Config) _very_ nicely with little overhead. It allows me to customize any section I want while still letting debconf handle the rest. Basically I just have to copy it and move it outside of the debconf part. Branden Well, that was the use case I had in mind when I wrote Branden XFree86's debconf support, but judging by the dozens of Branden config files I've seen, that's not the use case in widest Branden deployment. Branden Cutting and pasting a block of text is Too Hard. Perhaps you should add that to the comment: that the unit to copy is a section; and any section may be copied at will. Branden The scenario you enjoy will die, because People Will Not Branden Read. It's also arguably a violation of way Debconf is Branden supposed to work (there's not supposed to be any such thing Branden as a debconf area, and for files that aren't as Branden potentially insanely complex as XF86Config, I agree), so I'm Branden not getting any support from the Orthodox Church of Debconf, Branden either. Perhaps the solution is to grab the area between the debconf markers, store a md5sum of that area, and act like dpkg does with conffiles: allow the user to accept, refuse, view diffs, of the section inside the debconf area. Once I get ucf cleaned up and insert the diff3 functionality in there, you can use ucf to really preserve user changes. manoj -- People will do tomorrow what they did today because that is what they did yesterday. Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/%7Esrivasta/ 1024R/C7261095 print CB D9 F4 12 68 07 E4 05 CC 2D 27 12 1D F5 E8 6E 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: [desktop] Unix configuration nightmare
On Tue, Oct 22, 2002 at 06:00:48PM -0400, Matt Zimmerman wrote: As I said, I am suggesting we mimick the conffile mechanism. conffiles are not parsed, but their modification is noticed. My proposed system would not prevent the user from using the menu-driven configuration; it would simple offer them a choice about whether to generate a new configuration using the dialogs, or to preserve their existing configuration. This choice would only be necessary if the file had been modified by hand. I think I would probably migrate the XFree86 packages to using this if you implemented it. The more I think about what it would require to completely handle an XF86Config file with Debconf, the less I want to do it. It feels like the wrong direction to be going. Some sections of the file can appear 0, 1 or n times. You'd have to build menus based on the identifiers for various sections and have the user select which one to use. This means I'd be generating questions from templates all over the place, and using the names of debconf questions as values in a select template (this fact could be masked, but it would be happening). It's just grody. I get two kinds of complaints about the debconfization of the X server: 1) I didn't bother to read the top of the config file, or any documentation anywhere, my changes got clobbered, and I'm mad. 2) You don't support this obscure-ass option, and I think you should. With what you propose it would be easier to hew back towards my original intent for the XFree86 debconf templates, which was: Get a working, non-gross, single-headed X configuration going if at all possible so that, with DEBIAN_PRIORITY=high, newbies don't even have to know that there is such a thin as an XFree86 configuration file. I am profoundly disinterested in a full solution. To do it right would be to reinvent xf86cfg, which has to load the driver modules itself to ask them what options they support. That's a horrible thing to be doing in an installation scenario. Especially when the damn package isn't even unpacked yet. Yes, now that I've written this mail, I've pretty much made up my mind. I like your idea. If the user dicks with the autogenerated file, we slam on the brakes and toss him into the manual configuration ghetto where he belongs. His changes are respected and he gets to RTF XF86Config-4(5) page. People have demonstrated to my satisfaction that: ### BEGIN DEBCONF SECTION # XF86Config-4 (XFree86 server configuration file) generated by dexconf, the # Debian X Configuration tool, using values from the debconf database. # # Edit this file with caution, and see the XF86Config-4 manual page. # (Type man XF86Config-4 at the shell prompt.) # # If you want your changes to this file preserved by dexconf, only make changes # before the ### BEGIN DEBCONF SECTION line above, and/or after the # ### END DEBCONF SECTION line below. # # To change things within the debconf section, run the command: # dpkg-reconfigure xserver-xfree86 # as root. Also see How do I add custom sections to a dexconf-generated # XF86Config or XF86Config-4 file? in /usr/share/doc/xfree86-common/FAQ.gz. is just a very long way of saying fnord. People's minds just blot it out and they insist that the text does not exist. They. Will. Not. Read. Matt, I'd be more than happy to use xfree86 in unstable as a testbed for your proposal. If you agree, let's migrate this subthread over to debian-x. -- G. Branden Robinson| Debian GNU/Linux | // // // / / [EMAIL PROTECTED] | EI 'AANIIGOO 'AHOOT'E http://people.debian.org/~branden/ | pgpM0XyJ22EVH.pgp Description: PGP signature
Re: [desktop] Unix configuration nightmare
On Wed, Oct 23, 2002 at 10:09:58PM +0200, Andreas Metzler wrote: Time for a third opinion: I think your setup circumvents the problem (parsing XF86Config) _very_ nicely with little overhead. It allows me to customize any section I want while still letting debconf handle the rest. Basically I just have to copy it and move it outside of the debconf part. Well, that was the use case I had in mind when I wrote XFree86's debconf support, but judging by the dozens of config files I've seen, that's not the use case in widest deployment. You can tell people to put their shit outside the debconf area, but they won't. They just tilt their heads a little, smile politely at you, and walk straight into the wall, again and again. Make the change you want, but do it over here, please. No. Make the change you want, but do it over here, please. No. Make the change you want, but do it over here, please. No. Make the change you want, but do it over here, please. No. Make the change you want, but do it over here, please. No. Make the change you want, but do it over here, please. No. [repeat until you vomit] Cutting and pasting a block of text is Too Hard. The scenario you enjoy will die, because People Will Not Read. It's also arguably a violation of way Debconf is supposed to work (there's not supposed to be any such thing as a debconf area, and for files that aren't as potentially insanely complex as XF86Config, I agree), so I'm not getting any support from the Orthodox Church of Debconf, either. I may sound bitter, and maybe I am a little, but I know where the fault lies; it lies with me, for misjudging my audience. I didn't think there was anything between the following two points on the user continuuum: * hates and fears text config files and will not touch them * will read and edit a text config file But there is such a third group: * will edit a text config file, but only given very precise and explicit instructions -- will not read manpages, will not read the config file itself, has tunnel vision, will only take action on suggestions like right after the line that says Driver mga in your XFree config file[sic], add a line that says Option WhizzBang and you'll be able to use the special Tomb Raider hack that let you play Lara Croft without any clothes on, and adds lots of mirrors to the level maps So, like I said, it's my fault. I didn't think many such people existed. They do, and because they are numerous it's my responsibility as a Debian package maintainer to accomodate their needs. What you want to do will still be possible, but you'll have to use interdiff or something. An easy thing will be made hard -- or at least tedious -- so that a different easy thing can be made even easier than it was, because even easy was too hard for some people. Thanks for the kind words, though. -- G. Branden Robinson| The software said it required Debian GNU/Linux | Windows 3.1 or better, so I [EMAIL PROTECTED] | installed Linux. http://people.debian.org/~branden/ | pgpTSn5kT5z3P.pgp Description: PGP signature
Configuration file handling (Re: [desktop] Unix configuration nightmare)
On Wed, Oct 23, 2002 at 02:02:12PM -0500, Branden Robinson wrote: Yes, now that I've written this mail, I've pretty much made up my mind. I like your idea. If the user dicks with the autogenerated file, we slam on the brakes and toss him into the manual configuration ghetto where he belongs. His changes are respected and he gets to RTF XF86Config-4(5) page. [...] Matt, I'd be more than happy to use xfree86 in unstable as a testbed for your proposal. If you agree, let's migrate this subthread over to debian-x. OK, let's spec things out a bit, then. I don't think that existing .config handling necessarily needs to change at this point, unless we want to provide a standard way to suppress all attempts at automatic configuration for a particular config file. In other maintainer scripts, we need to be able to say I have generated a configuration file /tmp/blah as a possible replacement for /etc/foo/bar. At that point, the maintainer script is done with the file, and passes control to us. We check again whether the file has been modified since last time by comparing it to a saved copy or checksum (the copy is optional, but gives much more flexibility than storing only a checksum). If it has not been modified, we overwrite the existing config with the new one, and update the saved copy to match. If the generated configuration file is identical to the existing one, then by some miracle, the user has made changes identical to what we would have made, and we update our saved copy of the config file and exit. If the files are different, then we attempt a merge, check whether it was successful, and interact with the user via debconf, explaining the situation and the result of the merge attempt (displaying a diff if the user cares about such things). At the end of the interaction, we should have decided on one of these courses of action: - Throw away the admin's changes to the file and replace it with the new one entirely (conffile 'y') - Ignore the package maintainer's changes to the file and keep the existing one (conffile 'n') - Merge the package maintainer's changes and give the user a chance to fix things up manually before continuing - (this option is only available if the merge was successful) Merge the package maintainer's changes and continue without further interaction In the common cases, this should be possible with a single prompt, though it could be split into two phases or selected from either a simple or advanced method, or even suppressed entirely for novice users if a sane default action sequence could be decided (always preserve? merge, and if that fails, preserve? warn?). At package build time, the source package machinery only needs to indicate which binary packages will be making use of the infrastructure, and a helper will ensure that the necessary templates are included and generate dependency substitutions. Rebuilding a package will automatically pull in the latest, best-translated templates from the helper. Open questions: - What should happen at preconfiguration time to minimize interaction for novice users? - What sort of preferences would be useful, either at a global scope or a per-package scope? always leave my foobar config alone always merge my changes if there are no merge conflicts Implementation issues: - How to store the saved config files, so that it is intuitively obvious to which package they belong, and their installed location, so that they are conveniently available to the admin? This should be a public interface. - Will it be necessary or desirable to modify debhelper so that there is a substitution interface for templates, similar to what is done for maintainer scripts? -- - mdz
Configuration file handling (Re: [desktop] Unix configuration nightmare)
On Wed, Oct 23, 2002 at 02:02:12PM -0500, Branden Robinson wrote: Yes, now that I've written this mail, I've pretty much made up my mind. I like your idea. If the user dicks with the autogenerated file, we slam on the brakes and toss him into the manual configuration ghetto where he belongs. His changes are respected and he gets to RTF XF86Config-4(5) page. [...] Matt, I'd be more than happy to use xfree86 in unstable as a testbed for your proposal. If you agree, let's migrate this subthread over to debian-x. OK, let's spec things out a bit, then. I don't think that existing .config handling necessarily needs to change at this point, unless we want to provide a standard way to suppress all attempts at automatic configuration for a particular config file. In other maintainer scripts, we need to be able to say I have generated a configuration file /tmp/blah as a possible replacement for /etc/foo/bar. At that point, the maintainer script is done with the file, and passes control to us. We check again whether the file has been modified since last time by comparing it to a saved copy or checksum (the copy is optional, but gives much more flexibility than storing only a checksum). If it has not been modified, we overwrite the existing config with the new one, and update the saved copy to match. If the generated configuration file is identical to the existing one, then by some miracle, the user has made changes identical to what we would have made, and we update our saved copy of the config file and exit. If the files are different, then we attempt a merge, check whether it was successful, and interact with the user via debconf, explaining the situation and the result of the merge attempt (displaying a diff if the user cares about such things). At the end of the interaction, we should have decided on one of these courses of action: - Throw away the admin's changes to the file and replace it with the new one entirely (conffile 'y') - Ignore the package maintainer's changes to the file and keep the existing one (conffile 'n') - Merge the package maintainer's changes and give the user a chance to fix things up manually before continuing - (this option is only available if the merge was successful) Merge the package maintainer's changes and continue without further interaction In the common cases, this should be possible with a single prompt, though it could be split into two phases or selected from either a simple or advanced method, or even suppressed entirely for novice users if a sane default action sequence could be decided (always preserve? merge, and if that fails, preserve? warn?). At package build time, the source package machinery only needs to indicate which binary packages will be making use of the infrastructure, and a helper will ensure that the necessary templates are included and generate dependency substitutions. Rebuilding a package will automatically pull in the latest, best-translated templates from the helper. Open questions: - What should happen at preconfiguration time to minimize interaction for novice users? - What sort of preferences would be useful, either at a global scope or a per-package scope? always leave my foobar config alone always merge my changes if there are no merge conflicts Implementation issues: - How to store the saved config files, so that it is intuitively obvious to which package they belong, and their installed location, so that they are conveniently available to the admin? This should be a public interface. - Will it be necessary or desirable to modify debhelper so that there is a substitution interface for templates, similar to what is done for maintainer scripts? -- - mdz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
