Re: Does iproute2 moving config files to /usr/lib violate section 10.7.2?

[Simon McVittie]

On Sun, 17 Sep 2023 at 02:03:52 +0200, Santiago Vila wrote:
> If the files are intended to be modified "in place" by the
> system admin, we call them configuration files (and we try hard
> to put them in /etc). If they are not intended to be modified by
> the system admin, we don't call them configuration files (and
> we try hard not to put them in /etc).

Perhaps an interesting example of this is that in packages originally
developed on an OS with less elaborate conffile handling than dpkg's
(like dbus, originally developed on/by Red Hat), there used to be a
common pattern of having:

1. a file in /etc that sysadmins are specifically asked *not* to edit,
   for which non-dpkg package managers would be asked to default to
   the equivalent of dpkg's "use the maintainer's version" (for example
   %config in RPM), like /etc/dbus-1/system.conf

2. a separate file for local changes, which might or might not exist with
   empty or template contents by default, for which non-dpkg package
   managers would be asked to default to the equivalent of dpkg's "keep
   the locally modified version" (for example %config(noreplace) in RPM),
   like /etc/dbus-1/system-local.conf

/etc/dbus-1/system.conf was traditionally a dpkg conffile because
of its location, but it wasn't really a configuration file by
the definition Santiago gave here (and it has now been moved to
/usr/share/dbus-1/system.conf as a result): it's more like part of the
implementation of dbus-daemon --system that, as an implementation detail,
happens to have been written in the dbus-daemon XML configuration language
rather than in C.

/etc/dbus-1/system-local.conf isn't created by default, but sysadmins
can create it if they want to, and it's a "true" configuration file
(although in practice I'd recommend "drop-ins" in /etc/dbus-1/system.d
as a better version of /etc/dbus-1/system-local.conf).

smcv

Re: Does iproute2 moving config files to /usr/lib violate section 10.7.2?

[Bill Allombert]

On Sun, Sep 17, 2023 at 12:12:18AM +0200, Daniel Gröber wrote:
> Sam, Russ, Bill,
> 
> Thanks for your input. To be quite frank I still don't see how the
> interpretation of allowing configuration files outside of /etc can be
> supported based on the policy text.
> 
> Ultimately I'm just concerned about the UX aspects of admins suddenly
> having to go hunting for config files all over their system when packages
> start implementing this config-in-/usr business en mass.

It seems your issue is more with the documentation and transition to these
new schemes, which is a valid concern I mentionned in my answer.

There are two common schemes:
1/ the default file is read and then the config file is read to 
override/supplement it.
In that case the package should ship with dummy config files that document 
where is the file
they override

2/ the default file is only read when the config file does not exist.
This is more annoying since this precludes shipping with dummy config files.
In that case I recommend including a README file listing the path to the
default files that can be overrided.
Especially if this is a new scheme that users cannot yet be aware of.

Cheers,
Bill.

Re: Does iproute2 moving config files to /usr/lib violate section 10.7.2?

[Santiago Vila]

El 17/9/23 a las 0:12, Daniel Gröber escribió:

Sam, Russ, Bill,

Thanks for your input. To be quite frank I still don't see how the
interpretation of allowing configuration files outside of /etc can be
supported based on the policy text.


Hello. I apologize for not having read the discussion in full.

I believe there is an underlying semantic problem here.

Those files were shipped in /etc before. They are not shipped
in /etc anymore. Instead, files with identical contents
are now shipped in /usr/lib.

In your eyes, this is enough to consider that the files
have "moved", and maybe that's why you are still calling
them "configuration files outside of /etc".

But this is all about intent. It's not the format of the file
or the way they modify the behaviour of the program what
makes them to be "configuration files". It's the intent.

If the files are intended to be modified "in place" by the
system admin, we call them configuration files (and we try hard
to put them in /etc). If they are not intended to be modified by
the system admin, we don't call them configuration files (and
we try hard not to put them in /etc).


So I maintain that the current policy text doesn't allow configuration
files outside of /etc.


I agree, and this is why I think there is a semantic problem. The way I see it,
the problem is not really that there are "configuration files outside of /etc",
the problem is calling them configuration files when they are not anymore.

The definition of "configuration file" may not be perfect, but we have
to be careful not to twist it too much, because we might find ourselves
in the difficult position that lots of files in /usr/lib would also be
"configuration files outside of /etc".

That's why I believe intent is essential here.

Thanks.

Re: Does iproute2 moving config files to /usr/lib violate section 10.7.2?

[Russ Allbery]

Daniel Gröber  writes:

> Ultimately I'm just concerned about the UX aspects of admins suddenly
> having to go hunting for config files all over their system when
> packages start implementing this config-in-/usr business en mass.

I think the expectation is that you read the documentation of the package
that you're configuring, just as if the defaults were all hard-coded in
the binary.  I realize that some people *prefer* to have all of the
configuration of a package documented in comments or settings in a
template file installed into /etc, but this has never been something we
*required*, just a convention that some packages follow and some do not.

> I don't really mind if we go that way but then policy needs to be
> updated and some convention or mechanism[1] put in place to make working
> with config by hand as easy as it's been in the past, but that work is
> on the devs pushing the project in that direction.

This is a change that is mostly happening in the upstream development of
some packages, so I think your attribution of cause may not be entirely
accurate.  People are adapting Debian packages to a configuration model
that started in other distributions and has been adopted by the upstreams
of those packages, and some of them are also advocating for this
configuration practice as a good idea, but the push of Debian in that
direction would be happening regardless of the opinions of Debian
maintainers about this configuration method.  Some of our upstreams are
adopting this and it's highly unlikely that we would maintain local
patches to undo that change.

I'm not sure how much of the documentation belongs in Policy, since this
is really user-facing documentation, which is entirely outside the remit
of Policy.  Policy is only about packaging; we don't expect users of
Debian systems to read it.  But I'd be happy to review a patch that
updated the Policy wording of configuration files to make it clear that
"configuration files" only refers to the files edited by administrators in
the course of configuring the system, not to wherever the read-only
configuration defaults may be stored.

Obviously if someone wants to write better tools to manage these
configuration files, that would be great, but that's outside the scope of
this list, and I am dubious that we would refuse to update to newer
upstream versions of our packages until someone writes such a tool.  We
are part of a larger ecosystem, and some critical components of that
ecosystem are moving in this direction regardless of what we do.

> On Thu, Sep 14, 2023 at 09:41:00AM -0700, Russ Allbery wrote:

>> Configuration file has a very specific meaning in Policy: it's a file
>> that the local system administrator changes in order to configure the
>> software.

> Sorry, I don't buy that. Policy is kind enough to have an explicit
> definition for what it means by "configuration file" in
> 10.7.1. Definitions:

>> configuration file
>> 
>> A file that affects the operation of a program, or provides site- or
>> host-specific information, or otherwise customizes the behavior of a
>> program. 

> "A file that affects the operation of a program" pretty clear cut to
> mean exactly the type of files at issue here.

I understand that this is not what you want this part of Policy to mean,
but a "configuration file" in Policy has always been the file that you
edit to change the operation of the program.  It is not a *requirement*
that every file shipping with the package that contains configuration
defaults be installed in /etc.  Given the number of packages that
hard-code configuration defaults into the binary and put them nowhere else
on the file system at all, this would not be possible.

It is *common* for a package to install a configuration file that has some
or all of the defaults, as a template for local administrators to modify,
but there have always been packages that do not do this, and other
packages that do this for some configuration files and not others.  There
have always been some packages that put the defaults or the configuration
template elsewhere, whether that be /usr/share/doc//examples or
hard-coded into some binary.

Packages that use this "empty default /etc" configuration approach do not
ship *any* configuration files under the Policy definition.  They only
ship binaries and data files for those binaries that contain default
settings.  Those files are not modifiable and are not configuration files
in the Policy sense.  From a Policy perspective, they're exactly like the
files installed in /usr/share/pam-configs, /usr/share/doc-base, or
/usr/share/applications, which are also files that affect the operation of
a program.

I agree that the wording in Policy could be clearer.

-- 
Russ Allbery (r...@debian.org)  

Re: Does iproute2 moving config files to /usr/lib violate section 10.7.2?

[Daniel Gröber]

Sam, Russ, Bill,

Thanks for your input. To be quite frank I still don't see how the
interpretation of allowing configuration files outside of /etc can be
supported based on the policy text.

Ultimately I'm just concerned about the UX aspects of admins suddenly
having to go hunting for config files all over their system when packages
start implementing this config-in-/usr business en mass.

I don't really mind if we go that way but then policy needs to be updated
and some convention or mechanism[1] put in place to make working with
config by hand as easy as it's been in the past, but that work is on the
devs pushing the project in that direction.

[1]: Josh had an interesting idea over in a relate d-devel thread:
"sysadmin configuration of sparse-/etc vs prepopulated-/etc?"
https://lists.debian.org/debian-devel/2023/09/msg00192.html

Perhaps by having /usr/etc which must mirror where files go in /etc?
Right now different packages seem to put files in random places, some in
/usr/share some in /usr/lib. Madness.

If this becomes more common I fear it's going to cause a lot of friction
for root users. Remember: those are the ones with the power to just
uninstall Debian and replace it ;) 

On Thu, Sep 14, 2023 at 09:41:00AM -0700, Russ Allbery wrote:
> Configuration file has a very specific meaning in Policy: it's a file that
> the local system administrator changes in order to configure the software.

Sorry, I don't buy that. Policy is kind enough to have an explicit
definition for what it means by "configuration file" in
10.7.1. Definitions:

> configuration file
> 
> A file that affects the operation of a program, or provides site- or
> host-specific information, or otherwise customizes the behavior of a
> program. 

"A file that affects the operation of a program" pretty clear cut to mean
exactly the type of files at issue here. Policy goes on to suggest that
such a file might typically be intended to be edited by the admin:

> Typically, configuration files are intended to be modified by the system
> administrator (if needed or desired) to conform to local policy or to
> provide more useful site-specific behavior.

Note how this is not a requirement for a file to be captured by this
definition.

So I maintain that the current policy text doesn't allow configuration
files outside of /etc.

--Daniel


signature.asc
Description: PGP signature

Re: Does iproute2 moving config files to /usr/lib violate section 10.7.2?

[Bill Allombert]

On Thu, Sep 14, 2023 at 04:01:05PM +0200, Daniel Gröber wrote:
> Hello debian-policy,
> 
> iproute2 has moved it's config files that were traditionally at
> /etc/iproute2 to /usr/lib/iproute2 due to an upstream change. I've tried to
> convince the maintainer(s) that this is a bad idea in Bug#1051577, when
> this was shot down I filed Bug#1051847 as severity:serious on the basis of
> Debian policy section 10.7.2 section 10.7.2. "Configuration files /
> Location" which states as follows:
> 
> > Any configuration files created or used by your package must reside
> > in /etc.
> 
> Pretty clear cut in my reading, however this was promptly shot down by
> Bastian  with the justification:
> 
> > No, it does not.  The files in /usr are defaults.  Those should be copied
> > to /etc for modification, which is config.
> 
> Am I going nuts? Somehow long established unix convention and usability
> doesn't matter anymore and policy doesn't mean what it says?

The idea is that files in /usr/ are default files that can be 
overrided or supplemented by a configuration file in /etc/ with the same name.
When properly documented, this is a very good scheme since that make update of
configuration file easier.
The kpathsea system of TeX is an extreme version of that scheme that allow
several level of override simultaneously.

For a simple example, popularity-contest ships
/usr/share/popularity-contest/default.conf as default file
which is overrided by the configuration file
/etc/popularity-contest.conf

/usr/share/popularity-contest/default.conf says PARTICIPATE="no"
and users that want to participate set PARTICIPATE="yes" in
/etc/popularity-contest.conf (through debconf) to override it.

It is recommended that /etc/ carry dummy configuration files that provide
information on the default file they override.

Of course changing the configuration files scheme is a different issue and need
to be done carefully so that users understand what is expected of them.

Cheers,
-- 
Bill. 

Imagine a large red swirl here. 

Re: Does iproute2 moving config files to /usr/lib violate section 10.7.2?

[Russ Allbery]

Daniel Gröber  writes:

>> Any configuration files created or used by your package must reside
>> in /etc.

> Pretty clear cut in my reading, however this was promptly shot down by
> Bastian  with the justification:

Configuration file has a very specific meaning in Policy: it's a file that
the local system administrator changes in order to configure the software.
If the file is not intended to be modified to configure the software, it's
not a configuration file, even if it contains configuration defaults.
See:

https://www.debian.org/doc/debian-policy/ch-files.html#configuration-files

although it doesn't make this as clear as it should be because it was
written before split configurations became common.

That's because the main point of distinguishing configuration files (in
addition to collecting them in /etc for the use of things like etckeeper)
is to ensure that we handle the modifications correctly, don't lose them
during upgrades, etc.  Static files that aren't modified by the local
administrator don't need that special handling.

Having the configuration *defaults* exist as a static file on disk in
/usr/lib isn't really that different than having the configuration
defaults exist hard-coded into the binary, which has been common for many
years.  Either way, what Debian requires is that the defaults be modified
by putting files into /etc, so as long as that continues to be true, this
is satisfying Debian's configuration requirement.  This isn't that
different than long-standing UNIX precedent, as seen in packages such as
postfix or INN or any number of other programs that support configuration
files in /etc but have a large number of hard-coded defaults that apply if
those files are missing or don't set that option.

I should say explicitly that none of this addresses the *preference*
argument between people who prefer to have the defaults in /usr and
overrides in /etc, and people who prefer to have all of the settings in
/etc so that they are readily visible for modification.  Some people
prefer one, some people prefer the other, and both sides feel strongly
about it and probably won't convince the other side.  But this conflict
has existed for essentially forever in the form of defaults that are
hard-coded into the binary and not expressed in the sample configuration
file, so it's not really a new conflict, and I think it's more transparent
to have defaults in a separate file in /usr than hard-coded into a
compiled binary.

I do understand why folks who prefer the "all settings in one modifyable
file in /etc" are annoyed when packages move to the split configuration
model with a file of defaults in /usr, but it's not something that Policy
requires one way or the other and there are vigorous advocates of both
methods.

-- 
Russ Allbery (r...@debian.org)  

Re: Does iproute2 moving config files to /usr/lib violate section 10.7.2?

[Sam Hartman]

> "Daniel" == Daniel Gröber  writes:


>> Any configuration files created or used by your package must
>> reside in /etc.

It's fine for packages to store defaults outside of /etc.  But that 's
only true if you can override those defaults by placing a file in /etc.

Does iproute2 moving config files to /usr/lib violate section 10.7.2?

[Daniel Gröber]

Hello debian-policy,

iproute2 has moved it's config files that were traditionally at
/etc/iproute2 to /usr/lib/iproute2 due to an upstream change. I've tried to
convince the maintainer(s) that this is a bad idea in Bug#1051577, when
this was shot down I filed Bug#1051847 as severity:serious on the basis of
Debian policy section 10.7.2 section 10.7.2. "Configuration files /
Location" which states as follows:

> Any configuration files created or used by your package must reside
> in /etc.

Pretty clear cut in my reading, however this was promptly shot down by
Bastian  with the justification:

> No, it does not.  The files in /usr are defaults.  Those should be copied
> to /etc for modification, which is config.

Am I going nuts? Somehow long established unix convention and usability
doesn't matter anymore and policy doesn't mean what it says?

I'd appreciate any input and advice on how to proceed here since I'm new to
contributing to Debian outside of packaging :)

Thanks,
--Daniel


signature.asc
Description: PGP signature