Mass filing wish list bugs against package depending on dbconfig-common [WAS: Re: dbconfig-common: near future change in dependency stack]

[Paul Gevers]

Hi all,

The new style dbconfig-common packages live already for some time in the
archive, and I haven't received any complaint/bug yet about the database
dependent packages. This means that I believe it is time to bug the
maintainers of packages that depend on dbconfig-common with a wish list
bug to change their dependencies to the right package.

Because there are lots of packages involved, I want to share my proposed
report here and check that there are no objections.

Please find my proposed text attached, as well as a dd list for the
maintainers involved (ironic note, I'm on the list myself).

I'd love to hear your comments.

Paul
Title: please depend on database specific dbconfig- package instead of 
dbconfig-common

Dear maintainer,

Your package depends on dbconfig-common. Some time ago I introduced database
type specific packages in the dbconfig framework. I announced this on
debian-devel¹. Now it is time to request you to change the dependency of your
package to depend on the dbconfig- packages matching the database(s)
that your package supports. For the convenience of system administrators that
don't want dbconfig-common to handle the database handling, I also introduced a
package called dbconfig-no-thanks, which will prevent dbconfig-common from
doing anything (just like answering no to the first dbconfig-common
question). You should always have dbconfig-no-thanks as an (last) alternative to
the dbconfig- package(s).

So, how to make use of these new packages? The only change you have to do² is
revisit your (pre-)dependencies/recommends/suggest. If you properly followed the
dbconfig-common documentation, you have a dependency on dbconfig-common and at
least a recommends (but probably a depends) on the command-line client(s) for
the database type(s) you support. You should replace these with a depends on
dbconfig- | dbconfig-no-thanks. Two examples.

a) your package supports PostgreSQL, your dependencies now are
Depends: dbconfig-pgsql | dbconfig-no-thanks

b) your package supports sqlite3 or sqlite, your dependencies now are
Depends: dbconfig-sqlite3 | dbconfig-sqlite | dbconfig-no-thanks

If you don't need the command-line databse client for your package itself,
i.e. you only added it because dbconfig-common needs it, you can now remove
that as well, because now the  packages take care themselves.

For those of you that backport their packages via the Debian backports achive,
I provide backports of dbconfig-common to jessie-backports. On request I can do
the same with wheezy-backports-sloppy.

For those of you that also provide packages elsewhere where you may not have
dbconfig-common version 2.0.0 or higher, I can recommend the trick done by
phpmyadmin:
dbconfig-mysql | dbconfig-no-thanks | dbconfig-common (<< 2.0.0)

Paul

¹ https://lists.debian.org/debian-devel/2015/12/msg00044.html
² Be aware, if your package supports multiple databases, you still need
to set the dbc_dbtypes variable in you config script.
Alexander Wirt 
   icinga (U)
   icinga-web (U)
   icinga2 (U)

Andreas Henriksson 
   bandwidthd

Andreas Tille 
   manila (U)

Bareos Packaging Team 
   bareos

Cacti Maintainer 
   cacti-spine

Carsten Leonhardt 
   bacula (U)

Corey Bryant 
   murano (U)

Craig Small 
   jffnms

Dain Nilsson 
   yubikey-ksm (U)
   yubikey-val (U)

Daniel Pocock 
   yubikey-ksm (U)
   yubikey-val (U)

Dario Minnucci 
   dotclear

David Gil 
   phpgacl

David Prévot 
   phpbb3 (U)

Debian Authentication Maintainers 
   yubikey-ksm
   yubikey-val

Debian Bacula Team 
   bacula

Debian Nagios Maintainer Group 
   icinga
   icinga-web
   icinga2
   ndoutils (U)

Debian QA Group 
   semanticscuttle
   webissues-server

Debian Request Tracker Group 

   request-tracker4

Debian Roundcube Maintainers 
   roundcube

Debian Science Maintainers 
   tango

Debian Sympa team 
   sympa

Dominic Hargreaves 
   request-tracker4 (U)

Emmanuel Bouthenot 
   sympa (U)

Evgeni Golov 
   bareos (U)
   bley

Francesco Paolo Lovergine 
   openacs (U)

Francisco Manuel Garcia Claramonte 
   letodms

Gaudenz Steinlin 
   postfixadmin (U)

Gonéri Le Bouder 
   glpi (U)

Guilhem Moulin 
   roundcube (U)

Gunnar Wolf 

Re: dbconfig-common: near future change in dependency stack

[Clint Byrum]

Excerpts from The Wanderer's message of 2016-01-30 15:49:51 -0800:
> On 2016-01-30 at 08:49, Clint Byrum wrote:
> 
> > Excerpts from The Wanderer's message of 2016-01-30 04:28:42 -0800:
> > 
> >> On 2016-01-30 at 04:51, Paul Gevers wrote:
> 
> >>> Actually, I don't think that is in scope of dbconfig-common. I
> >>> would rather expect that MariaDB would provide that
> >>> functionality. It is required for more packages and situations
> >>> than just those supported by dbconfig-common.
> >> 
> >> Are there even cases where this is necessary?
> >> 
> >> Within the last year, I encountered an unacceptable - but
> >> intentional - change in the MySQL client interface, so I removed
> >> the MySQL packages and installed the MariaDB ones.
> > 
> > Which client interface would that be? libmysqlclient18 is still
> > provided by mysql, even if you install MariaDB.
> 
> The one invoked with the command 'mysql'.
> 
> The underlying library is still the same (though as far as I can see,
> the dependency chain from mariadb-client-core-10.0 to libmysqlclient18
> only exists because libdbd-mysql-perl depends on the latter, so I'm not
> convinced that ), but the client itself behaves differently.
> 
> The specific UI change which drove me away from "pure" MySQL was the
> change in line-editing library, away from readline, so that my habitual
> "jump around the line while editing the query" practices would no longer
> function - and, as far as I recall, there was no suitable replacement. I
> dug around for a while looking for a way to turn the readline behaviors
> back on, started digging into the process of building my own packages
> which revert to the old behavior, then discovered that MariaDB had not
> followed that change and decided to just migrate instead.
> 

Funny story.. they do speak the same protocol, and you can use the
mariadb client with mysql server just fine. :)

> >> My existing database was picked up and used without issues; the 
> >> transition was, on that level, pretty much seamless as far as I
> >> recall. I might have needed to re-apply some configuration tweaks
> >> in different config files, but nothing more than that.
> > 
> > This is a one-way trip as of MariaDB 10. MariaDB 5.5 was compatible
> > with MySQL 5.5 and allowed using the same on-disk files. But MySQL
> > may not know how to read all of the files produced by MariaDB 10+. So
> > I would not count on this working again in the future. They're truly
> > forks, and you will need to backup/restore to make this work.
> 
> This is valuable to know for future reference, though I'm not sure I'm
> ever likely to need or want to migrate in that direction between the
> two; I was only still on MySQL rather than MariaDB out of inertia.
> 

MySQL is adding features and pushing scale upwards at a very fast rate.
So some users encountering this thread might be in the opposite situation
of needing to go from MariaDB to MySQL.

> Since the topic at hand was specifically migrating from MySQL to
> MariaDB, however, rather than bidirectional migration between the two,
> my question stands: are there cases where migration from MySQL to
> MariaDB needs to be done explicitly per-database?
> 

I don't really understand the context of your question.

Re: dbconfig-common: near future change in dependency stack

[Paul Gevers]

Hi all,

On 06-12-15 14:23, Paul Gevers wrote:
> For those of you that backport their packages via the Debian backports
> archive, I will provide a backport of dbconfig-common once version 2.0.0
> reaches testing.

This has happened. dbconfig-common, dbconfig- and
dbconfig-no-thanks are now in sid, testing and jessie-backports.

For those package maintainers that have packages depending on
dbconfig-common, now is a good time to remove the dependency on
dbconfig-common and add the appropriate dbconfig- |
dbconfig-no-thanks dependency instead.

Paul



signature.asc
Description: OpenPGP digital signature

Re: dbconfig-common: near future change in dependency stack

[Paul Gevers]

Hi,

On 30-01-16 12:30, PICCA Frederic-Emmanuel wrote:
>> If a package relies on the dbconfig-common framework for database setup
>> and maintenance, installing dbconfig-no-thanks instead of one of
>> dbconfig's database-specific packages will block this function. It is
>> intended for cases where the system administrator desires or requires
>> full control of the database or where dbconfig-common makes bad choices,
>> and typically leaves the depending packages non-functional until
>> manually configured.
>> """
> 
> If the no-thanks package is installed, what could be expected from
> the package maintainer in order to deal with the system administrator
> database mis configuration.

If the administrator installs dbconfig-no-thanks, he must accept that
the database for the package is not configured. By the way, this has
always been the case, but until now only via the debconf questions.
There are too many use cases where the admin wants to handle the
database creation and population differently, so a package should not
abort on that. Ironically, quite some package even (wrongly and in my
opinion unnecessarily) ignore the exit status of dbconfig-common just
for that reason.

If a package really, really, really must have the proper database during
installation, I would say you can only have that if you use a local file
type database like sqlite. MySQL/MariaDB/PostgreSQL are too often
running on a remote server where you have absolutely no guarantee that
you can perform the required actions. If you have such a (local file
type) case I guess it may makes sense to not alternatively depend on
dbconfig-no-thanks, such that you will always pull in the right
dbconfig- package. However, this will get in the way of the
administrator when she want to install another package that uses
dbconfig-common but doesn't want the dbconfig-common help.

> We just need to let the package un-configure in order to explain that
> the database is wrongly configure.

I don't think so, and I have the feeling (due to previous discussions
and several bug reports) that a lot of developers and system admins
disagree with you here. They want Debian to install the package, and
handle the database in a different way. If the package would remain
unconfigured, that disturbs other installs in an unpleasant way.

> Do we have something in dbconfig-common whcih could say, here ok I do
> not manage the database but with this sysadmin database
> configuration, the package is not working.

I don't think dbconfig-common can do that. If dbconfig-common goes out
of the way, either because dbconfig-no-thanks is installed or because of
a debconf answer, the admin should expect that the package doesn't work,
because he told "us" to not preform the required actions. I truly
believe that packages that require a working database connection should
not run havoc, neither during install, nor during automatic startup,
when the required database connection and content is not available.

Paul



signature.asc
Description: OpenPGP digital signature

Re: dbconfig-common: near future change in dependency stack

[Paul Gevers]

Hi Frederic-Emmanuel,

On 30-01-16 09:30, PICCA Frederic-Emmanuel wrote:
> I am the maintainer of tango-db which use dbconfig-common and a mysql 
> database.
> It seems that there is currently a discussion about he support of mysql and 
> mariadb for Debian 9

Can you please point me to the relevant discussion?

> Do you know if dbconfig-common will integrate a way to switch from mysql to 
> mariadb in the near futur.
> something whcih can help do the migration database from mysql to mariadb.

Actually, I don't think that is in scope of dbconfig-common. I would
rather expect that MariaDB would provide that functionality. It is
required for more packages and situations than just those supported by
dbconfig-common.

> Also, I have the feeling that the new dbconfig-no-thanks is too coarse.
> It mean no database of any kind supported by dbconfig-common could be install 
> on this machine.

There must be a misunderstanding here (and I would like to learn where
it comes from). dbconfig-no-thanks does NOTHING to get in the way of any
database. The ONLY thing that it does it prevent dbconfig-common from
managing the database for the package that depends on the
dbconfig-common framework. As the description reads now:
"""
If a package relies on the dbconfig-common framework for database setup
and maintenance, installing dbconfig-no-thanks instead of one of
dbconfig's database-specific packages will block this function. It is
intended for cases where the system administrator desires or requires
full control of the database or where dbconfig-common makes bad choices,
and typically leaves the depending packages non-functional until
manually configured.
"""

dbconfig-no-thanks only conflicts with all dbconfig- packages so
it doesn't block anything else.

> But I would like to express, no mysql on my computer, but I could allow 
> postgresql for other packages.
> Is it possible to have this use case and how should we instrument our package 
> for this?

I am not sure what exactly you want, but you CAN'T use the
dbconfig-common framework to prevent installation of MySQL, it is not
intended for that. With dbconfig-no-thanks installed ANY package that
relies on the dbconfig-common framework will not configure its database.
Without installing dbconfig-no-thanks, you can still (as has always been
the case) opt-out of dbconfig-common support per package, but this
requires either preseeding or responding to the corresponding debconf
question.

Paul



signature.asc
Description: OpenPGP digital signature

Re: dbconfig-common: near future change in dependency stack

[The Wanderer]

On 2016-01-30 at 04:51, Paul Gevers wrote:

> Hi Frederic-Emmanuel,
> 
> On 30-01-16 09:30, PICCA Frederic-Emmanuel wrote:

>> Do you know if dbconfig-common will integrate a way to switch from
>> mysql to mariadb in the near futur. something whcih can help do the
>> migration database from mysql to mariadb.
> 
> Actually, I don't think that is in scope of dbconfig-common. I would
> rather expect that MariaDB would provide that functionality. It is
> required for more packages and situations than just those supported
> by dbconfig-common.

Are there even cases where this is necessary?

Within the last year, I encountered an unacceptable - but intentional -
change in the MySQL client interface, so I removed the MySQL packages
and installed the MariaDB ones.

My existing database was picked up and used without issues; the
transition was, on that level, pretty much seamless as far as I recall.
I might have needed to re-apply some configuration tweaks in different
config files, but nothing more than that.

This seems to imply that either migration is not required, or MariaDB
already performs the needed migration transparently. (Or else that I'm
forgetting some part of the transition process, which is not
impossible.)

I can imagine that there could be cases where migration would be
required, but I'm not aware of any, and it didn't even occur to me to
expect that I might need to do any in my own case. I expected that the
two would be seamlessly compatible on the database level, and that
expectation seems to have been borne out.

-- 
   The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man. -- George Bernard Shaw



signature.asc
Description: OpenPGP digital signature

Re: dbconfig-common: near future change in dependency stack

[Clint Byrum]

Excerpts from The Wanderer's message of 2016-01-30 04:28:42 -0800:
> On 2016-01-30 at 04:51, Paul Gevers wrote:
> 
> > Hi Frederic-Emmanuel,
> > 
> > On 30-01-16 09:30, PICCA Frederic-Emmanuel wrote:
> 
> >> Do you know if dbconfig-common will integrate a way to switch from
> >> mysql to mariadb in the near futur. something whcih can help do the
> >> migration database from mysql to mariadb.
> > 
> > Actually, I don't think that is in scope of dbconfig-common. I would
> > rather expect that MariaDB would provide that functionality. It is
> > required for more packages and situations than just those supported
> > by dbconfig-common.
> 
> Are there even cases where this is necessary?
> 
> Within the last year, I encountered an unacceptable - but intentional -
> change in the MySQL client interface, so I removed the MySQL packages
> and installed the MariaDB ones.
> 

Which client interface would that be? libmysqlclient18 is still provided
by mysql, even if you install MariaDB.

> My existing database was picked up and used without issues; the
> transition was, on that level, pretty much seamless as far as I recall.
> I might have needed to re-apply some configuration tweaks in different
> config files, but nothing more than that.
> 

This is a one-way trip as of MariaDB 10. MariaDB 5.5 was compatible with
MySQL 5.5 and allowed using the same on-disk files. But MySQL may not
know how to read all of the files produced by MariaDB 10+. So I would
not count on this working again in the future. They're truly forks, and
you will need to backup/restore to make this work.

> This seems to imply that either migration is not required, or MariaDB
> already performs the needed migration transparently. (Or else that I'm
> forgetting some part of the transition process, which is not
> impossible.)
> 
> I can imagine that there could be cases where migration would be
> required, but I'm not aware of any, and it didn't even occur to me to
> expect that I might need to do any in my own case. I expected that the
> two would be seamlessly compatible on the database level, and that
> expectation seems to have been borne out.
> 

Yeah, that would be nice, but the reality is, code is only flowing
_away_ from MySQL at this point. MariaDB's changes don't go back into
MySQL. So the forks will just get further and further apart.

Re: dbconfig-common: near future change in dependency stack

[The Wanderer]

On 2016-01-30 at 08:49, Clint Byrum wrote:

> Excerpts from The Wanderer's message of 2016-01-30 04:28:42 -0800:
> 
>> On 2016-01-30 at 04:51, Paul Gevers wrote:

>>> Actually, I don't think that is in scope of dbconfig-common. I
>>> would rather expect that MariaDB would provide that
>>> functionality. It is required for more packages and situations
>>> than just those supported by dbconfig-common.
>> 
>> Are there even cases where this is necessary?
>> 
>> Within the last year, I encountered an unacceptable - but
>> intentional - change in the MySQL client interface, so I removed
>> the MySQL packages and installed the MariaDB ones.
> 
> Which client interface would that be? libmysqlclient18 is still
> provided by mysql, even if you install MariaDB.

The one invoked with the command 'mysql'.

The underlying library is still the same (though as far as I can see,
the dependency chain from mariadb-client-core-10.0 to libmysqlclient18
only exists because libdbd-mysql-perl depends on the latter, so I'm not
convinced that ), but the client itself behaves differently.

The specific UI change which drove me away from "pure" MySQL was the
change in line-editing library, away from readline, so that my habitual
"jump around the line while editing the query" practices would no longer
function - and, as far as I recall, there was no suitable replacement. I
dug around for a while looking for a way to turn the readline behaviors
back on, started digging into the process of building my own packages
which revert to the old behavior, then discovered that MariaDB had not
followed that change and decided to just migrate instead.

>> My existing database was picked up and used without issues; the 
>> transition was, on that level, pretty much seamless as far as I
>> recall. I might have needed to re-apply some configuration tweaks
>> in different config files, but nothing more than that.
> 
> This is a one-way trip as of MariaDB 10. MariaDB 5.5 was compatible
> with MySQL 5.5 and allowed using the same on-disk files. But MySQL
> may not know how to read all of the files produced by MariaDB 10+. So
> I would not count on this working again in the future. They're truly
> forks, and you will need to backup/restore to make this work.

This is valuable to know for future reference, though I'm not sure I'm
ever likely to need or want to migrate in that direction between the
two; I was only still on MySQL rather than MariaDB out of inertia.

Since the topic at hand was specifically migrating from MySQL to
MariaDB, however, rather than bidirectional migration between the two,
my question stands: are there cases where migration from MySQL to
MariaDB needs to be done explicitly per-database?

>> This seems to imply that either migration is not required, or
>> MariaDB already performs the needed migration transparently. (Or
>> else that I'm forgetting some part of the transition process, which
>> is not impossible.)
>> 
>> I can imagine that there could be cases where migration would be 
>> required, but I'm not aware of any, and it didn't even occur to me
>> to expect that I might need to do any in my own case. I expected
>> that the two would be seamlessly compatible on the database level,
>> and that expectation seems to have been borne out.
> 
> Yeah, that would be nice, but the reality is, code is only flowing 
> _away_ from MySQL at this point. MariaDB's changes don't go back
> into MySQL. So the forks will just get further and further apart.

That's more or less what I'd expect. It still seems possible that the
"downstream" fork would retain input compatibility with the "upstream"
parent, but certainly that's not guaranteed.

-- 
   The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man. -- George Bernard Shaw



signature.asc
Description: OpenPGP digital signature

Re: dbconfig-common: near future change in dependency stack

[Paul Gevers]

Hi Zigo,

On 10-12-15 01:12, Thomas Goirand wrote:
> Now, what should I do if my packages support all types of db currently
> supported by dbconfig-common (ie: sqlite, mysql and postgress)?

Wasn't my example in the first e-mail clear? I would expect:
Depends: dbconfig-sqlite | dbconfig-sqlite3 | dbconfig-mysql |
dbconfig-pgsql | dbconfig-no-thanks
Similar to what you now do with the connectors that you have to all
these databases. (Or am I missing something?)

Paul



signature.asc
Description: OpenPGP digital signature

Re: dbconfig-common: near future change in dependency stack

[Gunnar Wolf]

Mathias Behrle dijo [Tue, Dec 08, 2015 at 10:16:38AM +0100]:
> Some questions for Tryton server:
> 
> The situation is
> 
> - Tryton server runs out of the box without configuration with a SQLite
>   database
> - The (strongly) recommended database is PostgreSQL, but there is also support
>   for MySQL (and beta for Oracle)
> - There is a standalone package called Neso (Tryton Neso), that provides a
>   simple combined setup of Tryton client and server (using SQLite), depending
>   on those two.
> 
> Would it be possible for the last point to skip completely the database
> configuration (e.g. dbconfig* not jumping in at all, when tryton-neso pulls in
> tryton-server)?
> 
> What would be the best way to handle the strong preference for PostgreSQL?
> MySQL is supported for Tryton, but fails to complete some tests due to missing
> Standard SQL compliance. So I dislike generally the idea to install a
> client package for MySQL on a default system. The solution would probably be 
> to
> not implement at all the option for MySQL, right?

Ummm, I think this level of detail is where you should not leave it up
to the package manager only, but explain the situation to the user, be
it (very succintly) as part of your debconf prompts, or (better?) via
a README.Debian

Re: dbconfig-common: near future change in dependency stack

[Thomas Goirand]

On 12/06/2015 06:22 PM, Clint Byrum wrote:
> Excerpts from Paul Gevers's message of 2015-12-06 05:23:07 -0800:
>> Hi all,
>>
>> TL;DR;1 if your package depends on dbconfig-common please update your
>> dependencies when my version 2.0.0 hits the archive (I expect in two
>> weeks).
>> TL;DR;2 should the new dbconfig- packages recommend or suggest
>> the database server packages?
>>
>> Since I took over the dbconfig-common package I have worked on the
>> following feature in the dbconfig-common framework: binary packages to
>> specify in the dependency chain which database types a package supports.
>>
>> The idea is the following. Each package that used the dbconfig-common
>> framework to set up databases, should depend on dbconfig- |
>> dbconfig-no-thanks instead of depending on dbconfig-common (as used to
>> be the case and still works). What this solves is multiple issues:
>>
> 
> This is great! Thanks Paul. I've never been very happy with
> dbconfig-common because it kind of assumes databases are on the same
> server as apps, which is increasingly not the case with smaller server
> instances running in VMs, on the cloud and in containers.

Hum ...

dpkg-reconfigure dbconfig-common

> However, I also think that postinstall is not the right time to
> be configuring your database, and I'd rather see guidance in the
> documentation and the fine wizard that is dbconfig suggested as a CLI tool
> for users to use once they have thought through their database options.

I don't agree. Postinst is the perfect place. I don't see why we would
make things manual when we have automation tooling.

> So adding it to the Recommends just adds mystery to this process and
> doesn't actually help users find their way to best practices.

I though agree with the 2 lines above! :)

Cheers,

Thomas Goirand (zigo)

Re: dbconfig-common: near future change in dependency stack

[Thomas Goirand]

On 12/06/2015 02:23 PM, Paul Gevers wrote:
> As a bonus, I can now add the database server packages to recommends,
> which should make life of the less experienced user easier. Do you think
> I should do this, or should I leave the database server package at the
> suggests level?

Please don't put it as Recommends:, Suggests: is enough.

Now, what should I do if my packages support all types of db currently
supported by dbconfig-common (ie: sqlite, mysql and postgress)?

Cheers,

Thomas Goirand (zigo)

Re: dbconfig-common: near future change in dependency stack

[Mathias Behrle]

* Paul Gevers: " dbconfig-common: near future change in dependency stack" (Sun,
  06 Dec 2015 14:23:07 +0100):

Hi Paul,

> TL;DR;1 if your package depends on dbconfig-common please update your
> dependencies when my version 2.0.0 hits the archive (I expect in two
> weeks).
> TL;DR;2 should the new dbconfig- packages recommend or suggest
> the database server packages?

Suggest.
 
> Since I took over the dbconfig-common package I have worked on the
> following feature in the dbconfig-common framework: binary packages to
> specify in the dependency chain which database types a package supports.
> 
> The idea is the following. Each package that used the dbconfig-common
> framework to set up databases, should depend on dbconfig- |
> dbconfig-no-thanks instead of depending on dbconfig-common (as used to
> be the case and still works). What this solves is multiple issues:
> 
> 0) Bug: 353617¹
> 
> 1) Because there is an alternative, dbconfig- can depend on the
> command-line client for dbtype, instead of  recommending
> it. Thus properly signifying the hard requirement of dbconfig-common to
> have the command-line client available.
> 
> 2) For multiple dbtype supported packages the system administrator now
> has a way outside of debconf to select which dbtype he wants by
> installing the right dbconfig- package. Currently the question
> will still be asked though.
> 
> 3) The system administrator now has a way to say no-thanks to
> dbconfig-common (by installing the dbconfig-no-thanks package) on the
> system level, rather than per package via debconf.
> 
> As a bonus, I can now add the database server packages to recommends,
> which should make life of the less experienced user easier. Do you think
> I should do this, or should I leave the database server package at the
> suggests level?
> 
> Anyways, so what do you need to do? If your package depends on
> dbconfig-common (dd-list attached), the only thing you need to do² is
> revisit your dependencies/recommends/suggest. If you properly followed
> the dbconfig-common documentation, you have a dependency on
> dbconfig-common and at least a recommends (but probably a depends) on
> the command-line client(s) for the database type(s) you support. You
> should replace these with a depends on dbconfig- |
> dbconfig-no-thanks. Two examples.
> 
> a) your package supports PostgreSQL, your dependencies now are
> Depends: dbconfig-pgsql | dbconfig-no-thanks
> 
> b) your package supports sqlite3 or sqlite, your dependencies now are
> Depends: dbconfig-sqlite3 | dbconfig-sqlite | dbconfig-no-thanks

Some questions for Tryton server:

The situation is

- Tryton server runs out of the box without configuration with a SQLite
  database
- The (strongly) recommended database is PostgreSQL, but there is also support
  for MySQL (and beta for Oracle)
- There is a standalone package called Neso (Tryton Neso), that provides a
  simple combined setup of Tryton client and server (using SQLite), depending
  on those two.

Would it be possible for the last point to skip completely the database
configuration (e.g. dbconfig* not jumping in at all, when tryton-neso pulls in
tryton-server)?

What would be the best way to handle the strong preference for PostgreSQL?
MySQL is supported for Tryton, but fails to complete some tests due to missing
Standard SQL compliance. So I dislike generally the idea to install a
client package for MySQL on a default system. The solution would probably be to
not implement at all the option for MySQL, right?

Since the production use of tryton-server usually needs some other tweaking to
be done in the configuration file, the use of dbconfig* for Tryton is limited
anyway. I prefer until now to provide a detailed configuration file and README,
but of course I am open to suggestions.

Thanks,
Mathias

> For those of you that backport their packages via the Debian backports
> achive, I will provide a backport of dbconfig-common once version 2.0.0
> reaches testing.
> 
> Please speak up now if you think this is a ridiculous idea, if you have
> suggestions on improvements, if you have questions or otherwise.
> 
> If people want to see how it all works for discussion, I am open to
> upload to experimental.
> 
> And as always, please report bugs as you find them including wish bugs.
> 
> Paul
> 
> ¹ https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=353617
> ² Be aware, if your package supports multiple databases, you still need
> to set the dbc_dbtypes variable in you config script.



-- 

Mathias Behrle
PGP/GnuPG key availabable from any keyserver, ID: 0x8405BBF6


pgphSLW4xPmDE.pgp
Description: Digitale Signatur von OpenPGP

Re: dbconfig-common: near future change in dependency stack

[Vincent Danjean]

Le 06/12/2015 20:48, Paul Gevers a écrit :
> I am totally flabbergasted. dbconfig-common goes through a lot of hoops
> just to make sure this is not true. Sure, the *default* questions and
> answers are geared towards local database servers. But by either
> (re-)configuring dbconfig-common to raise the priority level of remote
> questions by default, or by raising the priority level during package
> configuring will get you the questions needed to configure your remote
> database. (Of course we can discuss what the proper priorities are but I
> really hope no DD will think (anymore) that dbconfig-common assumes
> local databases.)

When local configuration does not work or when local databases cannot
be reached or when local database server packages are not installed
the priority of dbconfig-common question about remote database should
be modified so that the user can configure a remote database without
failing the current installation. I know I already talked about that.
I do not remember if this is already implemented or not.

  Regards,
Vincent

-- 
Vincent Danjean   GPG key ID 0xD17897FA vdanj...@debian.org
GPG key fingerprint: 621E 3509 654D D77C 43F5  CA4A F6AE F2AF D178 97FA
Unofficial pkgs: http://moais.imag.fr/membres/vincent.danjean/deb.html
APT repo:  deb http://people.debian.org/~vdanjean/debian unstable main

Re: dbconfig-common: near future change in dependency stack

[Paul Gevers]

Hi Mathias,

On 08-12-15 10:16, Mathias Behrle wrote:
> Would it be possible for the last point to skip completely the database
> configuration (e.g. dbconfig* not jumping in at all, when tryton-neso pulls in
> tryton-server)?

I believe this is something that you would need to implement inside the
configuration of tryton-neso/server maintainer scripts. I.e. don't call
the dbconfig-common dbc_go commands in the tryton-server maintainer
scripts when it detects (non-trivial) that also tryson-neso is (about to
be) installed.

> What would be the best way to handle the strong preference for PostgreSQL?
> MySQL is supported for Tryton, but fails to complete some tests due to missing
> Standard SQL compliance. So I dislike generally the idea to install a
> client package for MySQL on a default system. The solution would probably be 
> to
> not implement at all the option for MySQL, right?

I assume you don't like the dbconfig-pgsql | dbconfig-mysql |
dbconfig-no-thanks dependency alternatives? By default the user will
then get pgsql. Then I believe your option is to not activate the MySQL
support in your package.

> Since the production use of tryton-server usually needs some other tweaking to
> be done in the configuration file, the use of dbconfig* for Tryton is limited
> anyway. I prefer until now to provide a detailed configuration file and 
> README,
> but of course I am open to suggestions.

Sounds reasonable, but if the db support can be handled, why not. What
would you say you are missing from dbconfig-common?

Paul



signature.asc
Description: OpenPGP digital signature

Re: dbconfig-common: near future change in dependency stack

[Paul Gevers]

Hi Vincent,

On 08-12-15 09:49, Vincent Danjean wrote:
> When local configuration does not work or when local databases cannot
> be reached or when local database server packages are not installed
> the priority of dbconfig-common question about remote database should
> be modified so that the user can configure a remote database without
> failing the current installation. I know I already talked about that.
> I do not remember if this is already implemented or not.

I never heard of it, nor is it implemented, unless you mean this in the
context of error recovery. In case of ANY error, the priority level of
all questions is raised, which allow you to see the all the questions if
you hit "Retry" (without the "(skip questions)").

Paul



signature.asc
Description: OpenPGP digital signature

Re: dbconfig-common: near future change in dependency stack

[Vincent Danjean]

Le 08/12/2015 20:25, Paul Gevers a écrit :
> I assume you don't like the dbconfig-pgsql | dbconfig-mysql |
> dbconfig-no-thanks dependency alternatives? By default the user will
> then get pgsql.

Not if the user has dbconfig-mysql or dbconfig-no-thanks already
installed (probably due to other packages).

Dependency alternatives are rarely a good hint for preference, but if
the system is installing for the first time (as in autobuilder)

  Regards,
Vincent

-- 
Vincent Danjean   GPG key ID 0xD17897FA vdanj...@debian.org
GPG key fingerprint: 621E 3509 654D D77C 43F5  CA4A F6AE F2AF D178 97FA
Unofficial pkgs: http://moais.imag.fr/membres/vincent.danjean/deb.html
APT repo:  deb http://people.debian.org/~vdanjean/debian unstable main

Re: dbconfig-common: near future change in dependency stack

[Vincent Danjean]

Le 08/12/2015 20:14, Paul Gevers a écrit :
> Hi Vincent,
> 
> On 08-12-15 09:49, Vincent Danjean wrote:
>> When local configuration does not work or when local databases cannot
>> be reached or when local database server packages are not installed
>> the priority of dbconfig-common question about remote database should
>> be modified so that the user can configure a remote database without
>> failing the current installation. I know I already talked about that.
>> I do not remember if this is already implemented or not.
> 
> I never heard of it, nor is it implemented, unless you mean this in the
> context of error recovery. In case of ANY error, the priority level of
> all questions is raised, which allow you to see the all the questions if
> you hit "Retry" (without the "(skip questions)").

Yes, I was thinking to that feature.

> Paul
> 


-- 
Vincent Danjean   GPG key ID 0xD17897FA vdanj...@debian.org
GPG key fingerprint: 621E 3509 654D D77C 43F5  CA4A F6AE F2AF D178 97FA
Unofficial pkgs: http://moais.imag.fr/membres/vincent.danjean/deb.html
APT repo:  deb http://people.debian.org/~vdanjean/debian unstable main

Re: dbconfig-common: near future change in dependency stack

[Vincent Bernat]

 ❦  6 décembre 2015 14:23 +0100, Paul Gevers  :

> TL;DR;2 should the new dbconfig- packages recommend or suggest
> the database server packages?

Suggest.

Otherwise, your plan looks just fine.
-- 
All generalizations are false, including this one.
-- Mark Twain


signature.asc
Description: PGP signature

Re: dbconfig-common: near future change in dependency stack

[Clint Byrum]

Excerpts from Paul Gevers's message of 2015-12-06 05:23:07 -0800:
> Hi all,
> 
> TL;DR;1 if your package depends on dbconfig-common please update your
> dependencies when my version 2.0.0 hits the archive (I expect in two
> weeks).
> TL;DR;2 should the new dbconfig- packages recommend or suggest
> the database server packages?
> 
> Since I took over the dbconfig-common package I have worked on the
> following feature in the dbconfig-common framework: binary packages to
> specify in the dependency chain which database types a package supports.
> 
> The idea is the following. Each package that used the dbconfig-common
> framework to set up databases, should depend on dbconfig- |
> dbconfig-no-thanks instead of depending on dbconfig-common (as used to
> be the case and still works). What this solves is multiple issues:
> 

This is great! Thanks Paul. I've never been very happy with
dbconfig-common because it kind of assumes databases are on the same
server as apps, which is increasingly not the case with smaller server
instances running in VMs, on the cloud and in containers.

> 0) Bug: 353617¹
> 
> 1) Because there is an alternative, dbconfig- can depend on the
> command-line client for dbtype, instead of  recommending
> it. Thus properly signifying the hard requirement of dbconfig-common to
> have the command-line client available.
> 
> 2) For multiple dbtype supported packages the system administrator now
> has a way outside of debconf to select which dbtype he wants by
> installing the right dbconfig- package. Currently the question
> will still be asked though.
> 
> 3) The system administrator now has a way to say no-thanks to
> dbconfig-common (by installing the dbconfig-no-thanks package) on the
> system level, rather than per package via debconf.
> 
> As a bonus, I can now add the database server packages to recommends,
> which should make life of the less experienced user easier. Do you think
> I should do this, or should I leave the database server package at the
> suggests level?
> 

I have mixed feelings about this. I understand that in the very basic
case, doing this means that the user gets to move forward without having
to think about the server package.

However, I also think that postinstall is not the right time to
be configuring your database, and I'd rather see guidance in the
documentation and the fine wizard that is dbconfig suggested as a CLI tool
for users to use once they have thought through their database options.
So adding it to the Recommends just adds mystery to this process and
doesn't actually help users find their way to best practices.

But I understand there are many who do not agree with this position,
so my thoughts on this may not resonate with you. :)

Re: dbconfig-common: near future change in dependency stack

[Clint Byrum]

Excerpts from Paul Gevers's message of 2015-12-06 11:48:41 -0800:
> Hi Clint,
> 
> On 06-12-15 18:22, Clint Byrum wrote:
> > I've never been very happy with
> > dbconfig-common because it kind of assumes databases are on the same
> > server as apps, which is increasingly not the case with smaller server
> > instances running in VMs, on the cloud and in containers.
> 
> I am totally flabbergasted. dbconfig-common goes through a lot of hoops
> just to make sure this is not true. Sure, the *default* questions and
> answers are geared towards local database servers. But by either
> (re-)configuring dbconfig-common to raise the priority level of remote
> questions by default, or by raising the priority level during package
> configuring will get you the questions needed to configure your remote
> database. (Of course we can discuss what the proper priorities are but I
> really hope no DD will think (anymore) that dbconfig-common assumes
> local databases.)
> 
> And yes, there is still bug 700895¹ (improve dbadmin-less database
> configuration workflow).

As I said, "kind of assumes [local]". I understand there are ways to
make it behave as well as possible. However, I, nor any of my peers
who deploy, things bigger than one server on a regular basis, want the
package manager to do anything this deep for me. We mostly want it to
get out of our way. I also understand that there are plenty of DD's who
do want these types of things done for them. I applaud our diversity of
opinions, and so will not push further than expressing my own position.

Re: dbconfig-common: near future change in dependency stack

[Paul Gevers]

Hi Clint,

On 06-12-15 18:22, Clint Byrum wrote:
> I've never been very happy with
> dbconfig-common because it kind of assumes databases are on the same
> server as apps, which is increasingly not the case with smaller server
> instances running in VMs, on the cloud and in containers.

I am totally flabbergasted. dbconfig-common goes through a lot of hoops
just to make sure this is not true. Sure, the *default* questions and
answers are geared towards local database servers. But by either
(re-)configuring dbconfig-common to raise the priority level of remote
questions by default, or by raising the priority level during package
configuring will get you the questions needed to configure your remote
database. (Of course we can discuss what the proper priorities are but I
really hope no DD will think (anymore) that dbconfig-common assumes
local databases.)

And yes, there is still bug 700895¹ (improve dbadmin-less database
configuration workflow).

Paul
¹ https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=700895



signature.asc
Description: OpenPGP digital signature