John Fawcett via Postfix-users:
> Hi Wietse
>
> it seems like a sound approach. I was only wondering one thing: instead
> of an on-demand conversion, would it be possible to do the conversion
> when postfix is started and then do away with the need for having a
> separate daemon that will run for ever after? I am assuming that it is
> possible to identify all files needing conversion by reading the
> existing configuration.
The problem is 'discovery'. If you want to convert databases (and
therefore, the Postfix configurations that use them) before Postfix
programs and daemons can access those databases, then you are asking
for the rejected approach to parse/modify/unparse Postfix configuration
files. That is not impossible, it is just more complex code.
The proposed approach 'discovers' databases by the 'open' requests
that Postfix commands or daemons make. An unprivileged Postfix
daemon cannot re-index a database that must be owned by root.
The downside of the helper daemmon is that 'postmap' commands
that specify a legacy database can't access the helper daemon if
Postfix is not running. For that, some logic would have to be
duplicated in the hash/btree client code.
> For cache files with no source file, since it is an optimization and
> postfix can run fine even if they are removed, I would just start from
> scratch with the new db type after logging an appropriate warning about
> it, leaving any other migration needs to the sysadmin if they think it
> is worth it.
Start from scratch is all that one can do on a system without
Berkeley DB support.
Wietse
> On 04/01/2026 23:47, Wietse Venema via Postfix-users wrote:
> > The removal of Berkeley DB from Linux systems will affect a large
> > number of Postfix installations. Some (smaller) Linux distributions
> > have already removed Berkeley DB support, and RedHat will remove
> > it in Enterprise Linux version 10.
> >
> > The goal: a Postfix configuration that uses 'hash' or 'btree' can
> > be migrated to a configuration that does not use Berkeley DB support.
> > This would require automatic transformation of configuration files
> > (and automatic database re-indexing), or emulation where a request
> > for an obsolete database type is automatically serviced with a
> > supported type.
> >
> > I have considered, and rejected, automatic transformation of
> > configuration files. The problem is that database types (such as
> > hash or btree) and database pathnames may be the result of $name
> > expansion; to do a proper job one has to parse configuration files
> > like Postfix programs do, replace each legacy database type, unparse
> > the result, and then write an updated setting back to a Postfix
> > configuration file.
> >
> > That leaves an implementation based on emulation; I opted for a
> > variant with on-demand database re-indexing in the background. This
> > does not require that configuration files are updated, though making
> > such updates would save a few CPU cycles later.
> >
> > The basic idea:
> >
> > - A Postfix daemon or command (called 'database client' below) wants
> > to open a database with a legacy name hash:/path/to/file or
> > btree:/path/to/file. The 'postmap' command could be such a client.
> >
> > - The 'hash' and 'btree' database client implementation sends the
> > legacy name to a privileged helper daemon that replies with a
> > 'new' database name (e.g., cdb:/path/to/file or lmdb:/path/to/file,
> > but ONLY IF the legacy name is legitimate (listed in $proxy_read_maps
> > or $proxy_write_maps). This requires that proxy_read_maps and
> > proxy_write_maps are left at their default settings.
> >
> > - If /path/to/file exists (i.e. the table source data), but the
> > '.cdb' or '.lmdb' indexed file does not yet exist, the helper
> > daemon also runs 'postmap' or postalias' to create the '.cdb' or
> > '.lmdb' indexed file (unless the database client uses O_TRUNC or
> > O_CREAT) before replying to the database client. The helper daemon
> > must be privileged because many database files must be owned by
> > 'root'. The 'postmap' and postalias' commands already already
> > drop 'root' privileges when indexing a file that is owned by a
> > non-root user.
> >
> > - The database client receives the replacement database name from
> > the helper daemon and opens that database. The old hash and btree
> > '.db' files are not used.
> >
> > Nitty-gritty details:
> >
> > - This does not re-index cache files (verify, postscreen) because
> > there is no table source file. One would need Berkeley DB support
> > to list the entries in the obsolete file, and that is not possible
> > on systems without Berkeley DB support. But it might be feasible
> > if the migration is done on a system that still supports Berkeley
> > DB. I'll have to think about that. Cache files are populated by
> > Postfix daemons and they are only an optimization.
> >
> > - The helper service will run 'postalias' if a database matches
> > $alias_maps or $alias_database, otherwise it will run 'postmap';
> > and it will specify a file type of $default_database_type if the
> > database type:/path/to/file are listed in $proxy_read_maps, or
> > $default_cache_db_type if the type:/path/to/file are listed in
> > $proxy_write_maps. The proxy_*_maps parameters are specified in
> > a file (main.cf or master.cf) that is writable only be root. This
> > guarantees that the helper daemon will not touch other files.
> >
> > - The helper service will be single-threaded to avoid collisions
> > when it re-indexes a database file.
> >
> > After a table is re-indexed, the helper daemon will occasionally log
> > a reminder that the table has been migrated. The user can then at
> > their convenience update Postfix configuration files and replace
> > the obsolete name (hash:/path/to/file or btree:/path/to/file) with
> > the new name (e.g., cdb:/path/to/file or lmdb:/path/to/file), and
> > they can delete the obsolete hash or btree '.db' file. But is is
> > not the end of the world if the user ignores the reminders. Postfix
> > will keep working just fine.
> >
> > For maximal benefit the helper service should be accessible by local
> > programs that are not run by 'root' or 'postfix' users. For example,
> > Mailman3 wants to run 'postmap' commands after adding or removing
> > a mailing list. There may be other programs that also want to run
> > postmap or postalias commands.
> >
> > BTW Mailman3 was updated 5 months ago to allow Postfix databases
> > other than 'hash' or 'regexp'. Postfix emulation may be needed until
> > Mailman3 has been adopted by distributions.
> >
> > If we make the UNIX-domain helper socket world-accessible, then a
> > local process can attack the service and degrade Postfix performance,
> > but they cannot use the service to gain 'postfix' or 'root' privileges.
> >
> > This should require very little new code; if we do not count the
> > boilerplater code, likely less than the size of this message. It
> > will definitely be in Postfix 3.11. If people think this will be
> > useful, then I will consider back porting.
> >
> > Wietse
> > _______________________________________________
> > Postfix-users mailing list -- [email protected]
> > To unsubscribe send an email to [email protected]
> _______________________________________________
> Postfix-users mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
>
_______________________________________________
Postfix-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
