k8s: auto reload after cert renewal
Running postfix in k8s and using cert-manger to manage certificates it would be nice if postfix could pickup new certificates for long running processes like smtpd. Much like it picks up updated databases like those managed by postmap. I do not see any mention of this in man 5 postconf. For now I run a cronjob to just restart postfix every day, I gues with some serious hacking I could get postfix reload as a sidecar running in a loop but that would also be suboptimal. Any thoughts on why postfix cannot pick this up automatically? -- Leo Baltus, DevOps engineer serviced...@npo.nl, 035-6773555
Re: $daemon_directory [Re: upgrade to 2.10.1: pass_accept_attr: cannot receive connection attributes: Numerical result out of range'
Op 23/08/2013 om 09:51:07 -0400, schreef Wietse Venema:
> Wietse Venema:
> > Leo Baltus:
> > > Op 19/08/2013 om 13:11:04 -0400, schreef Wietse Venema:
> > > > Leo Baltus:
> > > > > > > However, I did notice that postfix exec()'s new processes using
> > > > > > > the
> > > > > > > path to the binaries it got from
> > > > > > > 'PATH=symlink_to_postfix/sbin postfix start'
> > > > > > > instead of compile-time arguments DEF_COMMAND_DIR DEF_DAEMON_DIR
> > > > > > > etc.
> > > > > >
> > > > > > The Postfix master(8) daemon executes programs with pathnames
> > > > > > that are derived from the $daemon_directory value.
> > > > > >
> > > > >
> > > > > Where $daemon_directory defaults to DEF_DAEMON_DIR?
> > > >
> > > > The Postfix installation/upgrade procedure, as distributed by me,
> > > > always sets daemon_directory in main.cf. Therefore, that setting
> > > > takes precedence over DEF_DAEMON_DIR.
> > > >
> > >
> > > We do not set daemon_directory etc. in main.cf because we like it to be
> > > set
> > > to DEF_DAEMON_DIR. I hope this is a supported feature because it allows
> > > us to quickly rollback changes, like in this case.
> >
> > You break the warranty, therefore you own all the problems that
> > result from not using (or from changing) the Postfix installation
> > or upgrade procedure.
>
> Instead of modding the install/upgrade scripts, you could symlink
> daemon_directory, command_directory, sendmail_path, mailq_path,
> manpath, and so on and avoid braking the warranty.
>
Changing the symlinks while running postfix would immediatly affect
newly forked processes, which is what I am trying to avoid.
Also the symlink is used by several instances on the machine, so
changing it would affect all of them.
> However, you must run "postfix upgrade-configuration" when you
> switch from an older Postfix version to a newer one. Otherwise
> there is no guarantee that it will work properly.
>
> Switching from a newer release to an older one requires careful
> backing out of configuration changes. For example, files with long
> queue ID names will not be recognized by Postfix versions that
> pre-date long queue ID support. You have to revert to old queue IDs
> as documented in the RELEASE_NOTES file.
>
> Wietse
>
> > > After examining further I found in the environment of the master process
> > > that with postfix-2.9.6 daemon_directory is set to $PATH/libexec and
> > > command_directory is set to $PATH/sbin, however sendmail_path,
> >
> > Postfix as distributed by me does not prepend the PATH environment
> > to its program names, and no version of Postfix has done this.
I think I found what is going on.
We start postfix:
env -i PATH=/local/pf/sbin postfix -c /my/mx/main.cf start
Where /local/pf -> /software/postfix-x.y
Before this however we run
/local/pf/libexec/post-install queue_directory=... create-missing
so the neccesary directories are created. I must say that I find it
redundant, our management system should have created them in the
first place.
It seems that this command *changes* main.cf and adds daemon_directory
using the path it got from starting postfix:
eval test \"\$$name\" = \"`$POSTCONF -c $config_directory -h $name`\" || {
with daemon_directory it evaluates to
test "/local/pf/libexec" = "/software/postfix-x.y/libexec" || {
and so it decides to change main.cf
master now uses /local/pf/libexec as its daemon_directory.
Now, our management system regularly overwrites all configuration so I
wasn't aware of this change.
After a stop and start, without post-install because all directories are
there now, master uses the compiled-in daemon_directory.
So, I think I know what is going on and how to stop postfix from
changing main.cf.
I am aware that this is by no means a standard set-up so I apologize for
the confusion this may have caused.
--
Leo Baltus, internetbeheerder /\
NPO ICT Internet Services/NPO/\
Sumatralaan 45, 1217 GP Hilversum, Filmcentrum, west \ /\/
serviced...@omroep.nl, 035-6773555\/
Re: $daemon_directory [Re: upgrade to 2.10.1: pass_accept_attr: cannot receive connection attributes: Numerical result out of range
Op 19/08/2013 om 13:11:04 -0400, schreef Wietse Venema: > Leo Baltus: > > > > However, I did notice that postfix exec()'s new processes using the > > > > path to the binaries it got from > > > > 'PATH=symlink_to_postfix/sbin postfix start' > > > > instead of compile-time arguments DEF_COMMAND_DIR DEF_DAEMON_DIR etc. > > > > > > The Postfix master(8) daemon executes programs with pathnames > > > that are derived from the $daemon_directory value. > > > > > > > Where $daemon_directory defaults to DEF_DAEMON_DIR? > > The Postfix installation/upgrade procedure, as distributed by me, > always sets daemon_directory in main.cf. Therefore, that setting > takes precedence over DEF_DAEMON_DIR. > We do not set daemon_directory etc. in main.cf because we like it to be set to DEF_DAEMON_DIR. I hope this is a supported feature because it allows us to quickly rollback changes, like in this case. After examining further I found in the environment of the master process that with postfix-2.9.6 daemon_directory is set to $PATH/libexec and command_directory is set to $PATH/sbin, however sendmail_path, mailq_path manpage_directory and newaliases_path seem to follow the compile- time defaults (DEF_SENDMAIL_PATH, DEF_MAILQ_PATH and DEF_NEWALIAS_PATH) Running postfix-2.10.1 they all seem to point to the compile-time settings. So it seems this is working now as I expected but not so in the past, although I am convinced that it used to be like this. -- Leo Baltus, internetbeheerder /\ NPO ICT Internet Services/NPO/\ Sumatralaan 45, 1217 GP Hilversum, Filmcentrum, west \ /\/ serviced...@omroep.nl, 035-6773555\/
$daemon_directory [Re: upgrade to 2.10.1: pass_accept_attr: cannot receive connection attributes: Numerical result out of range
Op 19/08/2013 om 10:14:40 -0400, schreef Wietse Venema: > Leo Baltus: > > However, I did notice that postfix exec()'s new processes using the > > path to the binaries it got from > > 'PATH=symlink_to_postfix/sbin postfix start' > > instead of compile-time arguments DEF_COMMAND_DIR DEF_DAEMON_DIR etc. > > The Postfix master(8) daemon executes programs with pathnames > that are derived from the $daemon_directory value. > Where $daemon_directory defaults to DEF_DAEMON_DIR? DEF_DAEMON_DIR is set compile time should be constant. I set it to e.g. /software/postfix-2.10.1/libexec > master_ent.c:serv->path = concatenate(var_daemon_dir, "/", command, (char > *) 0); > master_spawn.c: execvp(serv->path, serv->args->argv); > > The kernel resolves the pathname each time Postfix invokes execvp(). > > > I may have been playing with symlink_to_postfix while hunting this down. > > If you swap symlinks, then the result of execvp() will change. > But that would imply that postfix uses a runtime value ($PATH?). I used to think it would use /software/postfix-2.10.1/libexec untill a new version comes along and that master would start using /software/postfix-x.y/libexec -- Leo Baltus, internetbeheerder /\ NPO ICT Internet Services/NPO/\ Sumatralaan 45, 1217 GP Hilversum, Filmcentrum, west \ /\/ serviced...@omroep.nl, 035-6773555\/
Re: upgrade to 2.10.1: pass_accept_attr: cannot receive connection attributes: Numerical result out of range
Op 19/08/2013 om 12:12:49 +, schreef Viktor Dukhovni: > On Mon, Aug 19, 2013 at 01:28:45PM +0200, Leo Baltus wrote: > > > We are upgrading our postfix instances from 2.9.6 to 2.10.1. > > Have you considered reading the release notes (for Postfix 2.10)? > Well, I did. I didn't think it would apply because we always shut down the old version before starting the new one. > Major changes - load-balancer support > - > > [Incompat 20120625] The postscreen(8)-to-smtpd(8) protocol has > changed. To avoid "cannot receive connection attributes" warnings > and dropped connections, execute the command "postfix reload". No > mail will be lost as long as the remote SMTP client tries again > later. > > The reload should be performed immediately after installing the new > binaries. If you prefer you can stop Postfix before the upgrade, and > restart after (that's what I would do). > On re-reading I should have given it more thought. We do not overwrite old binaries, instead we place them parallel. We stop the old instance (postfix stop) and start (postfix start), so postscreen and smtpd will always be running from the same software repository. I would say that that should take care of this issue. However, I did notice that postfix exec()'s new processes using the path to the binaries it got from 'PATH=symlink_to_postfix/sbin postfix start' instead of compile-time arguments DEF_COMMAND_DIR DEF_DAEMON_DIR etc. I may have been playing with symlink_to_postfix while hunting this down. Maybe this has changed in the past or I always misunderstood. Should I reconsider our stopping/starting approach? Or is this unrelated? -- Leo Baltus, internetbeheerder /\ NPO ICT Internet Services/NPO/\ Sumatralaan 45, 1217 GP Hilversum, Filmcentrum, west \ /\/ serviced...@omroep.nl, 035-6773555\/
upgrade to 2.10.1: pass_accept_attr: cannot receive connection attributes: Numerical result out of range
Hi, We are upgrading our postfix instances from 2.9.6 to 2.10.1. On our moderately busy (800 connections/minute) incoming mailservers I noticed that postscreen did not pass connections through to smtpd- instances for some 90 seconds. Here's the semi-automated stop and start: Aug 19 12:31:46 host02 mx0/master[12831]: terminating on signal 15 Aug 19 12:31:46 host02 mx0/postfix-script[16590]: starting the Postfix mail system During the period of outage I see: Aug 19 12:33:18 host02 mx0/postscreen[16598]: PASS OLD [157.55.2.20]:50154 Aug 19 12:33:19 host02 mx0/postscreen[16598]: PASS NEW [59.40.157.135]:1230 Aug 19 12:33:19 host02 mx0/postscreen[16598]: PASS OLD [64.79.84.178]:52424 Aug 19 12:33:19 host02 mx0/postscreen[16598]: PASS OLD [88.148.95.6]:1555 Aug 19 12:33:20 host02 mx0/postscreen[16598]: PASS NEW [85.243.61.94]:12852 Aug 19 12:33:21 host02 mx0/postscreen[16598]: CONNECT from [212.54.34.165]:57502 to [145.58.30.181]:25 Aug 19 12:33:21 host02 mx0/postscreen[16598]: PASS OLD [212.54.34.165]:57502 Aug 19 12:33:21 host02 mx0/postscreen[16598]: CONNECT from [95.87.61.111]:1735 to [145.58.30.181]:25 Aug 19 12:33:21 host02 mx0/postscreen[16598]: PASS OLD [95.87.61.111]:1735 Aug 19 12:33:22 host02 mx0/postscreen[16598]: CONNECT from [213.135.174.189]:64295 to [145.58.30.181]:25 Aug 19 12:33:22 host02 mx0/postscreen[16598]: CONNECT from [200.124.239.209]:59105 to [145.58.30.181]:25 Aug 19 12:33:22 host02 mx0/postscreen[16598]: PASS OLD [200.124.239.209]:59105 Aug 19 12:33:22 host02 mx0/postscreen[16598]: CONNECT from [200.124.239.209]:59104 to [145.58.30.181]:25 Aug 19 12:33:22 host02 mx0/postscreen[16598]: PASS OLD [200.124.239.209]:59104 Aug 19 12:33:23 host02 mx0/postscreen[16598]: CONNECT from [200.92.242.208]:41357 to [145.58.30.181]:25 Aug 19 12:33:23 host02 mx0/postscreen[16598]: PASS OLD [200.92.242.208]:41357 Aug 19 12:33:24 host02 mx0/postscreen[16598]: CONNECT from [82.166.162.225]:52138 to [145.58.30.181]:25 Aug 19 12:33:24 host02 mx0/postscreen[16598]: CONNECT from [82.166.162.225]:52139 to [145.58.30.181]:25 Aug 19 12:33:24 host02 mx0/postscreen[16598]: CONNECT from [82.166.162.225]:52140 to [145.58.30.181]:25 So nothing gets accepted. Connecting to it at that time results in a connection without response: $ swaks -s mx0.mail.omroep.nl -t ... -f ... === Trying mx0.mail.omroep.nl:25... === Connected to mx0.mail.omroep.nl. <** Timeout (30 secs) waiting for server response Looking in the logs I found: Aug 19 12:33:30 host02 mx0/postscreen[16515]: warning: timeout sending connection to service private/smtpd4 Aug 19 12:33:30 host02 mx0/smtpd[16609]: warning: pass_accept_attr: cannot receive connection attributes: Numerical result out of range Aug 19 12:33:30 host02 mx0/smtpd[16609]: error: accept connection: Numerical result out of range Aug 19 12:33:30 host02 mx0/postscreen[16598]: warning: timeout sending connection to service private/smtpd4 Aug 19 12:33:30 host02 mx0/postscreen[16598]: warning: timeout sending connection to service private/smtpd4 After this postfix seem to behave as expected. I have not made any configuration changes, not even smtpd_relay_restrictions so the compiled-in default should apply. I have 2 postscreen instances running from master.cf, one for ipv4 and another for ipv6: 145.58.30.181:25inet n - n - 1 postscreen -o smtpd_service_name=smtpd4 -o postscreen_cache_map=$data_directory/postscreen_cache_v4 2a02:458:101:30::b5:25 inet n - n - 1 postscreen -o smtpd_service_name=smtpd6 -o postscreen_cache_map=$data_directory/postscreen_cache_v6 This seems to occur only when upgrading. A stop/start with the same version, either 2.9.6 of 2.10.1, seems to work without a problem. Does anyone have an explanation for this? -- Leo Baltus, internetbeheerder /\ NPO ICT Internet Services/NPO/\ Sumatralaan 45, 1217 GP Hilversum, Filmcentrum, west \ /\/ serviced...@omroep.nl, 035-6773555\/
db.h in a non-standard place
Hi, Our Linux compile environment has changed so that db.h is not in a standard place any more. However makedefs seem to insist that is should be in /usr/include My initial make reads something like so: make -f Makefile.init makefiles CCARGS="-DHAS_DB -I$db/include" \ AUXLIBS="-L$db/lib -ldb" Like http://www.postfix.org/DB_README.html tells me. Attached patch detects '-DHAS_DB' and just skips further tests. Would that be acceptable for future releases? -- Leo Baltus, internetbeheerder /\ NPO ICT Internet Services/NPO/\ Sumatralaan 45, 1217 GP Hilversum, Filmcentrum, west \ /\/ serviced...@omroep.nl, 035-6773555\/ --- postfix-2.10.1/makedefs.orig2013-08-01 15:07:56.0 +0200 +++ postfix-2.10.1/makedefs 2013-08-01 15:09:01.0 +0200 @@ -278,6 +278,7 @@ Linux.2*) SYSTYPE=LINUX2 case "$CCARGS" in *-DNO_DB*) ;; +*-DHAS_DB*) ;; *) if [ -f /usr/include/db.h ] then : we are all set @@ -350,6 +351,7 @@ Linux.3*) SYSTYPE=LINUX3 case "$CCARGS" in *-DNO_DB*) ;; +*-DHAS_DB*) ;; *) if [ -f /usr/include/db.h ] then : we are all set
Re: not logging to syslog
Op 10/07/2012 om 06:55:43 -0400, schreef Wietse Venema: > Leo Baltus: > > It would be nice if postfix could jut open a logfile and reopen on a > > signal like any other daemon does. > > Sending sighup to sendmail, postsuper, postqueue, etc. would not > be productive. > Postfix could have its own daemon to keep the logfile open and have its own interface for receiving whatever is necessary to reopen it. Just thinking out loud here, I know this is not in postfix now. -- Leo Baltus, internetbeheerder /\ NPO ICT Internet Services/NPO/\ Sumatralaan 45, 1217 GP Hilversum, Filmcentrum, west \ /\/ beh...@omroep.nl, 035-6773555 \/
Re: not logging to syslog
Op 09/07/2012 om 13:52:04 -0400, schreef Wietse Venema: > Leo Baltus: > > I would like postfix to not log to the default syslog-daemon to > > have better control over where each specific postfix instance logs to. I > > am running multiple instances on a server. > > Postfix has syslog_facility to split different syslog streams. syslog facilities are quite limited, only 24 of which only 8 local. It would be nice if postfix could jut open a logfile and reopen on a signal like any other daemon does. > > Otherwise propose a new syslog API and get everyone to adopt it. > I believe many have tried. Journal/Journald looks promising but is too much depending on systemd if you ask me. -- Leo Baltus, internetbeheerder /\ NPO ICT Internet Services/NPO/\ Sumatralaan 45, 1217 GP Hilversum, Filmcentrum, west \ /\/ beh...@omroep.nl, 035-6773555 \/
not logging to syslog
Hi, I would like postfix to not log to the default syslog-daemon to have better control over where each specific postfix instance logs to. I am running multiple instances on a server. Postfix only knows about logging to syslog which is unfortunate, although I have read that someday postfix will change: Op 23/02/2010 om 09:18:29 -0500, schreef Wietse Venema: > At some point, the logging of Postfix will be configurable. It's > one of the first-generation pieces of code that is completely > hard-coded, as part of the drive to get a mail system released. Anyway, to separate the logs of each instance I could run almost all postfix services chrooted (/foo/bar) and then start a syslogd for each postfix instance listening in /foo/bar/dev/log Questions: - would this setup work? - would postfix still 'leak' messages to /dev/log? PS Wietse: +1 for this configurable logging subsystem -- Leo Baltus, internetbeheerder /\ NPO ICT Internet Services/NPO/\ Sumatralaan 45, 1217 GP Hilversum, Filmcentrum, west \ /\/ beh...@omroep.nl, 035-6773555 \/
Re: complementary groups for mail_owner
Hi Wietse, Op 31/07/2008 om 11:52:18 -0400, schreef Wietse Venema: > Unfortunately (for you), Postfix currently does not use supplementary > groups, anywhere. It's not a quick hack to change this. > I can wait :-) Could you elaborate on this, is this a design decision you made? -- Leo Baltus, internetbeheerder /\ NPO ICT Internet Services/NPO/\ Sumatralaan 45, 1217 GP Hilversum, Filmcentrum, west \ /\/ [EMAIL PROTECTED], 035-6773555 \/
complementary groups for mail_owner
Hi, We just started rolling out 2.5.3. On starting up we see: fatal: open lock file $data_directory/master.lock This seems to be a result of (HISTORY): 20080220 Safety: the master daemon now sets an exclusive lock on a file $data_directory/master.lock, so that the data directory can't be shared between multiple Postfix instances. This would corrupt files that rely on single-writer updates (examples: verify(8) cache, tlsmgr(8) caches, etc.). File: master/master.c. It appears that if the master sets the lock it does this as mail_owner but does not have its supplementary groups set, according to /etc/groups. Our $data_directory is a path consisting of several subdirectories, one of them having permissions 750 group set at a supplementary group. Group membership of this group gives access to sysadmins and cronjobs and such. I could set this group as the primary group for mail_owner, but that would break consistency as we embrace the idea of each uid having its own unique gid. Now, is there a compelling reason why mail_owner is not allowed to have supplementary groups? -- Leo Baltus, internetbeheerder /\ NPO ICT Internet Services/NPO/\ Sumatralaan 45, 1217 GP Hilversum, Filmcentrum, west \ /\/ [EMAIL PROTECTED], 035-6773555 \/
