Upgrading dovecot 2.2 to 2.3 without downtime when using proxy/director?

[Niels Kobschätzki]

Hi,

I have a setup with several dovecot-servers (2.2.35) and I use dovecot
proxy. I upgraded one server to 2.3.1 and got the configs fixed so far
that it started again. But when I tried to add it into the proxying
again with "doveadm director add" I see the following in the logfiles:

May  6 07:19:30 host dovecot: director: Warning: Director
xxx.xxx.xxx.176:9090/out disconnected us with reason: Invalid input: OPTIONS
May  6 07:19:30 $host dovecot: director: Connecting to
xxx.xxx.xxx.171:9090 (as xxx.xxx.xxx.176): Reconnecting after error

xxx.xxx.xxx.176 is the upgraded host, xxx.xxx.xxx.171 is one of the
other hosts (the applicable hosts are in the range xxx.xxx.xxx.171-176).

Do I have to remove all the hosts from the proxy, upgrade them and then
add them again? Or is there a way to handle the upgrade without a
downtime? After all one of the reasons I use several hosts is that I do
not want to have a downtime.

Niels

2.3.1 Replication is throwing scary errors

[Andy Weal]

Hi all,

New to the mailing lists but have joined up because of above */2.3.1 
Replication is throwing scary errors



/*Brief system configuration
    MX1 - Main
        Freebsd 11.1-Release-p9
        Hosted on a Vultr VM in Sydney AU
        MTA = Postfix 3.4-20180401
        Dovecot = 2.3.1
        File system = ufs
    MX2 - Backup
        Freebsd 11.1-Release-p9
     Running on bare metal - no VM or jails
        MTA = Postfix 3.4-20180401
        Dovecot = 2.3.1
        File system = ufs ( on SSD)

*/
/*Brief sequence of events

1.      apx 10 days back upgraded both mx1 and mx2 to dovecot 2.3.1_2
   from 2.3.0   (service dovecot stop, portmaster upgrade, service
   dovecot start)
2.      both systems ran ok with no errors for 10 days.
3.      Last night I shutdown mx2 and restarted it a few hours later
4.      within minutes i was getting the following types of errors on mx2

 May 06 12:56:29 doveadm: Error: Couldn't lock 
/var/mail/vhosts/example.net/user1/.dovecot-sync.lock: 
fcntl(/var/mail/vhosts/example.net/user1/.dovecot-sync.lock, write-lock, 
F_SETLKW) locking failed: Timed out after 30 seconds (WRITE lock held by 
pid 1960)


  Before i venture down the rabbit hole of fault finding and excess 
coffee consumption I was wondering if any of you had any updates on the 
problems discussed below.



Cheers for now,
Andy



Hi,

[Formatting is a bit rough, replying from a trimmed digest email]

/Message: 1 />/Date: Fri, 6 Apr 2018 15:04:35 +0200 />/From: Michael Grimm > />/To: Dovecot Mailing List > />/Subject: Re: 2.3.1 Replication is throwing scary errors />/Message-ID: > />/Content-Type: text/plain; charset=utf-8 />//>/Reuben Farrelly wrote: />>/From: Michael Grimm > />//>>>/[This is Dovecot 2.3.1 at FreeBSD STABLE-11.1 running in two jails at 
distinct servers.] />>>/I did upgrade from 2.2.35 to 2.3.1 today, and I do become pounded by 
error messages at server1 (and vice versa at server2) as follows: />>>/| Apr 2 17:12:18  server1.lan dovecot: doveadm: Error: 
dsync(server2.lan): I/O has stalled, \ />>>/no activity for 600 seconds (last sent=mail_change, last 
recv=mail_change (EOL)) />>>/| Apr 2 17:12:18  server1.lan dovecot: doveadm: Error: Timeout 
during state=sync_mails \ />>>/(send=changes recv=mail_requests) />>>/[?] />>>/| Apr 2 18:59:03  server1.lan dovecot: doveadm: Error: 
dsync(server2.lan): I/O has stalled, \ />>>/no activity for 600 seconds (last sent=mail, last recv=mail (EOL)) />>>/| Apr 2 18:59:03  server1.lan dovecot: doveadm: Error: Timeout 
during state=sync_mails \ />>>/(send=mails recv=recv_last_common) />>>/I cannot see in my personal account any missing replications, *but* I 
haven't tested this thoroughly enough. I do have customers being 
serviced at these productive servers, *thus* I'm back to 2.2.35 until I 
do understand or have learned what is going on. />//>/In my reply to this statement of mine I mentioned that I have seen those 
timeouts quite some times during the past year. Thus, I upgraded to 
2.3.1 again, and boom: after some hours I ended up in hanging processes 
[1] like (see Remko's mail in addition) ... />//>/doveadm-server: [IP4/6  SOME/MAILBOX import:0/0] (doveadm-server) />//>/? at server2 paired with a file like ? />//>/-rw--- 1 vmail dovecot uarch 0 Apr 3 16:52 
/home/to/USER1/.dovecot-sync.lock />//>/Corresponding logfile entries at server2 are like ? />//>/Apr 3 17:10:49  server2.lan dovecot: doveadm: Error: Couldn't 
lock /home/to/USER1/.dovecot-sync.lock: \ />/fcntl(/home/to/USER1/.dovecot-sync.lock, write-lock, F_SETLKW) locking 
failed: Timed out after 30 seconds \ />/(WRITE lock held by pid 51110) />//>/[1] Even stopping dovecot will not end those processes. One has to 
manually kill those before restarting dovecot. />//>/After one day of testing 2.3.1 with a couple of those episodes of 
locking/timeout, and now missing mails depending with server your MUA 
will connect to, I went back to 2.2.35. After two days at that version I 
never had such an episode again. />//>>/It's not just you. This issue hit me recently, and it was impacting />>/replication noticeably. I am following git master-2.3 . />/[...] />>/There is also a second issue of a long standing race with replication />>/occurring somewhere whereby if a mail comes in, is written to disk, is />>/replicated and then deleted in short succession, it will reappear />>/again to the MUA. I suspect the mail is being replicated back from />>/the remote. A few people have reported it over the years but it's not />>/reliable or consistent, so it has never been fixed. />>/And lastly there has been an ongoing but seemingly minor issue />>/relating to locking timing out after 30s particularly on the remote />>/host that is being replicated to. I rarely see the problem on my />>/local disk where alm

Re: expiring mail from root's Maildirs?

[@lbutlr]

On 2018-05-05 (11:19 MDT), Aki Tuomi  wrote:
> 
> You could use tool like sudo or su to run mutt as non-root, instead of 
> running it as root?

Thanks, that did work. Took a little, but no more sent mail in ~root.

-- 
Vi Veri Veniversum Vivus Vici

Re: expiring mail from root's Maildirs?

[Benny Pedersen]

Larry Rosenman skrev den 2018-05-05 19:19:

What happens if you put the following in /root/.muttrc:
Set record='/file/to/put/mail/in'
?


indeed possible, but this file will only be possible to read from root 
user


if it can be readed by other, its a security problem

Re: expiring mail from root's Maildirs?

[Larry Rosenman]

The /file/to/put/mail/in?

Or ?

-- 
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 214-642-9640 E-Mail: larry...@gmail.com
US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106

On 5/5/18, 3:21 PM, "dovecot on behalf of @lbutlr" 
 wrote:

On 2018-05-05 (11:19 MDT), Larry Rosenman  wrote:
> 
> What happens if you put the following in /root/.muttrc:
> Set record='/file/to/put/mail/in'

I get a mail still owned by root.

-- 
May you live in interesting times

Re: expiring mail from root's Maildirs?

[@lbutlr]

On 2018-05-05 (11:19 MDT), Larry Rosenman  wrote:
> 
> What happens if you put the following in /root/.muttrc:
> Set record='/file/to/put/mail/in'

I get a mail still owned by root.

-- 
May you live in interesting times

Re: expiring mail from root's Maildirs?

[@lbutlr]

On 2018-05-05 (11:19 MDT), Aki Tuomi  wrote:
> 
> You could use tool like sudo or su to run mutt as non-root, instead of 
> running it as root?

Maybe. Not sure if piping output in front to su will work, but I can certainly 
try it. 

-- 
"Making music should not be left to the professionals." -  Michelle
Shocked

Re: expiring mail from root's Maildirs?

[Aki Tuomi]

> On 05 May 2018 at 20:14 "@lbutlr"  wrote:
> 
> 
> On 2018-05-05 (06:52 MDT), Benny Pedersen  wrote:
> > 
> >> And yet it is.
> > 
> > in muttrc you miss ~ or $(HOME)/something
> 
> No.
> 
> >> Root has been aliased for decades. This has no impact on where and how
> >> mutt stores the mail it sends as root out of root’s crontab.
> > 
> > root user can read mail files for all unix users, thats your fail, maybe 
> > crontab miss $(HOME) where it matters, but if it have and you start mutt as 
> > root it does not help, and you see the problem in allow root to do too much
> 
> There seems to be a basic misunderstanding here.
> 
> root has a crontab. Part of that crontab parses some system log files (that 
> only root has access to) for user data and generates an email in HTML. That 
> email is sent via mutt because the basic mail commands could not send an HTML 
> email properly.
> 
> Mutt stores that email in ~root, and the email comes from the root user 
> because that is who wins the crontab.
> 
> The $HOME fro the user is /root/
> 
> > i dont know mutt in detail, but its a fail to start mutt as root
> 
> Since the mail is generated out of root's cron, however the mail is sent, 
> that process is going to be started by root.

You could use tool like sudo or su to run mutt as non-root, instead of running 
it as root?

Aki

Re: expiring mail from root's Maildirs?

[Larry Rosenman]

What happens if you put the following in /root/.muttrc:
Set record='/file/to/put/mail/in'
?

-- 
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 214-642-9640 E-Mail: larry...@gmail.com
US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106

On 5/5/18, 12:15 PM, "dovecot on behalf of @lbutlr" 
 wrote:

On 2018-05-05 (06:52 MDT), Benny Pedersen  wrote:
> 
>> And yet it is.
> 
> in muttrc you miss ~ or $(HOME)/something

No.

>> Root has been aliased for decades. This has no impact on where and how
>> mutt stores the mail it sends as root out of root’s crontab.
> 
> root user can read mail files for all unix users, thats your fail, maybe 
crontab miss $(HOME) where it matters, but if it have and you start mutt as 
root it does not help, and you see the problem in allow root to do too much

There seems to be a basic misunderstanding here.

root has a crontab. Part of that crontab parses some system log files (that 
only root has access to) for user data and generates an email in HTML. That 
email is sent via mutt because the basic mail commands could not send an HTML 
email properly.

Mutt stores that email in ~root, and the email comes from the root user 
because that is who wins the crontab.

The $HOME fro the user is /root/

> i dont know mutt in detail, but its a fail to start mutt as root

Since the mail is generated out of root's cron, however the mail is sent, 
that process is going to be started by root.

> mutt must be started as some-other-unix-login

See above.

> the remaining fail is then in that users muttrc

There is nothing wrong in the root muttrc.

-- 
U is for UNA who slipped down a drain
V is for VICTOR squashed by a train

Re: expiring mail from root's Maildirs?

[@lbutlr]

On 2018-05-05 (06:52 MDT), Benny Pedersen  wrote:
> 
>> And yet it is.
> 
> in muttrc you miss ~ or $(HOME)/something

No.

>> Root has been aliased for decades. This has no impact on where and how
>> mutt stores the mail it sends as root out of root’s crontab.
> 
> root user can read mail files for all unix users, thats your fail, maybe 
> crontab miss $(HOME) where it matters, but if it have and you start mutt as 
> root it does not help, and you see the problem in allow root to do too much

There seems to be a basic misunderstanding here.

root has a crontab. Part of that crontab parses some system log files (that 
only root has access to) for user data and generates an email in HTML. That 
email is sent via mutt because the basic mail commands could not send an HTML 
email properly.

Mutt stores that email in ~root, and the email comes from the root user because 
that is who wins the crontab.

The $HOME fro the user is /root/

> i dont know mutt in detail, but its a fail to start mutt as root

Since the mail is generated out of root's cron, however the mail is sent, that 
process is going to be started by root.

> mutt must be started as some-other-unix-login

See above.

> the remaining fail is then in that users muttrc

There is nothing wrong in the root muttrc.

-- 
U is for UNA who slipped down a drain
V is for VICTOR squashed by a train

Re: expiring mail from root's Maildirs?

[Benny Pedersen]

And yet it is.


in muttrc you miss ~ or $(HOME)/something


root: some-other-unix-login



Root has been aliased for decades. This has no impact on where and how
mutt stores the mail it sends as root out of root’s crontab.


root user can read mail files for all unix users, thats your fail, maybe 
crontab miss $(HOME) where it matters, but if it have and you start mutt 
as root it does not help, and you see the problem in allow root to do 
too much


i dont know mutt in detail, but its a fail to start mutt as root

mutt must be started as some-other-unix-login

the remaining fail is then in that users muttrc