Re: dsync with vurtial folder leads to mess of msg ID

2024-05-24 Thread Kirill A. Korinsky via dovecot
Greetings,

On Sun, 21 Apr 2024 21:52:41 +0100,
Kirill A. Korinsky  wrote:
> 
> Excluding INBOX from virtual folder seems that allows to avoid the issue. 
> 

I'd like to confirm that excluding INBOX from virtual folder indeed allows
to avoid that issue.

Any suggestion how can I help to debug that issue future?

-- 
wbr, Kirill
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org


Re: dsync with vurtial folder leads to mess of msg ID

2024-04-21 Thread Kirill A. Korinsky via dovecot
Greeting,

Since last status update I've switched my workflow from active using
virtual.All into virtual.Archive folder.

After the switch I've recreated virtual.All on both servers.

Folder definition:

  mx1$ cat /etc/dovecot/virtual/Archive/dovecot-virtual 
 
  !Archive
  Archive*
   all
  mx1$ cat /etc/dovecot/virtual/All/dovecot-virtual  
  *
  !Archive
  -Trash*
  -Junk*
all
  mx1$

an output for doveadm fetch "uid guid" -u kir...@korins.ky mailbox FOLDER | md5

  Archive
  mx1: 176fdedf6fe641b5580b5deeb3244539
  mx2: 176fdedf6fe641b5580b5deeb3244539
  All
  mx1: 4c1fcd6349bee770f2fbb8dc1bbe42a2
  mx2: 9af53aa8fdbfadf5d1d5cf01daef5617

Excluding INBOX from virtual folder seems that allows to avoid the issue. 

-- 
wbr, Kirill
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org


Re: dsync with vurtial folder leads to mess of msg ID

2024-04-18 Thread Kirill A. Korinsky via dovecot
Greeting,

On Thu, 18 Apr 2024 17:51:06 +0200,
Kirill A. Korinsky wrote:
> 
> All of this allows to reconstruct the workflow which leads to an issue:
>  1. mx2 had a new mail.
>  2. mx2 registered that mail as 147699 inside virtual folder.
>  3. by some reason this email wasn't replicated to mx1 for at least
> 30-40 seconds. Maybe it runs full-scan on all users, I have no idea.
>  4. mx1 had a new mail as well.
>  5. mx1 resitered that mail as 147699 inside virtual folder.
>  6. mx1's mail was replicated to mx2 as 147700
>  7. mx2's mail was replicated back to mx1 as 147701
> 

I was wrong. I run a series of expirements and achived a strategy which
allows to reproduce this issue.

I've used the following definition of virtual folder all:

  *
  !Archive
  -Trash
  -Trash/*
  -Junk
  -Junk/*
all

I also have two servers: mx1 and mx2.

So, to reproduce an issue with dsync msgid inside virtual folder I need:
 1. Connect to mx1 via imap
 2. Connect to mx2 via smtp
 3. Initialize sending a mail via smtp
 4. In parallel move some mail from Inbox to Virtual.All

With quite big probability I broke the order.

When I enable the debug log, it slow down things because dovecot and smtpd
uses the same /var/log/maillog on my system.

Another way to avoid such condition, is switch off multi core on machine.

Without parallerism I can't reproduce the issue.

-- 
wbr, Kirill
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org


Re: dsync with vurtial folder leads to mess of msg ID

2024-04-18 Thread Kirill A. Korinsky via dovecot
Greetings,

Seems that I had catched a proof that it is defently bug in dsync.

On Thu, 18 Apr 2024 16:59:23 +0200,
Kirill A. Korinsky wrote:
> 
> Anyway, I'll be back.
> 

As usual after I sent an update here, the issue is comeback.

  mx1# doveadm fetch "uid guid" -u kir...@korins.ky mailbox virtual.All | tail 
-n 30 


  uid: 147696
  guid: 1713451338.M678759P24080.mx2.catap.net,S=5182,W=5299


  uid: 147699
  guid: 1713452322.M905815P78454.mx2.catap.net,S=6036,W=6143


  uid: 147700
  guid: 1713452365.M594673P95241.mx1.catap.net,S=3958,W=4066


  uid: 147701
  guid: 1713452405.M535379P78454.mx2.catap.net,S=3351,W=3431


  uid: 147705
  guid: 1713453074.M691924P5954.mx1.catap.net,S=6333,W=6437


  uid: 147706
  guid: 1713453359.M591908P38204.mx1.catap.net,S=54792,W=56096


  uid: 147707
  guid: 1713453469.M178617P70020.mx2.catap.net,S=2406,W=2453


  uid: 147708
  guid: 1713452587.M931589P64530.mx1.catap.net,S=5044,W=5161


  uid: 147709
  guid: 1713453576.M23051P30161.mx1.catap.net,S=2044,W=2090


  uid: 147710
  guid: 1713452952.M729802P32094.mx1.catap.net,S=4873,W=5017
  mx1#

vs

  mx2# doveadm fetch "uid guid" -u kir...@korins.ky mailbox virtual.All | tail 
-n 30 


  uid: 147696
  guid: 1713451338.M678759P24080.mx2.catap.net,S=5182,W=5299


  uid: 147699
  guid: 1713452365.M594673P95241.mx1.catap.net,S=3958,W=4066


  uid: 147701
  guid: 1713452322.M905815P78454.mx2.catap.net,S=6036,W=6143


  uid: 147702
  guid: 1713452405.M535379P78454.mx2.catap.net,S=3351,W=3431


  uid: 147707
  guid: 1713452587.M931589P64530.mx1.catap.net,S=5044,W=5161


  uid: 147708
  guid: 1713452952.M729802P32094.mx1.catap.net,S=4873,W=5017


  uid: 147709
  guid: 1713453074.M691924P5954.mx1.catap.net,S=6333,W=6437


  uid: 147710
  guid: 1713453359.M591908P38204.mx1.catap.net,S=54792,W=56096


  uid: 147711
  guid: 1713453469.M178617P70020.mx2.catap.net,S=2406,W=2453


  uid: 147712
  guid: 1713453576.M23051P30161.mx1.catap.net,S=2044,W=2090
  mx2#

as you may notice differences at begining of output:

mx1:
  uid: 147696
  guid: 1713451338.M678759P24080.mx2.catap.net,S=5182,W=5299

  uid: 147699
  guid: 1713452322.M905815P78454.mx2.catap.net,S=6036,W=6143

  uid: 147700
  guid: 1713452365.M594673P95241.mx1.catap.net,S=3958,W=4066

mx2:
  uid: 147696
  guid: 1713451338.M678759P24080.mx2.catap.net,S=5182,W=5299

  uid: 147699
  guid: 1713452365.M594673P95241.mx1.catap.net,S=3958,W=4066

  uid: 147701
  guid: 1713452322.M905815P78454.mx2.catap.net,S=6036,W=6143


and here the collistion in email with id 147699 which is different on
different servers.

Funny things that 1713452365.M594673P95241.mx1.catap.net,S=3958,W=4066 is my
email to the mail list.

Let see the logs for 16:58 and 16:59 on both servers.

mx1:

  Apr 18 16:58:36 mx1 dovecot: imap-login: Login: user=, 
method=PLAIN, rip=90.187.54.213, lip=162.55.82.72, mpid=33482, TLS, 
session=
  Apr 18 16:58:42 mx1 dovecot: imap(k...@catap.net)<33482>: 
Disconnected: Logged out in=529 out=100334 deleted=0 expunged=0 trashed=0 
hdr_count=5 hdr_bytes=16964 body_count=3 body_bytes=77607
  Apr 18 16:59:24 mx1 smtpd[99113]: 7824635eb33a7cc5 smtp connected 
address=90.187.54.213 host=sa31.catap.net
  Apr 18 16:59:24 mx1 smtpd[99113]: 7824635eb33a7cc5 smtp tls 
ciphers=TLSv1.3:TLS_AES_256_GCM_SHA384:256
  Apr 18 16:59:25 mx1 smtpd[99113]: 7824635eb33a7cc5 smtp authentication 
user=kir...@korins.ky result=ok
  Apr 18 16:59:25 mx1 smtpd[99113]: 7824635eb33a7cc5 smtp message 
msgid=12820125 size=3979 nrcpt=3 proto=ESMTP
  Apr 18 16:59:25 mx1 smtpd[99113]: 7824635eb33a7cc5 smtp envelope 
evpid=128201259a418712 from= to=
  Apr 18 16:59:25 mx1 smtpd[99113]: 7824635eb33a7cc5 smtp envelope 
evpid=12820125e2eb8c1d from= to=
  Apr 18 16:59:25 mx1 smtpd[99113]: 7824635eb33a7cc5 smtp envelope 
evpid=12820125cea959e4 from= to=
  Apr 18 16:59:25 mx1 dovecot: lmtp(95241): Connect from local
  Apr 18 16:59:25 mx1 smtpd[99113]: 7824635eb33a7cc5 smtp disconnected 
reason=quit
  Apr 18 16:59:25 mx1 dovecot: 
lmtp(kir...@korins.ky)<95241>: sieve: 
msgid=<7824635f3b544...@mx1.catap.net>: stored mail into mailbox 'INBOX'
  Apr 18 16:59:25 mx1 dovecot: lmtp(95241): Disconnect from local: Logged out 
(state=READY)
  Apr 18 16:59:25 mx1 smtpd[99113]: 782463605f75696d mda delivery 
evpid=12820125cea959e4 from= to= 
rcpt= user=vmail delay=0s result=Ok stat=Delivered
  Apr 18 16:59:25 mx1 smtpd[99113]: 782463663f7915e7 mta connecting 
address=smtp://87.191.57.186:25 host=87.191.57.186
  Apr 18 16:59:25 mx1 smtpd[99113]: 782463663f7915e7 mta connected
  Apr 18 16:59:26 mx1 smtpd[99113]: 78246367e5a94426 mta connecting 
address=smtp://94.237.105.223:25 host=talvi.dovecot.org
  Apr 18 16:59:26 mx1 smtpd[99113]: 78246367e5a94426 mta connected
  Apr 18 16:59:26 mx1 smtpd[99113]: 782463663f7915e7 mta tls 
ciphers=TLSv1.3:TLS_AES_256_GCM_SHA384:256
  Apr 18 16:59:26 mx1 smtpd[99113]: 782463663f7915e7 mta cert-check 
result="unverified" 

Re: dsync with vurtial folder leads to mess of msg ID

2024-04-18 Thread Kirill A. Korinsky via dovecot
Greetings,

On Wed, 28 Feb 2024 22:15:55 +0100,
Kirill A. Korinsky wrote:
> 
> As the next step I've reduced logs to verbose, let's see how it goes.
> 

Reducing verbosity re-triggered an issue.

Meanwhile, as a lucky guess I've increased the number of open files on all
users which is used by dovecot as:

  mx1$ cat /etc/login.conf.d/*
  _dovecot:\
  :openfiles=1024:\
  :tc=daemon:
  dsync:\
  :openfiles=1024:\
  :tc=daemon:
  vmail:\
  :openfiles=1024:\
  :tc=daemon:
  mx1$ 

My user has a lot of IMAP folders, and if I run doveadm fetch from shell
with default limits (open files 128 files) it puts a lot of errors like:

  doveadm(kir...@korins.ky): Error: fetch(guid) failed for box=virtual.All 
uid=5097: Internal error occurred. Refer to server log for more information. 
[2024-04-18 14:39:15] (for backend mailbox Archive.Texunatech)
  doveadm(kir...@korins.ky): Error: Mailbox Archive.Texunatech: 
open(/var/vmail/kir...@korins.ky/Maildir/.SomeFolder/dovecot-uidlist) failed: 
Too many open files

So, somewhere here I made a guess errors are swallow, but it exists.

So, I've moved future by this way.
 1. Shutdown smtpd and dovect.
 2. Disabled smtpd from autostart.
 3. Disabled replication on both servers.
 4. Update logins.conf to increase limits.
 5. Remove all virtual folders from all users.
 6. Reboot server.
 7. Wait couple of minuts (like 10) and enable replication.
 8. Wait one full sync cycle.
 9. Enable smtpd and start it.

After this I do have consisten output:

  mx1# doveadm fetch "uid guid" -u kir...@korins.ky mailbox virtual.All | md5 
  8978c5e506bf7bcf769232d48b964e00
  mx1#

and

  mx2# doveadm fetch "uid guid" -u kir...@korins.ky mailbox virtual.All | md5 
  8978c5e506bf7bcf769232d48b964e00
  mx2# 

Anyway, that setup works for a day or two and I can't say that it really
help because I need to update machines to OpenBSD 7.5 to update OpenSMTPD to
fix another issue. This update included install of dovecot to 2.3.21
(47349e2482).

So, if issue do not reappear in couple of weeks, I can't say that help:
increase of limits or update of dovecot.

Anyway, I'll be back.

-- 
wbr, Kirill
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org


Re: dsync with vurtial folder leads to mess of msg ID

2024-02-28 Thread Kirill A . Korinsky
Greetings,

Rigt now both of servers works with debug log of dsync and produces ~1G logs
per day.

Seems that such verbose output decreases probability of issue.

For a week no more desync of uid in virtual boxes.

  mx1# doveadm fetch -u kir...@korins.ky 'uid' mailbox virtual.All | grep 
'^uid:' | tail -n 5
  uid: 145708
  uid: 145709
  uid: 145711
  uid: 145713
  uid: 145714
  mx1# doveadm fetch -u kir...@korins.ky hdr mailbox virtual.All uid 145714 | 
md5
  859986e6c659f8df9bab5efa8002ee8e
  mx1# doveadm fetch -u kir...@korins.ky hdr mailbox virtual.All uid 145713 | 
md5 
  ae1e087ff70a00cb7df739d4dda9b933
  mx1# doveadm fetch -u kir...@korins.ky hdr mailbox virtual.All uid 145711 | 
md5 
  b70ad38a0ba4b1e8087f93e3c5cd438f

vs

  mx2# doveadm fetch -u kir...@korins.ky 'uid' mailbox virtual.All | grep 
'^uid:' | tail -n 5
  uid: 145708
  uid: 145709
  uid: 145711
  uid: 145713
  uid: 145714
  mx2# doveadm fetch -u kir...@korins.ky hdr mailbox virtual.All uid 145714 | 
md5
  859986e6c659f8df9bab5efa8002ee8e
  mx2# doveadm fetch -u kir...@korins.ky hdr mailbox virtual.All uid 145713 | 
md5 
  ae1e087ff70a00cb7df739d4dda9b933
  mx2# doveadm fetch -u kir...@korins.ky hdr mailbox virtual.All uid 145711 | 
md5 
  b70ad38a0ba4b1e8087f93e3c5cd438f

As the next step I've reduced logs to verbose, let's see how it goes.

-- 
wbr, Kirill
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org


Re: dsync with vurtial folder leads to mess of msg ID

2024-02-21 Thread Kirill A . Korinsky
Greetings,

As promised: I'm back.

After processing about 740 mails, it is again.

Via SSH based replication it isn't so dramatic, but exists.

See:

  mx1# doveadm fetch -u kir...@korins.ky 'uid' mailbox virtual.All | grep 
'^uid:' | tail -n 5
  uid: 145485
  uid: 145486
  uid: 145487
  uid: 145488
  uid: 145489
  mx1# doveadm fetch -u kir...@korins.ky hdr.Message-ID mailbox virtual.All uid 
145489
  hdr.message-id: 

  mx1#

vs

  mx2# doveadm fetch -u kir...@korins.ky 'uid' mailbox virtual.All | grep 
'^uid:' | tail -n 5
  uid: 145487
  uid: 145488
  uid: 145489
  uid: 145490
  uid: 145491
  mx2# doveadm fetch -u kir...@korins.ky hdr.Message-ID mailbox virtual.All uid 
145491
  hdr.message-id: 

  mx2# doveadm fetch -u kir...@korins.ky hdr.Message-ID mailbox virtual.All uid 
145489
  hdr.message-id: <87edd58shf.wl-kir...@korins.ky>
  mx2#

So, by some digging I discovered the mails which triggers an issue:

  mx1# doveadm fetch -u kir...@korins.ky 'uid' mailbox virtual.All | grep 
'^uid: 14484'
  uid: 14484
  uid: 144840
  uid: 144841
  uid: 144845
  uid: 144848
  uid: 144849
  mx1# doveadm fetch -u kir...@korins.ky hdr mailbox virtual.All uid 144841 | 
md5
  d65a22e12bd0c72d321a78d2efd3383b
  mx1# doveadm fetch -u kir...@korins.ky hdr mailbox virtual.All uid 144840 | 
md5
  25a8b53ecfd513cf9031dae0e9000d92
  mx1#

vs

  mx2# doveadm fetch -u kir...@korins.ky 'uid' mailbox virtual.All | grep 
'^uid: 14484'
  uid: 14484
  uid: 144840
  uid: 144842
  uid: 144846
  uid: 144849
  mx2# doveadm fetch -u kir...@korins.ky hdr mailbox virtual.All uid 144842 | 
md5
  d65a22e12bd0c72d321a78d2efd3383b
  mx2# doveadm fetch -u kir...@korins.ky hdr mailbox virtual.All uid 144840 | 
md5
  25a8b53ecfd513cf9031dae0e9000d92
  mx2#

this email 144841 / 144842 were delivered to mx1 at 17 Feb 2024 17:52:18
which allows me do check logs:

  mx1# zgrep '17:52:[123]' /var/log/maillog.3.gz
  Feb 17 17:52:15 mx1 smtpd[52094]: d7da603e4213f8ff smtp connected 
address=199.185.178.25 host=mail.openbsd.org
  Feb 17 17:52:15 mx1 smtpd[39991]: dnsbl: d7da603e4213f8ff DNS error 2 on 
cbl.abuseat.org
  Feb 17 17:52:17 mx1 smtpd[52094]: d7da603e4213f8ff smtp tls 
ciphers=TLSv1.3:TLS_AES_256_GCM_SHA384:256
  Feb 17 17:52:18 mx1 smtpd[52094]: d7da603e4213f8ff smtp message 
msgid=d8e3748c size=4552 nrcpt=1 proto=ESMTP
  Feb 17 17:52:18 mx1 smtpd[52094]: d7da603e4213f8ff smtp envelope 
evpid=d8e3748cc73371ec from= 
to=
  Feb 17 17:52:19 mx1 dovecot: lmtp(15471): Connect from local
  Feb 17 17:52:19 mx1 dovecot: 
lmtp(kir...@korins.ky)<15471>: sieve: 
msgid=<4bc76d19-00f2-b60b-a87f-315adfd08...@grey.my.domain>: stored mail into 
mailbox 'INBOX'
  Feb 17 17:52:19 mx1 dovecot: lmtp(15471): Disconnect from local: Logged out 
(state=READY)
  Feb 17 17:52:19 mx1 smtpd[52094]: d7da603f60cff302 mda delivery 
evpid=d8e3748cc73371ec from= 
to= rcpt= user=vmail delay=1s result=Ok 
stat=Delivered
  Feb 17 17:52:19 mx1 smtpd[52094]: d7da603e4213f8ff smtp disconnected 
reason=quit
  mx1#

nothing, let check mx2:

  mx2# zgrep '17:52:[123]' /var/log/maillog.3.gz
  Feb 17 17:52:17 mx2 smtpd[3556]: 99938cb221b4cb0b smtp connected 
address=199.185.178.25 host=mail.openbsd.org
  Feb 17 17:52:20 mx2 smtpd[3556]: 99938cb221b4cb0b smtp tls 
ciphers=TLSv1.3:TLS_AES_256_GCM_SHA384:256
  Feb 17 17:52:30 mx2 smtpd[3556]: 99938cb221b4cb0b smtp disconnected 
reason=quit
  mx2#

nothing as well.

I do have full sync each 15 minutes, let check any dsync logs on bigger
interval (an hour):

  mx1# zgrep '17 1[78]:' /var/log/maillog.3.gz | grep dsync
  mx1#

and

  mx2# zgrep '17 1[78]:' /var/log/maillog.3.gz | grep dsync
  mx2#

Also hasn't got anything.

It seems that SSH based replication is better than TCP, and both of at
least works that I can't say about TCPS.

But it may introduce a kind of collision inside virtual boxes.

So, to future investigate an issue I've started to run dsync-server as:
doveadm -Dv dsync-server, I hoep that it allows to catch this.

And as usual re-created vurtial boxes.

--
wbr, Kirill
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org


Re: dsync with vurtial folder leads to mess of msg ID

2024-02-16 Thread Kirill A . Korinsky
Greetings,

I'd like to report that switch from TCP based replication to SSH based seems to
solve this issue.

SSH-based replication last for almost 24h. After I've switched to it I had
removed all virtual folders on all users, that triggres re-creating it.

Since then, I can't reproduce it.

  mx1# doveadm fetch -u kir...@korins.ky 'uid' mailbox virtual.All | grep 
'^uid:' | tail -n 5  
  uid: 144747
  uid: 144748
  uid: 144749
  uid: 144750
  uid: 144751
  mx1# 

and

  mx2# doveadm fetch -u kir...@korins.ky 'uid' mailbox virtual.All | grep 
'^uid:' | tail -n 5  
  uid: 144747
  uid: 144748
  uid: 144749
  uid: 144750
  uid: 144751
  mx2#

If error appears back, I'll be back to this thread.

-- 
wbr, Kirill
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org


Re: dsync with vurtial folder leads to mess of msg ID

2024-01-26 Thread Kirill A . Korinsky
On Fri, 26 Jan 2024 01:44:06 +0100,
Kirill A. Korinsky wrote:
> 
> So far so good.
>

And here we go again

  mx1# doveadm fetch -u kir...@korins.ky 'uid' mailbox virtual.All | grep 
'^uid:' | tail -n 20 
  uid: 144044
  uid: 144045
  uid: 144046
  uid: 144047
  uid: 144048
  uid: 144049
  uid: 144050
  uid: 144051
  uid: 144052
  uid: 144053
  uid: 144055
  uid: 144059
  uid: 144062
  uid: 144073
  uid: 144074
  uid: 144075
  uid: 144076
  uid: 144077
  uid: 144078
  uid: 144079
  mx1# 

vs

  mx2# doveadm fetch -u kir...@korins.ky 'uid' mailbox virtual.All | grep 
'^uid:' | tail -n 20 
  uid: 144044
  uid: 144045
  uid: 144046
  uid: 144047
  uid: 144048
  uid: 144049
  uid: 144050
  uid: 144051
  uid: 144052
  uid: 144053
  uid: 144054
  uid: 144057
  uid: 144060
  uid: 144071
  uid: 144072
  uid: 144073
  uid: 144074
  uid: 144075
  uid: 144076
  uid: 144077
  mx2#

notable to say that message 144055 / 144054 was BCC of my message to the list
where I said that so far so good.

BTW when I go to the logs, I haven't found anything that can explain it.

Any suggestion how can I debug it future?

-- 
wbr, Kirill
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org


Re: dsync with vurtial folder leads to mess of msg ID

2024-01-25 Thread Kirill A . Korinsky
On Sun, 21 Jan 2024 17:31:37 +0100,
Kirill A. Korinsky wrote:
>
> mail_replica = tcps:mx2.catap.net
>

I've discovered that using TCP+SSL replication leads to errors in logs:

   Error: Couldn't lock ../.dovecot-sync.lock: fcntl(../.dovecot-sync.lock, 
write-lock, F_SETLKW) locking failed: Timed out after 30 seconds (WRITE lock 
held by pid XXX)

I can't say that mails weren't replicated. It was, but it was quite unstable.

So, now I've switched to simple TCP replication, and recreated all virtual
folders.

So far so good.

-- 
wbr, Kirill
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org


Re: dsync with vurtial folder leads to mess of msg ID

2024-01-21 Thread Kirill A . Korinsky
On Sun, 21 Jan 2024 16:34:44 +0100,
Aki Tuomi wrote:
>
> Can you send output of doveconf -n?
> 

Sure, here it is:

  # 2.3.20 (80a5ac675d): /etc/dovecot/dovecot.conf
  # Pigeonhole version 0.5.20 (149edcf2)
  # OS: OpenBSD 7.4 amd64  
  # Hostname: mx1.catap.net
  default_vsz_limit = 10 G
  doveadm_password = # hidden, use -P to show it
  doveadm_port = 12345
  first_valid_uid = 1000
  mail_attribute_dict = file:%h/Maildir/dovecot-attributes
  mail_gid = vmail
  mail_location = maildir:~/Maildir
  mail_plugins = notify replication fts fts_flatcurve virtual
  mail_uid = vmail
  managesieve_notify_capability = mailto
  managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date index ihave duplicate 
mime foreverypart extracttext
  mbox_write_locks = fcntl
  mmap_disable = yes
  namespace inbox {
inbox = yes
location = 
mailbox Archive {
  auto = subscribe
  special_use = \Archive
}
mailbox Drafts {
  auto = subscribe
  special_use = \Drafts
}
mailbox Junk {
  auto = subscribe
  autoexpunge = 60 days
  special_use = \Junk
}
mailbox Sent {
  auto = subscribe
  special_use = \Sent
}
mailbox Trash {
  auto = subscribe
  autoexpunge = 60 days
  special_use = \Trash
}
prefix = 
separator = .
  }
  namespace virtual {
location = virtual:/etc/dovecot/virtual:INDEX=~/virtual
mailbox All {
  comment = All messages without Trash and Junk
  special_use = \All
}
mailbox Flagged {
  comment = All flagged messages without Trash and Junk
  special_use = \Flagged
}
prefix = virtual.
separator = .
  }
  passdb {
args = username_format=%u /etc/dovecot/users
driver = passwd-file
  }
  plugin {
fts = flatcurve
fts_filters = normalizer-icu snowball stopwords
fts_filters_en = lowercase snowball english-possessive stopwords
fts_flatcurve_substring_search = yes
fts_languages = en de ru es
fts_tokenizer_generic = algorithm=simple
fts_tokenizers = generic email-address
mail_replica = tcps:mx2.catap.net
sieve = file:~/sieve;active=~/.dovecot.sieve
sieve_before = /etc/dovecot/sieve_before.d/
  }
  protocols = imap lmtp sieve
  replication_dsync_parameters = -d -l 30 -U -x virtual.
  service aggregator {
fifo_listener replication-notify-fifo {
  user = vmail
}
unix_listener replication-notify {
  user = vmail
}
  }
  service doveadm {
inet_listener {
  port = 12345
  ssl = yes
}
  }
  service imap-login {
inet_listener imap {
  port = 143
}
inet_listener imaps {
  port = 993
  ssl = yes
}
  }
  service lmtp {
unix_listener lmtp {
  mode = 0666
}
  }
  service managesieve-login {
inet_listener sieve {
  port = 4190
}
  }
  service replicator {
process_min_avail = 1
unix_listener replicator-doveadm {
  mode = 0600
  user = vmail
}
  }
  ssl_cert = 

Re: dsync with vurtial folder leads to mess of msg ID

2024-01-21 Thread Aki Tuomi via dovecot
> On 20/01/2024 17:00 EET Kirill A. Korinsky  wrote:
> 
>  
> I'd like to add that allowing to save messages via virtual folder to some 
> folder
> by change it defintion to
> 
>   *
>   !Archive
>   -Trash
>   -Trash/*
>   -Junk
>   -Junk/*
> all
> 
> doesn't help and syncronization fails as usual:
> 
>   Jan 20 15:34:06 mx1 dovecot: 
> doveadm(kir...@korins.ky)<75563>: Error: Can't create 
> mailbox virtual.All: Can't create virtual mailboxes
> 
> And to share my setup of virtual folder which is quite simple:
> 
>   mail_plugins = $mail_plugins virtual
> 
>   namespace virtual {
> separator = .
> prefix = virtual.
> 
> location = virtual:/etc/dovecot/virtual:INDEX=~/virtual
> 
> mailbox All {
>   special_use = \All
>   comment = All messages without Trash and Junk
> }
> 
> mailbox Flagged {
>   special_use = \Flagged
>   comment = All flagged messages without Trash and Junk
> }
>   }
> 
> --
> wbr, Kirill

Can you send output of doveconf -n?

Aki
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org


Re: dsync with vurtial folder leads to mess of msg ID

2024-01-20 Thread Kirill A . Korinsky
I'd like to add that allowing to save messages via virtual folder to some folder
by change it defintion to

  *
  !Archive
  -Trash
  -Trash/*
  -Junk
  -Junk/*
all

doesn't help and syncronization fails as usual:

  Jan 20 15:34:06 mx1 dovecot: 
doveadm(kir...@korins.ky)<75563>: Error: Can't create 
mailbox virtual.All: Can't create virtual mailboxes

And to share my setup of virtual folder which is quite simple:

  mail_plugins = $mail_plugins virtual

  namespace virtual {
separator = .
prefix = virtual.

location = virtual:/etc/dovecot/virtual:INDEX=~/virtual

mailbox All {
  special_use = \All
  comment = All messages without Trash and Junk
}

mailbox Flagged {
  special_use = \Flagged
  comment = All flagged messages without Trash and Junk
}
  }

--
wbr, Kirill
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org


dsync with vurtial folder leads to mess of msg ID

2024-01-19 Thread Kirill A . Korinsky
Greetings,

I have a setup with two dovecot servers and dsync replication between them.

On both of them I have a virtual folders, forexample All with config:

  mx1# cat /etc/dovecot/virtual/All/dovecot-virtual
  *
  -Trash
  -Trash/*
  -Junk
  -Junk/*
all

All virtual folders are excluded from replication because it leads to broken
replication with errors like: virtual.All can't be created. So, I have.

  replication_dsync_parameters = -d -l 30 -U -x virtual.

Unfortently, after a couple of weeks I've noticed that msg id for the same
message on the different servers aren't match.

For example:

  mx1# doveadm fetch -u kir...@korins.ky 'uid' mailbox virtual.All header 
Message-ID 'macports/macports-ports/pull/22284/c1901369...@github.com'
  uid: 157786
  mx1# doveadm fetch -u kir...@korins.ky 'uid' mailbox INBOX header Message-ID 
'macports/macports-ports/pull/22284/c1901369...@github.com'   
  uid: 5561
  mx1#

vs

  mx2# doveadm fetch -u kir...@korins.ky 'uid' mailbox virtual.All header 
Message-ID 'macports/macports-ports/pull/22284/c1901369...@github.com'
  uid: 157766
  mx2# doveadm fetch -u kir...@korins.ky 'uid' mailbox INBOX header Message-ID 
'macports/macports-ports/pull/22284/c1901369...@github.com'  
  uid: 5561
  mx2#

as you may see msgid at INBOX are matched, but it isn't true for virtual.All,
here 157786 vs 157766

So, here the question: how can I sync msg id inside virtual folder on this 
setup?

--
wbr, Kirill
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org