Hello Oswald,
(I hit the wrong reply button before and so my reply went to Oswald
only. For reference here is a copy of my mail to the list.)
On 8/4/21 11:27 AM, Oswald Buddenhagen wrote:
> On Wed, Aug 04, 2021 at 08:22:33AM +0200, Uwe Kleine-König wrote:
>> from time to time I experience mbsync (from the 1.3.0-2.2 Debian
package) saying:
>>
>> Aug 04 07:31:13 taurus syncmail-work[1553327]: Warning: lost track
of 460840 pulled message(s)
>>
> that's supposed to happen only for messages that were "in flight"
when a sync run was interrupted.
> but the way you describe it (and the sheer number of messages)
suggests that this happens out of the blue for messages that should have
been committed long ago.
The current breakage isn't resynced yet.
A bigger context of the logs of an older run can be found at
https://www.kleine-koenig.org/~uwe/syncmail.log
>> Aug 04 07:31:14 taurus syncmail-work[1553327]: Socket error: secure
write to tunnel 'ssh -W imap:143 ptz.ptx': Broken pipe
>>
> that's after the above message, so while it's already trying to
resync, and it should abort due to that. that means that you must have
run mbsync again to observe this:
>
>> which results into mbsync taking several hours to resync [...]
right.
> that makes me wonder what other minor details you missed/omitted.
I run mbsync from a systemd user timer periodically.
> one that would matter: after the resync, are the messages duplicated?
on the client? the server?
The state from the last time this happend is gone, so I cannot tell for
sure, but given that notmuch reported "Added 768 new messages to the
database. Detected 490220 file renames." the last time I expect there
was no duplication on the client side. The following runs of
mbsync+notmuch didn't add a big number of new mails, so I assume further
there was no duplication on the server side either.
>> My .mbsyncrc looks as follows:
>>
> nothing obviously wrong with it, afaics.
>
>> Tunnel "ssh -W imap:143 ptz.ptx"
>> SSLType STARTTLS
>
> that's a weird combo. going around a firewall?
Yes, the IMAP server is only reachable from my company's internal network.
>> Any hint how to debug that?
>
> first verify that mbsync actually exits cleanly. otherwise the sync
journal would grow indefinitely, which might in principle cause a
ridiculous number of messages being reported as lost (though it would
require an additional problem somewhere).
I saw mbsync write to one of its files at 100% yesterday while the ssh
tunnel was already gone. This was before I suspended the machine, and
some time after wakeup today, the loss of 460840 messages was reported
from that process.
> then, have a look at the actual folders, specifically the
.mbsyncstate* files. there should be no .*.lock, .*.new or .*.journal files.
ok, will check.
> the way to do "real" debugging is to run with -D and save the logs.
but given the number of messages involved and the fact that this happens
only occasionally is likely to make this painful.
>
> have "fun" ...
Would it make sense to put my maildir into a git repo (including the
mbsync files) and commit after each run? Is there a more suitable place
where I can read about how mbsync tracks mail than the source code to
understand the content of these files?
I assume the first step is to update to the 1.4 series of mbsync, will
do that once my mails are in sync again.
Best regards
Uwe
_______________________________________________
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel
