Re: doveadm backup abort in imapc-client.c

2018-10-01 Thread Timo Sirainen


> On 27 Sep 2018, at 21.52, Evan Klitzke  wrote:
> 
> I am using Dovecot 2.2.36 and I am trying to use doveadm backup to migrate 
> email from Gmail to Dovecot. When I run doveadm backup, it goes for a while 
> and then eventually hits an assertion error (I've tried this a few times now 
> and it happens reliably).
> 
> The assertion failure looks like this:
> 
> dsync(e...@eklitzke.org): Panic: file imapc-client.c: line 179 
> (imapc_client_run_pre): assertion failed: (client->ioloop == NULL)
> 
> The last few lines of log output before the crash look like this: 
> https://gist.github.com/eklitzke/9a0dd77c44a6ee33e812f85d5773c24c
> 
> The GDB backtrace when the program panics looks like this: 
> https://gist.github.com/eklitzke/6b311474b08b546c462e0444c5cf479b

Looks like it crashes when:
 - you have multiple folders
 - dsync takes over 1 second between folders
 - a new mail arrives during dsync
 - you have imapc_features=gmail-migration

A simple workaround should be to comment out 
"list->refreshed_mailboxes_recently = FALSE" in imapc-sync.c. A better fix 
would probably be to remember an opened folder's mailbox_info_flags so it 
doesn't try to re-get them.



Re: doveadm backup abort in imapc-client.c

2018-09-29 Thread Evan Klitzke



Admin  writes:


"Reliably" means what exactly?
 - At progress of 66%?
 - after e.g. 9611 Mails?
 - whenever a specific Mail is touched?


Reliably means "before the sync finishes", but not at a specific point. 
I have about 2GB of mail I'm trying to sync, and the furthest the sync 
has gotten is about 1GB of the way in.


I looked at the core dump some more and I sort of get what's going on, 
although I don't understand the code well enough to fix the issue. The 
imapc_client is being called in a re-entrant way, in the sense that 
imapc_client_run() is called, and then while that imapc_client_run() 
function is being invoked and is on the call stack, the event loop gets 
triggered again and calls imapc_connection_input(). That causes 
imapc_client_run() to be called with the same imapc_client object, and 
this is what causes the assertion to fail. You can see this in the stack 
trace I posted, and I was able to confirm in GDB that the two 
invocations of imapc_client_run_pre() on the call stack both refer to 
the same imapc_client struct.


--
Evan Klitzke  pgp: 0x157EFCACBC648422
e: e...@eklitzke.org  w: https://eklitzke.org


Re: doveadm backup abort in imapc-client.c

2018-09-27 Thread Admin
"Reliably" means what exactly?
 - At progress of 66%?
 - after e.g. 9611 Mails?
 - whenever a specific Mail is touched? 

Von unterwegs gesendet

> Am 27.09.2018 um 20:52 schrieb Evan Klitzke :
> 
> I am using Dovecot 2.2.36 and I am trying to use doveadm backup to migrate 
> email from Gmail to Dovecot. When I run doveadm backup, it goes for a while 
> and then eventually hits an assertion error (I've tried this a few times now 
> and it happens reliably).
> 
> The assertion failure looks like this:
> 
> dsync(e...@eklitzke.org): Panic: file imapc-client.c: line 179 
> (imapc_client_run_pre): assertion failed: (client->ioloop == NULL)
> 
> The last few lines of log output before the crash look like this: 
> https://gist.github.com/eklitzke/9a0dd77c44a6ee33e812f85d5773c24c
> 
> The GDB backtrace when the program panics looks like this: 
> https://gist.github.com/eklitzke/6b311474b08b546c462e0444c5cf479b
> 
> The output of "dovecot --version" prints "2.2.36 (1f10bfa63)" and I'm using 
> the version provided by Fedora's dovecot-2.2.36-1.fc28.src.rpm package.
> 
> Is there any other information I can provide to assist in debugging? I could 
> provide a core dump provided there was a way to scrub my IMAP credentials.
> 
> --
> Evan Klitzke  pgp: 0x157EFCACBC648422
> e: e...@eklitzke.org  w: https://eklitzke.org



doveadm backup abort in imapc-client.c

2018-09-27 Thread Evan Klitzke
I am using Dovecot 2.2.36 and I am trying to use doveadm backup to 
migrate email from Gmail to Dovecot. When I run doveadm backup, it goes 
for a while and then eventually hits an assertion error (I've tried this 
a few times now and it happens reliably).


The assertion failure looks like this:

dsync(e...@eklitzke.org): Panic: file imapc-client.c: line 179 
(imapc_client_run_pre): assertion failed: (client->ioloop == NULL)


The last few lines of log output before the crash look like this: 
https://gist.github.com/eklitzke/9a0dd77c44a6ee33e812f85d5773c24c


The GDB backtrace when the program panics looks like this: 
https://gist.github.com/eklitzke/6b311474b08b546c462e0444c5cf479b


The output of "dovecot --version" prints "2.2.36 (1f10bfa63)" and I'm 
using the version provided by Fedora's dovecot-2.2.36-1.fc28.src.rpm 
package.


Is there any other information I can provide to assist in debugging? I 
could provide a core dump provided there was a way to scrub my IMAP 
credentials.


--
Evan Klitzke  pgp: 0x157EFCACBC648422
e: e...@eklitzke.org  w: https://eklitzke.org