backupd and sync_client IOERROR
Hello, I'm trying to configure backupd in rolling mode as a final setup. Running a first backup on few users sync_client -A -n bck -z -v -v after a while the process die with: cyrus/sync_client[9540]: MESSAGE received NO response: IMAP_PROTOCOL_ERROR Protocol error cyrus/sync_client[9540]: do_folders(): update failed: example.com!user.utente^archivista 'Bad protocol' cyrus/sync_client[9540]: IOERROR: do_user_main: Bad protocol for utente.archivi...@example.com to [no channel] (tst-msg03-bck.example.com) Error from do_user(utente.archivi...@example.com): bailing out! cyrus/sync_client[9540]: Error in do_user(utente.archivi...@example.com): bailing out! The backupd host doesn't complain: 2020-06-18T11:50:30.528584+02:00 tst-msg03-bck backupd[1308172]: creating sql_db /var/spool/cyr_backup/u/utente.archivista@example.com_cxlsmX.index 2020-06-18T11:56:44.971042+02:00 tst-msg03-bck backupd[1308172]: login: tst-msg03.example.com [10.102.42.168] cyr_backup LOGIN User logged in So I repeated the same command, but with less verbosity: sync_client -A -n bck -z -v and it worked well, without errors. Uhm... Maybe, before to run the first initial backup (sync_client -A), do I have to stop the sync_client already working in rolling mode? http://www.cyrusimap.org/imap/reference/admin/backups.html says "Update cyrus.conf(5) to add a sync_client(8) invocation to the SERVICES section specifying (at least) the -r and -n channel options.". Really maybe you intend START. In SERVICE I need to specify a listen too, this is not very clear to me. Around of backupd I would ask two other questions: 1) The backupd host always claims this: backupd[1309970]: DBERROR: reading /var/lib/imap/db/skipstamp, assuming the worst: No such file or directory Could you tell me how to fix that DBERROR too? 2) Is there a way to delete a user from backupd host? Thank you very much Cheers Marco master imapd.conf contains: bck_sync_host: tst-msg03-bck.example.com bck_sync_port: csync bck_sync_authname: cyr_backup bck_sync_password: * bck_sync_log: 1 bck_sync_repeat_interval: 1m bck_sync_try_imap: no force_sasl_client_mech: PLAIN, LOGIN sync_log: on sync_log_channels: squatter, bck master cyrus.conf: START { # do not delete this entry! recover cmd="ctl_cyrusdb -r" # backup backupcmd="/sbin/sync_client -r -n bck -z" } == backupd full imapd.conf is: # Configuration directory configdirectory: /var/lib/imap temp_path: /run/cyrus # Directories for proc and lock files proc_path: /run/cyrus/proc mboxname_lockpath: /run/cyrus/lock # One year backup backup_retention: 365 backuppartition-name: /var/spool/cyr_backup # Maybe to improve: backup_compact_minsize: 0 backup_compact_maxsize: 0 backup_compact_work_threshold: 1 admins: cyr_backup ### ## User Authentication settings ### # Allow plaintext logins by default (SASL PLAIN) allowplaintext: yes ### ## SASL library options (these are handled directly by the SASL ## libraries, refer to SASL documentation for an up-to-date list of ## these) ### # The mechanism(s) used by the server to verify plaintext passwords. # Possible values are "saslauthd", "auxprop", "pwcheck" and # "alwaystrue". They are tried in order, you can specify more than one, # separated by spaces. sasl_pwcheck_method: saslauthd # If enabled, the SASL library will automatically create authentication # secrets when given a plaintext password. Refer to SASL documentation sasl_auto_transition: no === backupd full cyrus.conf is: SERVICES { # backupd is probably the only service entry your backup server needs backupd cmd="backupd" listen="csync" proto="tcp4" prefork=0 } EVENTS { # this is required checkpoint cmd="ctl_cyrusdb -c" period=30 # arrange for compact to run at some interval compact cmd="ctl_backups compact -A" at=0400 } === Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: backupd and sync_client IOERROR
Hi Marco, On Thu, Jun 18, 2020, at 10:19 PM, Marco wrote: > Hello, > > I'm trying to configure backupd in rolling mode as a final setup. > Running a first backup on few users > > sync_client -A -n bck -z -v -v > > after a while the process die with: > > cyrus/sync_client[9540]: MESSAGE received NO response: > IMAP_PROTOCOL_ERROR Protocol error > cyrus/sync_client[9540]: do_folders(): update failed: > example.com!user.utente^archivista 'Bad protocol' > cyrus/sync_client[9540]: IOERROR: do_user_main: Bad protocol for > utente.archivi...@example.com to [no channel] (tst-msg03-bck.example.com) > Error from do_user(utente.archivi...@example.com): bailing out! > cyrus/sync_client[9540]: Error in > do_user(utente.archivi...@example.com): bailing out! > > The backupd host doesn't complain: > > 2020-06-18T11:50:30.528584+02:00 tst-msg03-bck backupd[1308172]: > creating sql_db > /var/spool/cyr_backup/u/utente.archivista@example.com_cxlsmX.index > 2020-06-18T11:56:44.971042+02:00 tst-msg03-bck backupd[1308172]: login: > tst-msg03.example.com [10.102.42.168] cyr_backup LOGIN User logged in > > So I repeated the same command, but with less verbosity: > > sync_client -A -n bck -z -v > > and it worked well, without errors. Uhm... > > Maybe, before to run the first initial backup (sync_client -A), do I > have to stop the sync_client already working in rolling mode? Rolling mode only makes incremental updates, so if you're starting from a server that already has existing data, you should do the first manual initial backups before enabling the rolling mode. It's generally safe to run a manual sync_client side-by-side with a rolling one (they will lock things properly and keep out of each other's way). But the exception to this is that a rolling mode replication can't coherently update an uninitialised replica (since it only sends new changes recorded in the sync log, but not the pre-existing data). You need to do a full manual replication first, to ensure both sides are the same, before rolling mode can work sensibly. This is the same regardless of whether your replica is a normal replica or a backupd one. As you saw, replicating again manually will probably fix you up, at least. :) > http://www.cyrusimap.org/imap/reference/admin/backups.html says "Update > cyrus.conf(5) to add a sync_client(8) invocation to the SERVICES section > specifying (at least) the -r and -n channel options.". Really maybe you > intend START. In SERVICE I need to specify a listen too, this is not > very clear to me. This is wrong. It should be the DAEMON section, not the SERVICES section. I'm fixing this now, so the website should be updated shortly. You can also run it from the START section, but note that if the sync_client exits early for some reason, master will not restart it. If you run it from the START section, you will need to monitor/restart it yourself. > Around of backupd I would ask two other questions: > > 1) The backupd host always claims this: > > backupd[1309970]: DBERROR: reading /var/lib/imap/db/skipstamp, assuming > the worst: No such file or directory > > Could you tell me how to fix that DBERROR too? I think you might need to add the usual recover entry to the START section: recover cmd="ctl_cyrusdb -r" I notice this is missing from the backups documentation -- please let me know if it this sorts it out, and I'll fix it up! > 2) Is there a way to delete a user from backupd host? That's an interesting question... Under normal use, if a user is deleted from their imap server, then, after the deletion has replicated to the backup server, and after the "backup_retention" period has elapsed, and after a subsequent "ctl_backups compact ..." has been run, then the backup file should be approximately empty. But I think it will still exist; nothing actually deletes it yet. To get rid of it manually, you can use "ctl_backups list ..." to find the filename where it exists on disk. You can also find it by using cyr_dbtool to look up the user's key in backups.db. You will then need to: a) use cyr_dbtool to delete that user's key from backups.db b) delete the named file (it will be like "/some/path/userid_XX") c) also delete the corresponding "/some/path/userid_XX.index" file Note that if the user still really exists, and a rolling sync_client replicates them to the same backup server again, then the backup file will be recreated (with a new XX) -- with the same problems as above wrt no-initial-sync. Cheers, ellie Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: backupd and sync_client IOERROR
Hi Ellie, Il 19/06/2020 03:01, ellie timoney ha scritto: I think you might need to add the usual recover entry to the START section: recover cmd="ctl_cyrusdb -r" I notice this is missing from the backups documentation -- please let me know if it this sorts it out, and I'll fix it up! wow, yes, it works. With this config, after the Cyrus restart the mailboxes.db and the skipstamps dbs are created and the error disappears from syslog. 2) Is there a way to delete a user from backupd host? That's an interesting question... Under normal use, if a user is deleted from their imap server, then, after the deletion has replicated to the backup server, and after the "backup_retention" period has elapsed, and after a subsequent "ctl_backups compact ..." has been run, then the backup file should be approximately empty. But I think it will still exist; nothing actually deletes it yet. To get rid of it manually, you can use "ctl_backups list ..." to find the filename where it exists on disk. You can also find it by using cyr_dbtool to look up the user's key in backups.db. You will then need to: a) use cyr_dbtool to delete that user's key from backups.db b) delete the named file (it will be like "/some/path/userid_XX") c) also delete the corresponding "/some/path/userid_XX.index" file Note that if the user still really exists, and a rolling sync_client replicates them to the same backup server again, then the backup file will be recreated (with a new XX) -- with the same problems as above wrt no-initial-sync. It works! Making some tests and the initial sync, it's easy with sync_client insert a wrong non-existent userid (I added "/user" before the userid). And that wrong userid is immediately created on the backupd server :) With the above procedure I can delete the wrong entry. Definitively I think that this backup feature is a very good idea. I suggested the possibility to choose the recovery date (ie: all messages and mailboxes deleted or expunged from to ). I hope you could add this feature. Thank you very very much for these detailed answers!! I now tried a restore, but I still have some problems. I very appreciate if you could read the following issue. My goal is to recover a message that was removed (expunged) from the IMAP server, ignoring already existing messages. It seems the "-x" flag could help. It seems it allows to restore and unexpunge a previously expunged messages in the IMAP server. Yesterday I expunged the message 16ceead9802286784d7a54c5bc782891f76f2f2e: 2020-06-18T15:44:37.374556+02:00 tst-msg03 cyrus/imap[9461]: auditlog: expunge sessionid= mailbox= uniqueid= uid=<32092> modseq=<46828> sysflags= guid=<16ceead9802286784d7a54c5bc782891f76f2f2e> I tried today to restore it: cyr_restore -v -U -x tst-msg03.example.com -u utente.archivi...@example.com 16ceead9802286784d7a54c5bc782891f76f2f2e example.com!user.utente^archivista: trying to keep uidvalidity(1550228105), uniqueid(nvf96uirjgfs6xaekwwc7g44), highestmodseq(46828) and I see: 2020-06-19T09:29:24.030624+02:00 tst-msg03 cyrus/imap[32742]: login: tst-msg03-bck.example.com [10.102.42.169] cyr_restore LOGIN User logged in SESSIONID= 2020-06-19T09:29:24.162909+02:00 tst-msg03 cyrus/imap[32742]: auditlog: append sessionid= mailbox= uniqueid= uid=<32096> modseq=<46829> sysflags= guid=<16ceead9802286784d7a54c5bc782891f76f2f2e> messageid= 2020-06-19T09:29:24.163058+02:00 tst-msg03 cyrus/imap[32742]: auditlog: expunge sessionid= mailbox= uniqueid= uid=<32096> modseq=<46829> sysflags= guid=<16ceead9802286784d7a54c5bc782891f76f2f2e> 2020-06-19T09:29:24.167561+02:00 tst-msg03 cyrus/imap[32742]: auditlog: modseq sessionid= mailbox= uniqueid= highestmodseq=<46829> 2020-06-19T09:29:24.215036+02:00 tst-msg03 cyrus/imap[32742]: USAGE cyr_restore user: 0.042492 sys: 0.015435 2020-06-19T09:29:24.215191+02:00 tst-msg03 cyrus/imap[32742]: auditlog: traffic sessionid= bytes_in=<2412> bytes_out=<1039> 2020-06-19T09:29:32.855379+02:00 tst-msg03 cyrus/sync_client[32566]: IOERROR: zero length response to MAILBOXES (end of file reached) 2020-06-19T09:29:32.855682+02:00 tst-msg03 cyrus/sync_client[32566]: IOERROR: zero length response to RESTART (end of file reached) 2020-06-19T09:29:32.855713+02:00 tst-msg03 cyrus/sync_client[32566]: Error in do_sync(): bailing out! Bad protocol 2020-06-19T09:29:32.855738+02:00 tst-msg03 cyrus/sync_client[32566]: Processing sync log file /var/lib/imap/sync/bck/log-run failed: Bad protocol I tried again and I see the same result, but without the IOERROR: 2020-06-19T09:36:51.153939+02:00 tst-msg03 cyrus/imap[32762]: login: tst-msg03-bck.example.com [10.102.42.169] cyr_restore LOGIN User logged in SESSIONID= 2020-06-19T09:36:51.199742+02:00 tst-msg03 cyrus/imap[32762]: example.com!user.utente^archivista: same message appears twice 32096 32097 (why twice? It expunges 32096 just above) 2020-06-19T09:36:51.225525+
Re: backupd and sync_client IOERROR
Hi Marco, On Fri, Jun 19, 2020, at 6:44 PM, Marco wrote: > wow, yes, it works. With this config, after the Cyrus restart the > mailboxes.db and the skipstamps dbs are created and the error disappears > from syslog. Great! I've updated the documentation, and the website should update shortly. > I suggested the possibility to choose the recovery date (ie: all > messages and mailboxes deleted or expunged from to ). I > hope you could add this feature. Yep, I saw that, thanks for the request :) > I now tried a restore, but I still have some problems. I very appreciate > if you could read the following issue. > > My goal is to recover a message that was removed (expunged) from the > IMAP server, ignoring already existing messages. > > It seems the "-x" flag could help. It seems it allows to restore and > unexpunge a previously expunged messages in the IMAP server. No, you've misunderstood. The "-x" flag is to ONLY restore expunged messages. If you've specified a list of mailboxes or messages on the command line, and some of them are not expunged, the "-x" option will cause it to ignore the ones that are not expunged, and only restore the expunged ones. The "-X" option does the inverse -- they're filters. Without one of these filters, it will restore everything you asked for, regardless of its expunged status. So like, maybe a user has deleted some stuff, and you don't want to mess around figuring out which individual messages they need restored, so you just want to restore everything, and let the user figure it out. This is what -x is for -- so you can say "restore the contents of mailbox foo, but only the expunged stuff" or "restore every mailbox (-a), but only the expunged stuff". Think about it: if the messages _weren't_ expunged, there would usually be no need to restore them from backup! You would simply remove the \Deleted flag over IMAP. So, you are almost always using restore to bring back expunged messages, thus there is no special flag needed to enable this functionality. (But, if you had lost an entire server to some disaster, and restoring to its replacement, then you would need to also restore the unexpunged stuff.) > Yesterday I expunged the message 16ceead9802286784d7a54c5bc782891f76f2f2e: > > 2020-06-18T15:44:37.374556+02:00 tst-msg03 cyrus/imap[9461]: auditlog: > expunge sessionid= > mailbox= > uniqueid= uid=<32092> modseq=<46828> > sysflags= guid=<16ceead9802286784d7a54c5bc782891f76f2f2e> > > > I tried today to restore it: > cyr_restore -v -U -x tst-msg03.example.com -u > utente.archivi...@example.com 16ceead9802286784d7a54c5bc782891f76f2f2e > example.com!user.utente^archivista: trying to keep > uidvalidity(1550228105), uniqueid(nvf96uirjgfs6xaekwwc7g44), > highestmodseq(46828) > > and I see: > 2020-06-19T09:29:24.030624+02:00 tst-msg03 cyrus/imap[32742]: login: > tst-msg03-bck.example.com [10.102.42.169] cyr_restore LOGIN User logged > in SESSIONID= > > 2020-06-19T09:29:24.162909+02:00 tst-msg03 cyrus/imap[32742]: auditlog: > append sessionid= > mailbox= > uniqueid= uid=<32096> modseq=<46829> > sysflags= guid=<16ceead9802286784d7a54c5bc782891f76f2f2e> > messageid= > > 2020-06-19T09:29:24.163058+02:00 tst-msg03 cyrus/imap[32742]: auditlog: > expunge sessionid= > mailbox= > uniqueid= uid=<32096> modseq=<46829> > sysflags= guid=<16ceead9802286784d7a54c5bc782891f76f2f2e> > > 2020-06-19T09:29:24.167561+02:00 tst-msg03 cyrus/imap[32742]: auditlog: > modseq sessionid= > mailbox= > uniqueid= highestmodseq=<46829> > > 2020-06-19T09:29:24.215036+02:00 tst-msg03 cyrus/imap[32742]: USAGE > cyr_restore user: 0.042492 sys: 0.015435 > > 2020-06-19T09:29:24.215191+02:00 tst-msg03 cyrus/imap[32742]: auditlog: > traffic sessionid= > bytes_in=<2412> bytes_out=<1039> > > 2020-06-19T09:29:32.855379+02:00 tst-msg03 cyrus/sync_client[32566]: > IOERROR: zero length response to MAILBOXES (end of file reached) > > 2020-06-19T09:29:32.855682+02:00 tst-msg03 cyrus/sync_client[32566]: > IOERROR: zero length response to RESTART (end of file reached) > > 2020-06-19T09:29:32.855713+02:00 tst-msg03 cyrus/sync_client[32566]: > Error in do_sync(): bailing out! Bad protocol > > 2020-06-19T09:29:32.855738+02:00 tst-msg03 cyrus/sync_client[32566]: > Processing sync log file /var/lib/imap/sync/bck/log-run failed: Bad protocol > > I tried again and I see the same result, but without the IOERROR: > > 2020-06-19T09:36:51.153939+02:00 tst-msg03 cyrus/imap[32762]: login: > tst-msg03-bck.example.com [10.102.42.169] cyr_restore LOGIN User logged > in SESSIONID= > > 2020-06-19T09:36:51.199742+02:00 tst-msg03 cyrus/imap[32762]: > example.com!user.utente^archivista: same message appears twice 32096 32097 > > (why twice? It expunges 32096 just above) > > 2020-06-19T09:36:51.225525+02:00 tst-msg03 cyrus/imap[32762]: auditlog: > append sessionid= > mailbox= > uniqueid= uid=<32097> modseq=<46834> > sysflags= guid=<16ceead9802286784d7a54c5bc782891f76f2f
Re: backupd and sync_client IOERROR
Hello, On 22/06/2020 03:29, ellie timoney has written: [...] So like, maybe a user has deleted some stuff, and you don't want to mess around figuring out which individual messages they need restored, so you just want to restore everything, and let the user figure it out. This is what -x is for -- so you can say "restore the contents of mailbox foo, but only the expunged stuff" or "restore every mailbox (-a), but only the expunged stuff". [...] thank you for all these explanations, I think I understand now. My use case is "the user accidentally deleted and expunged and now he ask for recovery". I have to unexpunge all messages after I recovered it with "restore -x". I think there isn't a all-in-one command for this use case: a user expunged some messages and deleted some folders somewhere. I want to recover all expunged messages and all the deleted folders which are no more present in the original IMAP server (because they were expired from cyr_expire). I have to set "-x" to avoid duplication of messages. With "-a -x" I recover all expunged messages and all deleted mailboxes. But messages inside deleted mailboxes are not marked as expunged, so these mailboxes are recovered empty in the IMAP server. I have to examine the output of the restore command and repeat a restore without the "-x" on each DELETED recovered mailboxes at the previous step. Another idea is to recover all in another empty IMAP server, without "-x" at all, and the user can look at the mailbox recovered there... Thank you very much!! Cheers Marco Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: backupd and sync_client IOERROR
> I think there isn't a all-in-one command for this use case: a user > expunged some messages and deleted some folders somewhere. I want to > recover all expunged messages and all the deleted folders which are no > more present in the original IMAP server (because they were expired from > cyr_expire). > > I have to set "-x" to avoid duplication of messages. > With "-a -x" I recover all expunged messages and all deleted mailboxes. > But messages inside deleted mailboxes are not marked as expunged, so > these mailboxes are recovered empty in the IMAP server. I would run restore twice in this case: once without -x, specifying just the deleted mailboxes (you can use "cyr_backup list mailboxes ..." to get a list of the mailboxes in the user's backup). And once with -x to get all the expunged stuff. For the -x invocation only, I would probably also use the -M option to dump all the recovered stuff into a new folder, so they can easily tell it apart from any new mail that might have arrived coincidentally at the same time. > Another idea is to recover all in another empty IMAP server, without > "-x" at all, and the user can look at the mailbox recovered there... We kinda had a similar idea! Putting it all into a folder with -M is much easier than setting up a separate server, but it'll lose the folder structure. Recovering to a separate server means the folder structure can be preserved. I guess it depends on the specific recovery situation, and how hard it is to spin up a server in your environment. Cheers, ellie Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: backupd and sync_client IOERROR
On 19/06/2020 03:01, ellie timoney has written: Rolling mode only makes incremental updates, so if you're starting from a server that already has existing data, you should do the first manual initial backups before enabling the rolling mode. Hello, about this error, I retried more times. It seems really reproducible all times. I don't enable rolling mode. At the initial sync if I do: $ sync_client -A -n bck -z -v USER d...@example.com MAILBOX example.com!user.dan MAILBOX example.com!user.dan.Sent QUOTA example.com!user.dan QUOTA example.com!user.dan.Sent USER d...@example.com MAILBOX example.com!user.din MAILBOX example.com!user.din.Spam MAILBOX example.com!user.din.Trash QUOTA example.com!user.din USER d...@example.com MAILBOX example.com!user.don QUOTA example.com!user.don USER utente.archivi...@example.com MAILBOX example.com!user.utente^archivista Error from do_user(utente.archivi...@example.com): bailing out! and the error log is: 2020-07-01T15:42:14.161553+02:00 tst-msg03 cyrus/sync_client[4861]: MESSAGE received NO response: IMAP_PROTOCOL_ERROR Protocol error 2020-07-01T15:42:14.296022+02:00 tst-msg03 cyrus/sync_client[4861]: do_folders(): update failed: example.com!user.utente^archivista 'Bad protocol' 2020-07-01T15:42:14.296220+02:00 tst-msg03 cyrus/sync_client[4861]: IOERROR: do_user_main: Bad protocol for utente.archivi...@example.com to [no channel] (tst-msg03-bck.example.com) 2020-07-01T15:42:14.296290+02:00 tst-msg03 cyrus/sync_client[4861]: Error in do_user(utente.archivi...@example.com): bailing out! If I repeat the "sync_client -A -n bck -z -v" the error still occurs every time. If I do: $ sync_client -n bck -z -v -u utente.archivi...@example.com USER utente.archivi...@example.com MAILBOX example.com!user.utente^archivista MAILBOX example.com!user.utente^archivista.Cest MAILBOX example.com!user.utente^archivista.Posta archiviata MAILBOX example.com!user.utente^archivista.Posta archiviata.Archivio [...] it works without errors. Uhm... Thank you again Cheers Marco Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: backupd and sync_client IOERROR
On Wed, Jul 1, 2020, at 11:57 PM, Marco wrote: > Uhm... Wow, that's wierd. I notice that the users that worked correctly with "sync_client -A" don't have dots in their address localparts. If you create another user that also has a dot, does it fail under -A in the same way? Does it fail in the same way replicating to a normal replica, instead of a backup server? Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: backupd and sync_client IOERROR
Hello, On 03/07/2020 05:08, ellie timoney has written: I notice that the users that worked correctly with "sync_client -A" don't have dots in their address localparts. If you create another user that also has a dot, does it fail under -A in the same way? I copied the content of the failing mailbox into another mailbox with a name without dots: springst...@example.com and the result is the same: # sync_client -A -n bck -z -v USER springst...@example.com MAILBOX example.com!user.springsteen Error from do_user(springst...@example.com): bailing out! 2020-07-06T15:31:51.538935+02:00 tst-msg03-bck backupd[2234574]: login: tst-msg03.example.com [10.102.102.102] cyr_backup LOGIN User logged in 2020-07-06T15:31:51.576283+02:00 tst-msg03-bck backupd[2234574]: creating sql_db /var/spool/cyr_backup/s/springsteen@example.com_t8Yjcb.index 2020-07-06T15:36:57.552290+02:00 tst-msg03 cyrus/sync_client[17488]: MESSAGE received NO response: IMAP_PROTOCOL_ERROR Protocol error 2020-07-06T15:36:57.753294+02:00 tst-msg03 cyrus/sync_client[17488]: do_folders(): update failed: example.com!user.springsteen 'Bad protocol' 2020-07-06T15:36:57.758121+02:00 tst-msg03 cyrus/sync_client[17488]: IOERROR: do_user_main: Bad protocol for springst...@example.com to [no channel] (tst-msg03-bck.example.com) 2020-07-06T15:36:57.765868+02:00 tst-msg03 cyrus/sync_client[17488]: Error in do_user(springst...@example.com): bailing out! Instead, if I do: # sync_client -n bck -z -v -u springst...@uc.csi.it USER springst...@example.com MAILBOX example.com!user.springsteen MAILBOX example.com!user.springsteen.#splitconversations MAILBOX example.com!user.springsteen.ALLARMI MAILBOX example.com!user.springsteen.ALLARMI.John [...] the program replicates as well and without any errors. Does it fail in the same way replicating to a normal replica, instead of a backup server? At this moment I don't have a normal replica server yet. I'll try... Thank you very much Cheers Marco Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: backupd and sync_client IOERROR
Hi Marco, On Tue, Jul 7, 2020, at 12:17 AM, Marco wrote: > I copied the content of the failing mailbox into another mailbox with a > name without dots: Oh that's very curious. It suggests that something about the mailbox contents is causing it to fail under -A but not under -u. I was hoping it would just be the dot in the name, cause something like that should be fairly easy to reproduce and debug. But it's sounding like it'll be more difficult! It might be worth running "mbexamine" over some of these mailboxes -- both some that work correctly, and the ones that are misbehaving -- and see if any interesting patterns emerge? The man page is here: https://www.cyrusimap.org/imap/reference/manpages/systemcommands/mbexamine.html > > Does it fail in the same way replicating to a normal replica, instead of a > > backup server? > > At this moment I don't have a normal replica server yet. I'll try... Thanks. I think this will be important for figuring out whether the problem is replication generally, or backupd specific. Cheers, ellie Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: backupd and sync_client IOERROR
Hi Ellie, On 08/07/2020 06:23, ellie timoney has written: Oh that's very curious. It suggests that something about the mailbox contents is causing it to fail under -A but not under -u. I was hoping it would just be the dot in the name, cause something like that should be fairly easy to reproduce and debug. But it's sounding like it'll be more difficult! It might be worth running "mbexamine" over some of these mailboxes -- both some that work correctly, and the ones that are misbehaving -- and see if any interesting patterns emerge? The man page is here:https://www.cyrusimap.org/imap/reference/manpages/systemcommands/mbexamine.html The mailbox which fails is quite large (over 32000 msgs and over 1GB). mbexamine produces a very large output for all mails. I can see that it doesn't fail: it terminates without errors and with 0 exit status. Even switches -c or -q terminates without errors. All quotas checks are correct. Does it fail in the same way replicating to a normal replica, instead of a backup server? At this moment I don't have a normal replica server yet. I'll try... Thanks. I think this will be important for figuring out whether the problem is replication generally, or backupd specific. I understand. As soon as I can, I'll setup a normal replica. Thank you Cheers Marco Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus