Re: backupd and sync_client IOERROR

2020-07-08 Thread Marco

Hi Ellie,

On 08/07/2020 06:23, ellie timoney has written:

Oh that's very curious.  It suggests that something about the mailbox contents 
is causing it to fail under -A but not under -u.  I was hoping it would just be 
the dot in the name, cause something like that should be fairly easy to 
reproduce and debug.  But it's sounding like it'll be more difficult!

It might be worth running "mbexamine" over some of these mailboxes -- both some 
that work correctly, and the ones that are misbehaving -- and see if any interesting 
patterns emerge?  The man page is 
here:https://www.cyrusimap.org/imap/reference/manpages/systemcommands/mbexamine.html


The mailbox which fails is quite large (over 32000 msgs and over 1GB). 
mbexamine produces a very large output for all mails. I can see that it 
doesn't fail: it terminates without errors and with 0 exit status.


Even switches -c or -q terminates without errors. All quotas checks are 
correct.



Does it fail in the same way replicating to a normal replica, instead of a 
backup server?

At this moment I don't have a normal replica server yet. I'll try...

Thanks. I think this will be important for figuring out whether the problem is 
replication generally, or backupd specific.


I understand. As soon as I can, I'll setup a normal replica.

Thank you

Cheers
Marco

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


Re: backupd and sync_client IOERROR

2020-07-07 Thread ellie timoney
Hi Marco,

On Tue, Jul 7, 2020, at 12:17 AM, Marco wrote:
> I copied the content of the failing mailbox into another mailbox with a 
> name without dots:

Oh that's very curious.  It suggests that something about the mailbox contents 
is causing it to fail under -A but not under -u.  I was hoping it would just be 
the dot in the name, cause something like that should be fairly easy to 
reproduce and debug.  But it's sounding like it'll be more difficult!

It might be worth running "mbexamine" over some of these mailboxes -- both some 
that work correctly, and the ones that are misbehaving -- and see if any 
interesting patterns emerge?  The man page is here: 
https://www.cyrusimap.org/imap/reference/manpages/systemcommands/mbexamine.html

> > Does it fail in the same way replicating to a normal replica, instead of a 
> > backup server?
> 
> At this moment I don't have a normal replica server yet. I'll try...

Thanks. I think this will be important for figuring out whether the problem is 
replication generally, or backupd specific.

Cheers,

ellie

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


Re: backupd and sync_client IOERROR

2020-07-06 Thread Marco

Hello,

On 03/07/2020 05:08, ellie timoney has written:

I notice that the users that worked correctly with "sync_client -A" don't have 
dots in their address localparts.  If you create another user that also has a dot, does 
it fail under -A in the same way?


I copied the content of the failing mailbox into another mailbox with a 
name without dots:


springst...@example.com

and the result is the same:

# sync_client -A -n bck -z -v

USER springst...@example.com
MAILBOX example.com!user.springsteen
Error from do_user(springst...@example.com): bailing out!


2020-07-06T15:31:51.538935+02:00 tst-msg03-bck backupd[2234574]: login: 
tst-msg03.example.com [10.102.102.102] cyr_backup LOGIN User logged in
2020-07-06T15:31:51.576283+02:00 tst-msg03-bck backupd[2234574]: 
creating sql_db /var/spool/cyr_backup/s/springsteen@example.com_t8Yjcb.index
2020-07-06T15:36:57.552290+02:00 tst-msg03 cyrus/sync_client[17488]: 
MESSAGE received NO response: IMAP_PROTOCOL_ERROR Protocol error
2020-07-06T15:36:57.753294+02:00 tst-msg03 cyrus/sync_client[17488]: 
do_folders(): update failed: example.com!user.springsteen 'Bad protocol'
2020-07-06T15:36:57.758121+02:00 tst-msg03 cyrus/sync_client[17488]: 
IOERROR: do_user_main: Bad protocol for springst...@example.com to [no 
channel] (tst-msg03-bck.example.com)
2020-07-06T15:36:57.765868+02:00 tst-msg03 cyrus/sync_client[17488]: 
Error in do_user(springst...@example.com): bailing out!


Instead, if I do:

# sync_client -n bck -z -v -u springst...@uc.csi.it

USER springst...@example.com
MAILBOX example.com!user.springsteen
MAILBOX example.com!user.springsteen.#splitconversations
MAILBOX example.com!user.springsteen.ALLARMI
MAILBOX example.com!user.springsteen.ALLARMI.John
[...]

the program replicates as well and without any errors.


Does it fail in the same way replicating to a normal replica, instead of a 
backup server?


At this moment I don't have a normal replica server yet. I'll try...

Thank you very much

Cheers
Marco

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


Re: backupd and sync_client IOERROR

2020-07-02 Thread ellie timoney
On Wed, Jul 1, 2020, at 11:57 PM, Marco wrote:
> Uhm...

Wow, that's wierd.

I notice that the users that worked correctly with "sync_client -A" don't have 
dots in their address localparts.  If you create another user that also has a 
dot, does it fail under -A in the same way?

Does it fail in the same way replicating to a normal replica, instead of a 
backup server?

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


Re: backupd and sync_client IOERROR

2020-07-01 Thread Marco

On 19/06/2020 03:01, ellie timoney has written:

Rolling mode only makes incremental updates, so if you're starting from a 
server that already has existing data, you should do the first manual initial 
backups before enabling the rolling mode.


Hello,

 about this error, I retried more times. It seems really reproducible 
all times.


I don't enable rolling mode. At the initial sync if I do:


$ sync_client -A -n bck -z -v
USER d...@example.com
MAILBOX example.com!user.dan
MAILBOX example.com!user.dan.Sent
QUOTA example.com!user.dan
QUOTA example.com!user.dan.Sent
USER d...@example.com
MAILBOX example.com!user.din
MAILBOX example.com!user.din.Spam
MAILBOX example.com!user.din.Trash
QUOTA example.com!user.din
USER d...@example.com
MAILBOX example.com!user.don
QUOTA example.com!user.don
USER utente.archivi...@example.com
MAILBOX example.com!user.utente^archivista
Error from do_user(utente.archivi...@example.com): bailing out!

and the error log is:

2020-07-01T15:42:14.161553+02:00 tst-msg03 cyrus/sync_client[4861]: 
MESSAGE received NO response: IMAP_PROTOCOL_ERROR Protocol error
2020-07-01T15:42:14.296022+02:00 tst-msg03 cyrus/sync_client[4861]: 
do_folders(): update failed: example.com!user.utente^archivista 'Bad 
protocol'
2020-07-01T15:42:14.296220+02:00 tst-msg03 cyrus/sync_client[4861]: 
IOERROR: do_user_main: Bad protocol for utente.archivi...@example.com to 
[no channel] (tst-msg03-bck.example.com)
2020-07-01T15:42:14.296290+02:00 tst-msg03 cyrus/sync_client[4861]: 
Error in do_user(utente.archivi...@example.com): bailing out!



If I repeat the "sync_client -A -n bck -z -v" the error still occurs 
every time.



If I do:

$ sync_client -n bck -z -v -u utente.archivi...@example.com
USER utente.archivi...@example.com
MAILBOX example.com!user.utente^archivista
MAILBOX example.com!user.utente^archivista.Cest
MAILBOX example.com!user.utente^archivista.Posta archiviata
MAILBOX example.com!user.utente^archivista.Posta archiviata.Archivio
[...]

it works without errors.

Uhm...

Thank you again
Cheers
Marco

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


Re: backupd and sync_client IOERROR

2020-06-23 Thread ellie timoney
> I think there isn't a all-in-one command for this use case: a user 
> expunged some messages and deleted some folders somewhere. I want to 
> recover all expunged messages and all the deleted folders which are no 
> more present in the original IMAP server (because they were expired from 
> cyr_expire).
> 
> I have to set "-x" to avoid duplication of messages.
> With "-a -x" I recover all expunged messages and all deleted mailboxes. 
> But messages inside deleted mailboxes are not marked as expunged, so 
> these mailboxes are recovered empty in the IMAP server.

I would run restore twice in this case:  once without -x, specifying just the 
deleted mailboxes (you can use "cyr_backup list mailboxes ..." to get a list of 
the mailboxes in the user's backup).  And once with -x to get all the expunged 
stuff.  For the -x invocation only, I would probably also use the -M option to 
dump all the recovered stuff into a new folder, so they can easily tell it 
apart from any new mail that might have arrived coincidentally at the same time.

> Another idea is to recover all in another empty IMAP server, without 
> "-x" at all, and the user can look at the mailbox recovered there...

We kinda had a similar idea!  Putting it all into a folder with -M is much 
easier than setting up a separate server, but it'll lose the folder structure.  
Recovering to a separate server means the folder structure can be preserved.  I 
guess it depends on the specific recovery situation, and how hard it is to spin 
up a server in your environment.

Cheers,

ellie

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


Re: backupd and sync_client IOERROR

2020-06-22 Thread Marco

Hello,

On 22/06/2020 03:29, ellie timoney has written:
[...]

So like, maybe a user has deleted some stuff, and you don't want to mess around figuring out which 
individual messages they need restored, so you just want to restore everything, and let the user 
figure it out.  This is what -x is for -- so you can say "restore the contents of mailbox foo, 
but only the expunged stuff" or "restore every mailbox (-a), but only the expunged 
stuff".

[...]


 thank you for all these explanations, I think I understand now. My use 
case is "the user accidentally deleted and expunged and now he ask for 
recovery". I have to unexpunge all messages after I recovered it with 
"restore -x".


I think there isn't a all-in-one command for this use case: a user 
expunged some messages and deleted some folders somewhere. I want to 
recover all expunged messages and all the deleted folders which are no 
more present in the original IMAP server (because they were expired from 
cyr_expire).


I have to set "-x" to avoid duplication of messages.
With "-a -x" I recover all expunged messages and all deleted mailboxes. 
But messages inside deleted mailboxes are not marked as expunged, so 
these mailboxes are recovered empty in the IMAP server.


I have to examine the output of the restore command and repeat a restore 
without the "-x" on each DELETED recovered mailboxes at the previous step.



Another idea is to recover all in another empty IMAP server, without 
"-x" at all, and the user can look at the mailbox recovered there...


Thank you very much!!

Cheers
Marco

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


Re: backupd and sync_client IOERROR

2020-06-21 Thread ellie timoney
Hi Marco,

On Fri, Jun 19, 2020, at 6:44 PM, Marco wrote:
> wow, yes, it works. With this config, after the Cyrus restart the 
> mailboxes.db and the skipstamps dbs are created and the error disappears 
> from syslog.

Great! I've updated the documentation, and the website should update shortly.

> I suggested the possibility to choose the recovery date (ie: all 
> messages and mailboxes deleted or expunged from  to ). I 
> hope you could add this feature.

Yep, I saw that, thanks for the request :)

> I now tried a restore, but I still have some problems. I very appreciate 
> if you could read the following issue.
> 
> My goal is to recover a message that was removed (expunged) from the 
> IMAP server, ignoring already existing messages.
> 
> It seems the "-x" flag could help. It seems it allows to restore and 
> unexpunge a previously expunged messages in the IMAP server.

No, you've misunderstood.  The "-x" flag is to ONLY restore expunged messages.  
If you've specified a list of mailboxes or messages on the command line, and 
some of them are not expunged, the "-x" option will cause it to ignore the ones 
that are not expunged, and only restore the expunged ones.  The "-X" option 
does the inverse -- they're filters.  Without one of these filters, it will 
restore everything you asked for, regardless of its expunged status.

So like, maybe a user has deleted some stuff, and you don't want to mess around 
figuring out which individual messages they need restored, so you just want to 
restore everything, and let the user figure it out.  This is what -x is for -- 
so you can say "restore the contents of mailbox foo, but only the expunged 
stuff" or "restore every mailbox (-a), but only the expunged stuff".

Think about it: if the messages _weren't_ expunged, there would usually be no 
need to restore them from backup!  You would simply remove the \Deleted flag 
over IMAP.  So, you are almost always using restore to bring back expunged 
messages, thus there is no special flag needed to enable this functionality.

(But, if you had lost an entire server to some disaster, and restoring to its 
replacement, then you would need to also restore the unexpunged stuff.)
 
> Yesterday I expunged the message 16ceead9802286784d7a54c5bc782891f76f2f2e:
> 
> 2020-06-18T15:44:37.374556+02:00 tst-msg03 cyrus/imap[9461]: auditlog: 
> expunge sessionid= 
> mailbox= 
> uniqueid= uid=<32092> modseq=<46828> 
> sysflags= guid=<16ceead9802286784d7a54c5bc782891f76f2f2e>
> 
> 
> I tried today to restore it:
> cyr_restore -v -U -x tst-msg03.example.com -u 
> utente.archivi...@example.com 16ceead9802286784d7a54c5bc782891f76f2f2e
> example.com!user.utente^archivista: trying to keep 
> uidvalidity(1550228105), uniqueid(nvf96uirjgfs6xaekwwc7g44), 
> highestmodseq(46828)
> 
> and I see:
> 2020-06-19T09:29:24.030624+02:00 tst-msg03 cyrus/imap[32742]: login: 
> tst-msg03-bck.example.com [10.102.42.169] cyr_restore LOGIN User logged 
> in SESSIONID=
> 
> 2020-06-19T09:29:24.162909+02:00 tst-msg03 cyrus/imap[32742]: auditlog: 
> append sessionid= 
> mailbox= 
> uniqueid= uid=<32096> modseq=<46829> 
> sysflags= guid=<16ceead9802286784d7a54c5bc782891f76f2f2e> 
> messageid=
> 
> 2020-06-19T09:29:24.163058+02:00 tst-msg03 cyrus/imap[32742]: auditlog: 
> expunge sessionid= 
> mailbox= 
> uniqueid= uid=<32096> modseq=<46829> 
> sysflags= guid=<16ceead9802286784d7a54c5bc782891f76f2f2e>
> 
> 2020-06-19T09:29:24.167561+02:00 tst-msg03 cyrus/imap[32742]: auditlog: 
> modseq sessionid= 
> mailbox= 
> uniqueid= highestmodseq=<46829>
> 
> 2020-06-19T09:29:24.215036+02:00 tst-msg03 cyrus/imap[32742]: USAGE 
> cyr_restore user: 0.042492 sys: 0.015435
> 
> 2020-06-19T09:29:24.215191+02:00 tst-msg03 cyrus/imap[32742]: auditlog: 
> traffic sessionid= 
> bytes_in=<2412> bytes_out=<1039>
> 
> 2020-06-19T09:29:32.855379+02:00 tst-msg03 cyrus/sync_client[32566]: 
> IOERROR: zero length response to MAILBOXES (end of file reached)
> 
> 2020-06-19T09:29:32.855682+02:00 tst-msg03 cyrus/sync_client[32566]: 
> IOERROR: zero length response to RESTART (end of file reached)
> 
> 2020-06-19T09:29:32.855713+02:00 tst-msg03 cyrus/sync_client[32566]: 
> Error in do_sync(): bailing out! Bad protocol
> 
> 2020-06-19T09:29:32.855738+02:00 tst-msg03 cyrus/sync_client[32566]: 
> Processing sync log file /var/lib/imap/sync/bck/log-run failed: Bad protocol
> 
> I tried again and I see the same result, but without the IOERROR:
> 
> 2020-06-19T09:36:51.153939+02:00 tst-msg03 cyrus/imap[32762]: login: 
> tst-msg03-bck.example.com [10.102.42.169] cyr_restore LOGIN User logged 
> in SESSIONID=
> 
> 2020-06-19T09:36:51.199742+02:00 tst-msg03 cyrus/imap[32762]: 
> example.com!user.utente^archivista: same message appears twice 32096 32097
> 
> (why twice? It expunges 32096 just above)
> 
> 2020-06-19T09:36:51.225525+02:00 tst-msg03 cyrus/imap[32762]: auditlog: 
> append sessionid= 
> mailbox= 
> uniqueid= uid=<32097> modseq=<46834> 
> sysflags= 

Re: backupd and sync_client IOERROR

2020-06-19 Thread Marco

Hi Ellie,

Il 19/06/2020 03:01, ellie timoney ha scritto:

I think you might need to add the usual recover entry to the START section:
 
recover   cmd="ctl_cyrusdb -r"


I notice this is missing from the backups documentation -- please let me know 
if it this sorts it out, and I'll fix it up!


wow, yes, it works. With this config, after the Cyrus restart the 
mailboxes.db and the skipstamps dbs are created and the error disappears 
from syslog.



2) Is there a way to delete a user from backupd host?

That's an interesting question...

Under normal use, if a user is deleted from their imap server, then, after the deletion has 
replicated to the backup server, and after the "backup_retention" period has elapsed, and 
after a subsequent "ctl_backups compact ..." has been run, then the backup file should be 
approximately empty.  But I think it will still exist; nothing actually deletes it yet.

To get rid of it manually, you can use "ctl_backups list ..." to find the 
filename where it exists on disk.  You can also find it by using cyr_dbtool to look up 
the user's key in backups.db.  You will then need to:

a) use cyr_dbtool to delete that user's key from backups.db
b) delete the named file (it will be like "/some/path/userid_XX")
c) also delete the corresponding "/some/path/userid_XX.index" file

Note that if the user still really exists, and a rolling sync_client replicates 
them to the same backup server again, then the backup file will be recreated 
(with a new XX) -- with the same problems as above wrt no-initial-sync.


It works!
Making some tests and the initial sync, it's easy with sync_client 
insert a wrong non-existent userid (I added "/user" before the userid). 
And that wrong userid is immediately created on the backupd server :)

With the above procedure I can delete the wrong entry.


Definitively I think that this backup feature is a very good idea.

I suggested the possibility to choose the recovery date (ie: all 
messages and mailboxes deleted or expunged from  to ). I 
hope you could add this feature.


Thank you very very much for these detailed answers!!


I now tried a restore, but I still have some problems. I very appreciate 
if you could read the following issue.


My goal is to recover a message that was removed (expunged) from the 
IMAP server, ignoring already existing messages.


It seems the "-x" flag could help. It seems it allows to restore and 
unexpunge a previously expunged messages in the IMAP server.


Yesterday I expunged the message 16ceead9802286784d7a54c5bc782891f76f2f2e:

2020-06-18T15:44:37.374556+02:00 tst-msg03 cyrus/imap[9461]: auditlog: 
expunge sessionid= 
mailbox= 
uniqueid= uid=<32092> modseq=<46828> 
sysflags= guid=<16ceead9802286784d7a54c5bc782891f76f2f2e>



I tried today to restore it:
cyr_restore -v -U -x tst-msg03.example.com -u 
utente.archivi...@example.com 16ceead9802286784d7a54c5bc782891f76f2f2e
example.com!user.utente^archivista: trying to keep 
uidvalidity(1550228105), uniqueid(nvf96uirjgfs6xaekwwc7g44), 
highestmodseq(46828)


and I see:
2020-06-19T09:29:24.030624+02:00 tst-msg03 cyrus/imap[32742]: login: 
tst-msg03-bck.example.com [10.102.42.169] cyr_restore LOGIN User logged 
in SESSIONID=


2020-06-19T09:29:24.162909+02:00 tst-msg03 cyrus/imap[32742]: auditlog: 
append sessionid= 
mailbox= 
uniqueid= uid=<32096> modseq=<46829> 
sysflags= guid=<16ceead9802286784d7a54c5bc782891f76f2f2e> 
messageid=


2020-06-19T09:29:24.163058+02:00 tst-msg03 cyrus/imap[32742]: auditlog: 
expunge sessionid= 
mailbox= 
uniqueid= uid=<32096> modseq=<46829> 
sysflags= guid=<16ceead9802286784d7a54c5bc782891f76f2f2e>


2020-06-19T09:29:24.167561+02:00 tst-msg03 cyrus/imap[32742]: auditlog: 
modseq sessionid= 
mailbox= 
uniqueid= highestmodseq=<46829>


2020-06-19T09:29:24.215036+02:00 tst-msg03 cyrus/imap[32742]: USAGE 
cyr_restore user: 0.042492 sys: 0.015435


2020-06-19T09:29:24.215191+02:00 tst-msg03 cyrus/imap[32742]: auditlog: 
traffic sessionid= 
bytes_in=<2412> bytes_out=<1039>


2020-06-19T09:29:32.855379+02:00 tst-msg03 cyrus/sync_client[32566]: 
IOERROR: zero length response to MAILBOXES (end of file reached)


2020-06-19T09:29:32.855682+02:00 tst-msg03 cyrus/sync_client[32566]: 
IOERROR: zero length response to RESTART (end of file reached)


2020-06-19T09:29:32.855713+02:00 tst-msg03 cyrus/sync_client[32566]: 
Error in do_sync(): bailing out! Bad protocol


2020-06-19T09:29:32.855738+02:00 tst-msg03 cyrus/sync_client[32566]: 
Processing sync log file /var/lib/imap/sync/bck/log-run failed: Bad protocol


I tried again and I see the same result, but without the IOERROR:

2020-06-19T09:36:51.153939+02:00 tst-msg03 cyrus/imap[32762]: login: 
tst-msg03-bck.example.com [10.102.42.169] cyr_restore LOGIN User logged 
in SESSIONID=


2020-06-19T09:36:51.199742+02:00 tst-msg03 cyrus/imap[32762]: 
example.com!user.utente^archivista: same message appears twice 32096 32097


(why twice? It expunges 32096 just above)


Re: backupd and sync_client IOERROR

2020-06-18 Thread ellie timoney
Hi Marco,

On Thu, Jun 18, 2020, at 10:19 PM, Marco wrote:
> Hello,
> 
>   I'm trying to configure backupd in rolling mode as a final setup. 
> Running a first backup on few users
> 
> sync_client -A -n bck -z -v -v
> 
> after a while the process die with:
> 
> cyrus/sync_client[9540]: MESSAGE received NO response: 
> IMAP_PROTOCOL_ERROR Protocol error
> cyrus/sync_client[9540]: do_folders(): update failed: 
> example.com!user.utente^archivista 'Bad protocol'
> cyrus/sync_client[9540]: IOERROR: do_user_main: Bad protocol for 
> utente.archivi...@example.com to [no channel] (tst-msg03-bck.example.com)
> Error from do_user(utente.archivi...@example.com): bailing out!
> cyrus/sync_client[9540]: Error in 
> do_user(utente.archivi...@example.com): bailing out!
> 
> The backupd host doesn't complain:
> 
> 2020-06-18T11:50:30.528584+02:00 tst-msg03-bck backupd[1308172]: 
> creating sql_db 
> /var/spool/cyr_backup/u/utente.archivista@example.com_cxlsmX.index
> 2020-06-18T11:56:44.971042+02:00 tst-msg03-bck backupd[1308172]: login: 
> tst-msg03.example.com [10.102.42.168] cyr_backup LOGIN User logged in
> 
> So I repeated the same command, but with less verbosity:
> 
> sync_client -A -n bck -z -v
> 
> and it worked well, without errors. Uhm...
> 
> Maybe, before to run the first initial backup (sync_client -A), do I 
> have to stop the sync_client already working in rolling mode?

Rolling mode only makes incremental updates, so if you're starting from a 
server that already has existing data, you should do the first manual initial 
backups before enabling the rolling mode.

It's generally safe to run a manual sync_client side-by-side with a rolling one 
(they will lock things properly and keep out of each other's way).  But the 
exception to this is that a rolling mode replication can't coherently update an 
uninitialised replica (since it only sends new changes recorded in the sync 
log, but not the pre-existing data).  You need to do a full manual replication 
first, to ensure both sides are the same, before rolling mode can work 
sensibly.  This is the same regardless of whether your replica is a normal 
replica or a backupd one.

As you saw, replicating again manually will probably fix you up, at least. :)

> http://www.cyrusimap.org/imap/reference/admin/backups.html says "Update 
> cyrus.conf(5) to add a sync_client(8) invocation to the SERVICES section 
> specifying (at least) the -r and -n channel options.". Really maybe you 
> intend START. In SERVICE I need to specify a listen too, this is not 
> very clear to me.

This is wrong.  It should be the DAEMON section, not the SERVICES section.  I'm 
fixing this now, so the website should be updated shortly.

You can also run it from the START section, but note that if the sync_client 
exits early for some reason, master will not restart it.  If you run it from 
the START section, you will need to monitor/restart it yourself.

> Around of backupd I would ask two other questions:
> 
> 1) The backupd host always claims this:
> 
> backupd[1309970]: DBERROR: reading /var/lib/imap/db/skipstamp, assuming 
> the worst: No such file or directory
> 
> Could you tell me how to fix that DBERROR too?

I think you might need to add the usual recover entry to the START section:

recover   cmd="ctl_cyrusdb -r"

I notice this is missing from the backups documentation -- please let me know 
if it this sorts it out, and I'll fix it up!
 
> 2) Is there a way to delete a user from backupd host?

That's an interesting question...

Under normal use, if a user is deleted from their imap server, then, after the 
deletion has replicated to the backup server, and after the "backup_retention" 
period has elapsed, and after a subsequent "ctl_backups compact ..." has been 
run, then the backup file should be approximately empty.  But I think it will 
still exist; nothing actually deletes it yet.

To get rid of it manually, you can use "ctl_backups list ..." to find the 
filename where it exists on disk.  You can also find it by using cyr_dbtool to 
look up the user's key in backups.db.  You will then need to:

a) use cyr_dbtool to delete that user's key from backups.db
b) delete the named file (it will be like "/some/path/userid_XX")
c) also delete the corresponding "/some/path/userid_XX.index" file

Note that if the user still really exists, and a rolling sync_client replicates 
them to the same backup server again, then the backup file will be recreated 
(with a new XX) -- with the same problems as above wrt no-initial-sync.

Cheers,

ellie

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus