cyrus replication question
while trying to setup cyrus replication i am getting the following error on the secondary server ( the one that is getting a copy of everything) Dec 9 12:35:34 mars master[7701]: about to exec /usr/local/cyrus/bin/sync_server Dec 9 12:35:34 mars syncserver[7701]: executed Dec 9 12:35:34 mars syncserver[7701]: accepted connection Dec 9 12:35:34 mars syncserver[7701]: cmdloop(): startup Dec 9 12:35:34 mars master[7694]: process 7701 exited, status 75 Dec 9 12:35:34 mars master[7694]: service syncserver pid 7701 in BUSY state: terminated abnormally here are the settings from both imapd.conf and cyrus.conf for both master and backup server. master server (from where everything is being copied) cyrus.conf under START section: syncclientcmd="/usr/local/cyrus/bin/sync_client -r -t" imapd.conf #replication configurations sync_host: server.mydomain.com sync_authname: cyrus sync_passwd: password sync_machineid: 1 sync_log: 1 secondary (to where everything is bein copied) cyrus.conf SERVICES { syncservercmd="sync_server" listen="csync" Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus replication question
another question : do i need to create mailboxes on the replica server or will they be created by the replication process ? > while trying to setup cyrus replication > i am getting the following error > on the secondary server ( the one that is getting a copy of everything) > > Dec 9 12:35:34 mars master[7701]: about to exec > /usr/local/cyrus/bin/sync_server > Dec 9 12:35:34 mars syncserver[7701]: executed > Dec 9 12:35:34 mars syncserver[7701]: accepted connection > Dec 9 12:35:34 mars syncserver[7701]: cmdloop(): startup > Dec 9 12:35:34 mars master[7694]: process 7701 exited, status 75 > Dec 9 12:35:34 mars master[7694]: service syncserver pid 7701 in BUSY > state: terminated abnormally > > here are the settings from both imapd.conf and cyrus.conf > for both master and backup server. > > master server > (from where everything is being copied) > > cyrus.conf > > under START section: > syncclientcmd="/usr/local/cyrus/bin/sync_client -r -t" > > > imapd.conf > > #replication configurations > sync_host: server.mydomain.com > sync_authname: cyrus > sync_passwd: password > sync_machineid: 1 > sync_log: 1 > > secondary > (to where everything is bein copied) > cyrus.conf > > SERVICES { > syncservercmd="sync_server" listen="csync" > > > > > > > Cyrus Home Page: http://cyrusimap.web.cmu.edu/ > Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki > List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html > Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus replication question
i started looking around for one i found that for whatever reason both sync and some other default process was using 2005/tcp port (in /etc/services ) after taking care of that i kicked off on sync_client on master server (after logging in as cyrus) on master server i get can not connect to server 'replica.mydomain.com trying in 15 seconds. and in /var/log/messages the following comes up (still on master server) Dec 11 16:09:56 master_server sync_client[93038]: TLS client engine: cannot load CA data Dec 11 16:09:56 master_server sync_client[93038]: Doing a peer verify Dec 11 16:09:56 master_server sync_client[93038]: verify error:num=18:self signed certificate Dec 11 16:09:56 master_server sync_client[93038]: starttls: TLSv1 with cipher AES256-SHA (256/256 bits new) no authentication Dec 11 16:09:57 master_server sync_client[93038]: No worthy mechs found Dec 11 16:09:57 master_server sync_client[93038]: couldn't authenticate to backend server: no mechanism available - /var/log/messeges on replica server is not saying anything at this point but /var/log/debug.log has the following : Dec 11 15:56:33 replica_server syncserver[92364]: accepted connection Dec 11 15:56:33 replica_server syncserver[92364]: cmdloop(): startup Dec 11 15:56:33 replica_server syncserver[92364]: mystore: starting txn 2147483668 Dec 11 15:56:33 replica_server syncserver[92364]: mystore: committing txn 2147483668 Dec 11 15:56:49 replica_server syncserver[92364]: accepted connection Dec 11 15:56:49 replica_server syncserver[92364]: cmdloop(): startup Dec 11 15:56:49 replica_server syncserver[92364]: mydelete: starting txn 2147483669 Dec 11 15:56:49 replica_server syncserver[92364]: mydelete: committing txn 2147483669 Dec 11 15:56:49 replica_server syncserver[92364]: mystore: starting txn 2147483670 Dec 11 15:56:49 replica_server syncserver[92364]: mystore: committing txn 2147483670 > while trying to setup cyrus replication > i am getting the following error > on the secondary server ( the one that is getting a copy of everything) > > Dec 9 12:35:34 mars master[7701]: about to exec > /usr/local/cyrus/bin/sync_server > Dec 9 12:35:34 mars syncserver[7701]: executed > Dec 9 12:35:34 mars syncserver[7701]: accepted connection > Dec 9 12:35:34 mars syncserver[7701]: cmdloop(): startup > Dec 9 12:35:34 mars master[7694]: process 7701 exited, status 75 > Dec 9 12:35:34 mars master[7694]: service syncserver pid 7701 in BUSY > state: terminated abnormally > > here are the settings from both imapd.conf and cyrus.conf > for both master and backup server. > > master server > (from where everything is being copied) > > cyrus.conf > > under START section: > syncclientcmd="/usr/local/cyrus/bin/sync_client -r -t" > > > imapd.conf > > #replication configurations > sync_host: server.mydomain.com > sync_authname: cyrus > sync_passwd: password > sync_machineid: 1 > sync_log: 1 > > secondary > (to where everything is bein copied) > cyrus.conf > > SERVICES { > syncservercmd="sync_server" listen="csync" > > > > > > > Cyrus Home Page: http://cyrusimap.web.cmu.edu/ > Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki > List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html > Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus replication question
On Mon, Dec 11, 2006 at 04:35:24PM -0500, [EMAIL PROTECTED] wrote: > > i started looking around > for one i found that for whatever reason > both sync and some other default process was using 2005/tcp port > (in /etc/services ) > after taking care of that > > i kicked off on sync_client on master server > (after logging in as cyrus) > on master server i get > can not connect to server 'replica.mydomain.com trying in 15 seconds. > > and in /var/log/messages the following comes up > (still on master server) > > Dec 11 16:09:56 master_server sync_client[93038]: TLS client engine: > cannot load CA data > Dec 11 16:09:56 master_server sync_client[93038]: Doing a peer verify > Dec 11 16:09:56 master_server sync_client[93038]: verify error:num=18:self > signed certificate > Dec 11 16:09:56 master_server sync_client[93038]: starttls: TLSv1 with > cipher AES256-SHA (256/256 bits new) no authentication > Dec 11 16:09:57 master_server sync_client[93038]: No worthy mechs found > Dec 11 16:09:57 master_server sync_client[93038]: couldn't authenticate to > backend server: no mechanism available > - Check tls_key_file: tls_ca_file: tls_cert_file: in your imapd.conf WBR -- Dmitriy Kirhlarov OILspace, 26 Leninskaya sloboda, bld. 2, 2nd floor, 115280 Moscow, Russia P:+7 495 105 7247 ext.208 F:+7 495 105 7246 E:[EMAIL PROTECTED] OILspace - The resource enriched - www.oilspace.com Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus replication question
On Mon, Dec 11, 2006 at 04:36:38PM -0500, [EMAIL PROTECTED] wrote: > > another question : > > do i need to create mailboxes > on the replica server or will they be created by the replication > process ? You don't need create mailboxes manually. WBR -- Dmitriy Kirhlarov OILspace, 26 Leninskaya sloboda, bld. 2, 2nd floor, 115280 Moscow, Russia P:+7 495 105 7247 ext.208 F:+7 495 105 7246 E:[EMAIL PROTECTED] OILspace - The resource enriched - www.oilspace.com Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus replication question
i did as you've said this is now the output of synctest S: * STARTTLS S: * OK mars.edpausa.com Cyrus sync server v2.3.7 Authentication failed. generic failure Security strength factor: 0 /var/log/messages shows : Dec 12 20:56:21 mars syncserver[489]: login: [192.168.0.201] cyrus PLAIN+TLS User logged in Dec 12 20:56:35 mars syncserver[313]: starttls: TLSv1 with cipher AES256-SHA (256/256 bits new) no authentication Dec 12 20:56:35 mars syncserver[313]: login: [192.168.0.201] cyrus PLAIN+TLS User logged in it seems that something did happen it replicated all the active mailboxes ( there are a few innactive from the people who recently left and a few test mailboxes) and replicated only active subfolders of these boxes. so out of 121GB only 58 GB have been replicated i see in logs things like this Dec 12 20:56:45 mars syncserver[313]: Expunged 1 messages from user.stas Dec 12 21:01:50 mars syncserver[313]: skiplist: recovered /var/imap/user/e/edpa.seen.313 (2 records, 11608 bytes) in 0 seconds so something is defetnly going on but why is it replicating only active mailboxes/subfolders ?? Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus replication question
On 12 Dec 2006, at 21:08, [EMAIL PROTECTED] wrote: but why is it replicating only active mailboxes/subfolders ?? If you're enabling replication on an already in-production server, you'll probably want to run something like: sync_client -l -v -u That will cause all existing mailboxes to be replicated. Thereafter, you shouldn't need to intervene (much). :wes Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus replication question
do thay have to be separated (user names i mean) by space or coma or something else ?? i am getting these Dec 12 21:48:34 mars syncserver[313]: can not unlink /var/imap/user/l/lenny,.seen: No such file or directory Dec 12 21:48:34 mars syncserver[313]: can not unlink /var/imap/user/l/lenny,.mboxkey: No such file or directory when i do name, "name, name" or "name name" or "name,name" > On 12 Dec 2006, at 21:08, [EMAIL PROTECTED] wrote: >> but why is it replicating only active mailboxes/subfolders ?? > > If you're enabling replication on an already in-production server, > you'll probably want to run something like: > > sync_client -l -v -u > > That will cause all existing mailboxes to be replicated. Thereafter, > you shouldn't need to intervene (much). > > :wes > Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus replication question
Like this: sync_client -l -v -u userA userB userC ... Most Unix commands function this way. :wes On 12 Dec 2006, at 21:56, [EMAIL PROTECTED] wrote: do thay have to be separated (user names i mean) by space or coma or something else ?? Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus replication question
did as you advised now i see folder fro all the users on replica machine but the size of the /imap/users has not changed. basically whatever is inactive be it a full mailbox or an active mailbox's subfolder is not being replicated. Wesley Craig wrote: > Like this: > > sync_client -l -v -u userA userB userC ... > > Most Unix commands function this way. > > :wes > > On 12 Dec 2006, at 21:56, [EMAIL PROTECTED] wrote: >> do thay have to be separated (user names i mean) by space or coma or >> something else ?? > > Cyrus Home Page: http://cyrusimap.web.cmu.edu/ > Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki > List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html > Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus replication question
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 still battling with replication :) i've noticed that for only one of the users i get the following error Error from do_user(-l): bailing out! in the list archives i found some one with a similar issue (at least is sounded like it) the advice was to to reconstruct the mailbox but on which machine - master or replica ? also i don't see that error in the logs it comes as output of sync_client -l -v -u (user list) -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBRYFisdbpM15f9s9nAQLLKQP+LZKXIPDkTr+GPKWjNxZxQMU/jco9xNxU aALb+Xsh04Fhu6IovI4rXLOHBc7m46kTI/2SHqhlTtdFcVxJYfp/sPFxSJncnPjd ECDDfeYte1Yj+jUO12yzUze8YWIPEZnUEnAE+ia4TK97C9oP//r/XRRzCt9E1XqA CNyoZxDQNzE= =8hR2 -END PGP SIGNATURE- Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus replication question
still battling with replication :) i've noticed that for only one of the users i get the following error Error from do_user(-l): bailing out! in the list archives i found some one with a similar issue (at least is sounded like it) the advice was to to reconstruct the mailbox but on which machine - master or replica ? also i don't see that error in the logs it comes as output of sync_client -l -v -u (user list) Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus replication question
On 14 Dec 2006, at 09:41, stas khromoy wrote: i've noticed that for only one of the users i get the following error Error from do_user(-l): bailing out! in the list archives i found some one with a similar issue (at least is sounded like it) That's the error that you'd get from sync_client. Probably a more detailed error is in syslog, and even more detail is likely to be available from sync_server on the replica. the advice was to to reconstruct the mailbox but on which machine - master or replica ? reconstruct on the master. Probably I'd run sync_reset on the user in question on the replica as well. Then sync_client -u again. :wes also i don't see that error in the logs it comes as output of sync_client -l -v -u (user list) The error in the logs is probably more detailed, something like "server responded with NO" etc. :wes Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus replication question
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 thanks for all your help, wesley i'll try this as soon as i can there is one question that is till puzzling me thou when it replicates, it ignores inactive folders for some reason i looked at the size of my imap/user folder on replica (1.9gb) and size of the same folder on master (2.4gb) i am sure other mailboxes have the same issue since the size of imap/user on replica is smaller then on master any ideas ? or is it normal i checked all the logs i dont see anything that could hint on the problem if there is one by inactive folders i mean folders that u use to store emails and don't touch often thanks again Wesley Craig wrote: > On 14 Dec 2006, at 09:41, stas khromoy wrote: >> i've noticed that for only one >> of the users >> i get the following error >> >> Error from do_user(-l): bailing out! >> >> in the list archives i found some one with a similar issue >> (at least is sounded like it) > > That's the error that you'd get from sync_client. Probably a more > detailed error is in syslog, and even more detail is likely to be > available from sync_server on the replica. > >> the advice was to to reconstruct the mailbox >> but on which machine - master or replica ? > > reconstruct on the master. Probably I'd run sync_reset on the user in > question on the replica as well. Then sync_client -u again. > > :wes > >> also i don't see that error in the logs >> it comes as output of sync_client -l -v -u (user list) > > The error in the logs is probably more detailed, something like "server > responded with NO" etc. > > :wes > -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBRYGbatbpM15f9s9nAQIAVwP+N2cdqdlhnhFsJdKvhqez2SvZ1KDI/HcT 2AzIdLtRiLztLr+k0aOwtO6lUXFogA/iWULDlkDyqhaHfa0v0mXzDK9fb8tJexXS e96AtNRjwb+sVrXKvRzcMsgtMMCdEij5MIC1rLm60lTNzi1T75WC0cWQ2BkKRDbs HVkQ4zEPXiA= =BG0M -END PGP SIGNATURE- Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus replication question
On 14 Dec 2006, at 13:43, stas khromoy wrote: i looked at the size of my imap/user folder on replica (1.9gb) and size of the same folder on master (2.4gb) First, I'd try to fix the "bailing out" problem you're having. Or, if you want to just get things closer to correct, invoke: sync_client -v -l -u username once for every user. That way, when it encounters one bad user, it won't stop. Of course, you may have many "bad" users, as well. If you still have problems once all the "bad" users are corrected, then we can debug the size differences. :wes Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html