Re: replication, how to see if it 'up to date'
Hi Bron!
Bron Gondwana wrote:
Our monitoring script that runs every 2 minutes does a du -s on the
{configdirectory}/sync directory and notifies us if it gets big (where
big is obviously a per-instance variable)
A less frequently run script searches the ps output for copies of
sync_client. It then looks in they sync directory, and for any
file called log-{N} it checks if there is a sync_client with PID
{N}. If not, it tries to run it with -r -f. Finally, if there
was no sync_client at all, it launches a new one with -r.
Interesting. So running the syn_client with -r -f file let it catch up?
Thirdly we have a tool which makes an imap connection to each of
the master and replica (we run an imapd on a different port on the
replica purely for this purpose) and does a bunch of checks
on each mailbox. It has to run as each user to get SEEN state.
This script has multiple levels of test, right up to one that
actually randomly picks a few messages and fetches their bodies
for comparison. It always checks at least flags and everything
you can STATUS for. This tool also has an optional flag which
can ssh to the apropriate servers and run reconstruct on the
mailboxes as well as sync_client -u on the offending user if it
finds inconsistencies. Obviously, there's a possibility that
it catches changes where sync_client just hasn't caught up yet,
but it retries multiple times with a tunable delay, so generally
it will get the same value on one of the retries unless the
folder is seeing a lot of traffic.
Also very interesting!
Thanks in advance!
--
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Rudy Gevaert [EMAIL PROTECTED] tel:+32 9 264 4734
Directie ICT, afd. Infrastructuur Direction ICT, Infrastructure dept.
Groep Systemen Systems group
Universiteit Gent Ghent University
Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: replication, how to see if it 'up to date'
On Thu, 28 Sep 2006, Rudy Gevaert wrote: Does anybody know how to see if the sync replication is up to date? All suggestions are welcome. I replicate all users from all masters to replicas about once a month and check the output for any inconsistancies. Something like: cyrus-23[cyrus:cyrus]$ replicate -s cyrus-24 -v -v -u dpc22 USER dpc22 USER_ALL dpc22 ENDUSER where replicate is just a little wrapper around sync_client. We also maintain databases of MD5 checksums for messages and cache entries, generated by make_md5. -- David Carter Email: [EMAIL PROTECTED] University Computing Service,Phone: (01223) 334502 New Museums Site, Pembroke Street, Fax: (01223) 334679 Cambridge UK. CB2 3QH. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: replication, how to see if it 'up to date'
On Thu, 28 Sep 2006 09:55:53 +0100 (BST), David Carter [EMAIL PROTECTED]
said:
On Thu, 28 Sep 2006, Rudy Gevaert wrote:
Does anybody know how to see if the sync replication is up to date? All
suggestions are welcome.
I replicate all users from all masters to replicas about once a month and
check the output for any inconsistancies. Something like:
cyrus-23[cyrus:cyrus]$ replicate -s cyrus-24 -v -v -u dpc22
USER dpc22
USER_ALL dpc22
ENDUSER
where replicate is just a little wrapper around sync_client.
We also maintain databases of MD5 checksums for messages and cache
entries, generated by make_md5.
Our monitoring script that runs every 2 minutes does a du -s on the
{configdirectory}/sync directory and notifies us if it gets big (where
big is obviously a per-instance variable)
A less frequently run script searches the ps output for copies of
sync_client. It then looks in they sync directory, and for any
file called log-{N} it checks if there is a sync_client with PID
{N}. If not, it tries to run it with -r -f. Finally, if there
was no sync_client at all, it launches a new one with -r.
Thirdly we have a tool which makes an imap connection to each of
the master and replica (we run an imapd on a different port on the
replica purely for this purpose) and does a bunch of checks
on each mailbox. It has to run as each user to get SEEN state.
This script has multiple levels of test, right up to one that
actually randomly picks a few messages and fetches their bodies
for comparison. It always checks at least flags and everything
you can STATUS for. This tool also has an optional flag which
can ssh to the apropriate servers and run reconstruct on the
mailboxes as well as sync_client -u on the offending user if it
finds inconsistencies. Obviously, there's a possibility that
it catches changes where sync_client just hasn't caught up yet,
but it retries multiple times with a tunable delay, so generally
it will get the same value on one of the retries unless the
folder is seeing a lot of traffic.
Bron.
--
Bron Gondwana
[EMAIL PROTECTED]
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
