Re: what is the best way to migrate
On Thu, 2006-06-22 at 20:53 +0200, Rudy Gevaert wrote: For no downtime we would kick of the sync in advance, and after some time we would have an almost identical copy. Then let the users connect to the new server(s) and let lmtp deliver to the new server(s). Then do a last sync to get the last mails over. You may also want to explore putting an IMAP proxy, like perdition, in front of the IMAP server(s) temporarily. This would let you migrate users one at a time and transparently. Cheers, Mike signature.asc Description: This is a digitally signed message part Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
what is the best way to migrate
Well we re planning to change our old cyrus imap server to a brand new one and we re wondering what would be the best way to do that. our old server is based on a cyrus 2.2.10 and we will migrate to a 2.3.6 version. we have to copy the mailboxes from an old disks to new ones with a different partition layout we move from 26 partiton to 4 partitions. we would love to keep the state of messages in our users folders and to minimize the downtine. what is the best way to migrate? Thanks in advance for any help Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: what is the best way to migrate
On Thu, 22 Jun 2006 11:40:46 +0200 Eric Doutreleau [EMAIL PROTECTED] wrote: Well we re planning to change our old cyrus imap server to a brand new one and we re wondering what would be the best way to do that. our old server is based on a cyrus 2.2.10 and we will migrate to a 2.3.6 version. we have to copy the mailboxes from an old disks to new ones with a different partition layout we move from 26 partiton to 4 partitions. we would love to keep the state of messages in our users folders and to minimize the downtine. what is the best way to migrate? Thanks in advance for any help http://www.linux-france.org/prj/imapsync/ helped me several times. it's an awesome tool imho ;) HTH, -- Timo Schoeler | http://riscworks.net/~tis | [EMAIL PROTECTED] RISCworks -- Perfection is a powerful message ISP | POWER PowerPC afficinados | Networking, Security, BSD services GPG Key fingerprint = B5F6 68A4 EC45 C309 6770 38C4 50E8 2740 9E0C F20A There are 10 types of people in the world. Those who understand binary and those who don't. Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: what is the best way to migrate
2006/6/22, Timo Schoeler [EMAIL PROTECTED]: On Thu, 22 Jun 2006 11:40:46 +0200 Eric Doutreleau [EMAIL PROTECTED] wrote: Well we re planning to change our old cyrus imap server to a brand new one and we re wondering what would be the best way to do that. our old server is based on a cyrus 2.2.10 and we will migrate to a 2.3.6 version. we have to copy the mailboxes from an old disks to new ones with a different partition layout we move from 26 partiton to 4 partitions. we would love to keep the state of messages in our users folders and to minimize the downtine. what is the best way to migrate? Thanks in advance for any help http://www.linux-france.org/prj/imapsync/ helped me several times. it's an awesome tool imho ;) HTH, Hi. Now i'm using imapsync to migrate over 40k accounts.. I'ts quite good tool ;-) greets AK Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: what is the best way to migrate
Timo Schoeler wrote: http://www.linux-france.org/prj/imapsync/ helped me several times. it's an awesome tool imho ;) I'm still looking for a way to do the sync without knowing the password of the user. Any idea's? Rudy -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert [EMAIL PROTECTED] tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur Direction ICT, Infrastructure dept. Groep Systemen Systems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: what is the best way to migrate
Rudy Gevaert wrote: Timo Schoeler wrote: http://www.linux-france.org/prj/imapsync/ helped me several times. it's an awesome tool imho ;) I'm still looking for a way to do the sync without knowing the password of the user. Any idea's? I've used a dummy ldap backend with a copy of all our accounts.. but with the same known password. You are down during the migration.. but it's fairly easy to setup and cutover. Jared Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: what is the best way to migrate
Running imapsync in auth mode. you should enable the IMAPS (port 993) for this. I use ssh to pipe the mail tar to the target server, extract the tar file on the user folder, run reconstruct, and finally use imapsync to subscribe the imap folders and recreate the mail flag. We have already built a complete cyrus migration tool with the help of reconstruct and imapsync. Patrick - Original Message - From: Rudy Gevaert [EMAIL PROTECTED] To: cyrus Mailing List info-cyrus@lists.andrew.cmu.edu Sent: Thursday, June 22, 2006 10:12 PM Subject: Re: what is the best way to migrate Timo Schoeler wrote: http://www.linux-france.org/prj/imapsync/ helped me several times. it's an awesome tool imho ;) I'm still looking for a way to do the sync without knowing the password of the user. Any idea's? Rudy -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert [EMAIL PROTECTED] tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur Direction ICT, Infrastructure dept. Groep Systemen Systems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: what is the best way to migrate
I'm still looking for a way to do the sync without knowing the password of the user. Any idea's? Rudy I modified imapsync and the Mail::IMAPClient it uses module slightly to let myself do exactly this a few months ago. Basically, I messed with IMAPClient to add in PLAIN authentication as well as SSL and then modified imapsync to authenticate as a cyrus admin (with *that* password), and authorize as the user given from imapsync's input. (Or did I switch up authenticate and authorize - ugh.. I always get those confused) I can then sync as if I were the user without any sort of a problem - I've been using it (and a modified uw-imapd) to migrate users from uw-imapd to cyrus now without needing a user's password.. My code itself is a horrible hackjob I'm embarassed to have actually been responsible for, but I was under pressure and time constraints (and honestly, I had had only a few months of perl experience at that point), I can show you the basic gist if you want. It was pretty straightforward to do, especially if you don't mind making it specific for your environment. Matt Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: what is the best way to migrate
Patrick T. Tsang wrote: Running imapsync in auth mode. you should enable the IMAPS (port 993) for this. I use ssh to pipe the mail tar to the target server, extract the tar file on the user folder, run reconstruct, and finally use imapsync to subscribe the imap folders and recreate the mail flag. We have already built a complete cyrus migration tool with the help of reconstruct and imapsync. Would you be so kind to share this tool with the list? :) Thanks in advance, Rudy Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: what is the best way to migrate
Jared Watkins wrote: I'm still looking for a way to do the sync without knowing the password of the user. Any idea's? I've used a dummy ldap backend with a copy of all our accounts.. but with the same known password. You are down during the migration.. but it's fairly easy to setup and cutover. Well, we are going to migrate nearly 4 accounts, with some accounts being 4GB. Our aim is no downtime. I had a look at the FAQ of imapsync and it seems that I can authenticate as the admin user, but authorize as a normal user. I hope that works! It would be great! For no downtime we would kick of the sync in advance, and after some time we would have an almost identical copy. Then let the users connect to the new server(s) and let lmtp deliver to the new server(s). Then do a last sync to get the last mails over. I haven't tested this, so maybe I'm to hopefull. Maybe we can do it for small/big accounts only. I'll see if I come to that. Rudy Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: what is the best way to migrate
On Thu, 2006-06-22 at 20:53 +0200, Rudy Gevaert wrote: I had a look at the FAQ of imapsync and it seems that I can authenticate as the admin user, but authorize as a normal user. I hope that works! It would be great! yes, it works. For no downtime we would kick of the sync in advance, and after some time we would have an almost identical copy. Then let the users connect to the new server(s) and let lmtp deliver to the new server(s). Then do a last sync to get the last mails over. the tricky bit is to kick the user out of the old server. I haven't found a good way to do this. so basically you'll do: 0) do a rough sync 1) kick user out (how?) 2) stop delivery for user 3) update which server is in charge of user 4) start delivery for user 5) sync remaining changes the second tricky bit is you can't run imapsync with --delete, since new messages can have been delivered, or created by the user (think INBOX.Sent). this means deleted (expunged) messages may reappear. a similar and more serious problem is that flags set on the new server before the final sync is done will be reset. I haven't tested this, so maybe I'm to hopefull. Maybe we can do it for small/big accounts only. I'll see if I come to that. my servers can only transfer mail data at about 150 KiB/s, so you need to have patience. syncing flags only is much faster, of course. what we do: * we add a move request to our user database * when time for the move is imminent, LDAP is updated to include an extra attribute, mailPause. * user authentication will not succeed when this attribute is True * mail will be queued while this attribute is True * we now start to move the data using imapsync * when it's done, the request is closed and the LDAP attributes mailPause and IMAPserver are updated. * queued e-mail is delivered doing a pre-sync to reduce downtime for the user would be a good idea, although it wouldn't make much difference in our current set up. typically the user is locked out for an hour, mostly due to our LDAP updates only happening every 50 minutes. pre-sync and instanteous LDAP updates would be nice, it should be down to 1-2 minutes then, even for big users. -- Kjetil T. Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
