Re: Adding archiving to an existing Cyrus installation
Hi Nic, Sorry I didn't get back to answering you on this the other day! So... this one is kinda tricky, because everything is going to be on "spool", but here's how I would do it. Before: /mnt/smalldisk/conf -> meta files only /mnt/bigdisk/spool -> all email right now Stage 1: splitting: /mnt/smalldisk/conf /mnt/bigdisk/spool /mnt/bigdisk/spool-archive -> empty And set archivepartition-default to /mnt/bigdisk/spoolarchive Now you need to run an initial cyr_expire. This will take a long time, but it should be able to use hardlinks to move the files - it's using cyrus_copyfile. Once cyr_expire has finished an most of your email is moved into spool- archive, shut down cyrus. mv /mnt/bigdisk/spool /mnt/smalldisk/spool And set partition-default to /mnt/smalldisk/spool That way your downtime is only while the small remaining spool gets moved to the other disk. Bron. On Sun, 5 Nov 2017, at 02:58, Nic Bernstein wrote: > Thanks much to you both for your comments and suggestions. We had > already considered creating a temporary "staging" partition and > shuffling mailboxes around, as Michael discussed, but have the same > reservations about it. Since we're dealing with nearly 6TB of data, > most of it old, this scheme would introduce considerable > disruption to a> very active mail system. We have a hard time getting a two > hour > maintenance window, and this would take days! > > Bron, other Fastmailers, any thoughts?? > -nic > > On 11/03/2017 11:20 AM, Michael Menge wrote: >> Hi, >> >> Quoting Reinaldo Gil Lima de Carvalho: >> >>> I think that singleinstancestore (message hard links) will not >>> survive when >>> moving from one partition to the other and storage total size will >>> increase >>> significantly. >>> >> >> thanks for the hint. This was not a problem while migration to the >> meta-data partition, >> as the mails stayed on the same partition (as in file-system and not>> >> cyrus-partition) >> and only hardlinks where change at all. >> >> So one more reason for an other migration path. >> >> >>> >>> 2017-11-03 12:22 GMT-03:00 Michael Menge >>> >> Hi Nic, Quoting Nic Bernstein : Friends, > I have a client with Cyrus 2.5.10 installed. Last year we > migrated> their > old 2.3.18 system to 2.5.10, with an eye towards an eventual > move to> 3.0.x. Based on Bron's most excellent email of last year, > ([Subject: Cyrus > database and file usage data] from Cyrus Devel of 8 January 2016)> we > used a > tiered layout for the storage: > > The main categories are: > > * Config directory (ssd) [/var/lib/imap] > o sieve > o seen > o sub > o quota > o mailboxes.db > o annotations.db > * Ephemeral [/var/run/cyrus -- in tmpfs] > o tls_sessions.db > o deliver.db > o statuscache.db > o proc (directory) > o lock (directory) > * Mailbox data [typical 2.5.X usage] > o Meta-data (ssd) > + header > + index > + cache > + expunge > + squat (search index) > + annotations > o Spool data (disk: raidX) > + messages (rfc822 blobs) > > We sized the Fast SSD pool (this is three-drive mirrors on ZFS) > to be> extra large, so it could eventually handle "Hot" data, and left > about 300GB > free there. Data, on spinning media, is currently 5.74TB with > 4.8TB free > (RAID10). Metadata is 35GB and /var/lib/imap is 8GB, all of which> > is in > the Fast pool. > > Now the client is ready to take the dive into v3.0, and I'm > trying to> figure out how to put "archive" operation in effect. > > I have read the documentation (hell, I wrote most of it) and > understand > the settings, but what I cannot quite wrap my brain around is > this:> There > is already all of this data sitting in all of these data > partitions> (we use > a total of 34 separate partitions each for data & metadata) so how> > do I > make the transition to separate archive partitions, since all that> > data is > on the "slow" drives? Can I just reassign all of the current data> > partitions to archivedata partitions, define the new set of "Hot" > data> partitions on the Fast pool, and let 'er rip, or what? > > I promise, if you tell me, I'll write it up as real > documentation. :-)> > We are interested in such a migration too. Our fallback plan, if we don't find a better way to do it is, do use the same method as we introduced the ssd meta-data partition. 1. We created a new partition in our cyrus configuration, 2. we moved moved
Re: Adding archiving to an existing Cyrus installation
Thanks much to you both for your comments and suggestions. We had already considered creating a temporary "staging" partition and shuffling mailboxes around, as Michael discussed, but have the same reservations about it. Since we're dealing with nearly 6TB of data, most of it old, this scheme would introduce considerable disruption to a very active mail system. We have a hard time getting a two hour maintenance window, and this would take days! Bron, other Fastmailers, any thoughts?? -nic On 11/03/2017 11:20 AM, Michael Menge wrote: Hi, Quoting Reinaldo Gil Lima de Carvalho: I think that singleinstancestore (message hard links) will not survive when moving from one partition to the other and storage total size will increase significantly. thanks for the hint. This was not a problem while migration to the meta-data partition, as the mails stayed on the same partition (as in file-system and not cyrus-partition) and only hardlinks where change at all. So one more reason for an other migration path. 2017-11-03 12:22 GMT-03:00 Michael Menge : Friends, I have a client with Cyrus 2.5.10 installed. Last year we migrated their old 2.3.18 system to 2.5.10, with an eye towards an eventual move to 3.0.x. Based on Bron's most excellent email of last year, ([Subject: Cyrus database and file usage data] from Cyrus Devel of 8 January 2016) we used a tiered layout for the storage: The main categories are: * Config directory (ssd) [/var/lib/imap] o sieve o seen o sub o quota o mailboxes.db o annotations.db * Ephemeral [/var/run/cyrus -- in tmpfs] o tls_sessions.db o deliver.db o statuscache.db o proc (directory) o lock (directory) * Mailbox data [typical 2.5.X usage] o Meta-data (ssd) + header + index + cache + expunge + squat (search index) + annotations o Spool data (disk: raidX) + messages (rfc822 blobs) We sized the Fast SSD pool (this is three-drive mirrors on ZFS) to be extra large, so it could eventually handle "Hot" data, and left about 300GB free there. Data, on spinning media, is currently 5.74TB with 4.8TB free (RAID10). Metadata is 35GB and /var/lib/imap is 8GB, all of which is in the Fast pool. Now the client is ready to take the dive into v3.0, and I'm trying to figure out how to put "archive" operation in effect. I have read the documentation (hell, I wrote most of it) and understand the settings, but what I cannot quite wrap my brain around is this: There is already all of this data sitting in all of these data partitions (we use a total of 34 separate partitions each for data & metadata) so how do I make the transition to separate archive partitions, since all that data is on the "slow" drives? Can I just reassign all of the current data partitions to archivedata partitions, define the new set of "Hot" data partitions on the Fast pool, and let 'er rip, or what? I promise, if you tell me, I'll write it up as real documentation. :-) We are interested in such a migration too. Our fallback plan, if we don't find a better way to do it is, do use the same method as we introduced the ssd meta-data partition. 1. We created a new partition in our cyrus configuration, 2. we moved moved the accounts from one partition to the other one by one. 3. (this will be new for the archive partition) run cyrus expire to move the old mails back to the slow disks. This method will have two downsides. 1. we have to copy all mails to the fast storage, and move the old mails back to the slow storage. So we have to move most of the mails twice. 2. the path of the old mail will change so they will be stored again in our file based backup so a method without these downsides will be appreciated Regards Michael M.Menge Tel.: (49) 7071/29-70316 Universität Tübingen Fax.: (49) 7071/29-5912 Zentrum für Datenverarbeitung mail: michael.me...@zdv.uni-tuebingen.de Wächterstraße 76 72074 Tübingen Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus M.Menge Tel.: (49) 7071/29-70316 Universität Tübingen Fax.: (49) 7071/29-5912 Zentrum für Datenverarbeitung mail: michael.me...@zdv.uni-tuebingen.de Wächterstraße 76 72074 Tübingen Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: