On Fri, 10 Nov 2017, at 00:02, Nic Bernstein wrote:
> Bron,
> Thanks for the response. Your solution has pointed us towards the
> proper approach. We are using ZFS, so would be performing ZFS
> send/receive replication rather than "mv", to move the filesystems.
> Our typical approach for this sort of thing, then, would be:>
> * Create a filesystem snapshot
>* zfs snapshot $ropt ${SOURCE_FS}@$newsnap
> * Perform ZFS send/receive, something like this:
>* zfs send -p $SENDVERBOSE $cloneorigin@$sourcestartsnap | zfs recv
> -u $RECVVERBOSE -F "$destfs"
> * Then, once we've completed a replication, we need to quiesce the
>filesystem and do the same, again, to catch-up to current state
That's basically exactly how FastMail's user moves work, and the sync-
protocol based XFER that Ken built based on it. Except inside Cyrus
rather than at the FS level.
> * Finally, replace the existing filesystem with the new replica, and
>discard the original> To quiesce the filesystem, we would normally like to
> tell whatever
> applications are using it to temporarily freeze operations, so the
> underlying filesystem is in a consistent state. When I was in
> Melbourne, back in April, we (you, Ellie & I) discussed what would be
> needed to introduce such a feature to Cyrus. I'm curious if there's
> been any further discussion or work on this? Should I open a feature
> request?
There has been discussion, not yet work.
https://github.com/cyrusimap/cyrus-imapd/issues/1763
> It would be nice to be able to complete consistent snapshots for
> filesystem operations like replication or backup, and this is a
> feature of many large applications, such as mail and DB servers.
Yep :) It's on our radar. There's a whole lot of work planned - Ken is
going to write up the notes on that at some point. We had a planning
meeting while he was in Australia.
> Thanks again for shining a welcome light on how to achieve the goal
> of adding archive to an existing system.>
> Cheers,
> -nic
>
>
> On 11/04/2017 10:59 PM, Bron Gondwana wrote:
>> Hi Nic,
>>
>> Sorry I didn't get back to answering you on this the other day!
>>
>> So... this one is kinda tricky, because everything is going to be on
>> "spool", but here's how I would do it.>>
>> Before:
>>
>> /mnt/smalldisk/conf -> meta files only
>> /mnt/bigdisk/spool -> all email right now
>>
>> Stage 1: splitting:
>>
>> /mnt/smalldisk/conf
>> /mnt/bigdisk/spool
>> /mnt/bigdisk/spool-archive -> empty
>>
>> And set archivepartition-default to /mnt/bigdisk/spoolarchive
>>
>> Now you need to run an initial cyr_expire. This will take a long
>> time, but it should be able to use hardlinks to move the files - it's
>> using cyrus_copyfile.>>
>> Once cyr_expire has finished an most of your email is moved into spool-
>> archive, shut down cyrus.>>
>> mv /mnt/bigdisk/spool /mnt/smalldisk/spool
>>
>> And set partition-default to /mnt/smalldisk/spool
>>
>> That way your downtime is only while the small remaining spool gets
>> moved to the other disk.>>
>> Bron.
>>
>>
>> On Sun, 5 Nov 2017, at 02:58, Nic Bernstein wrote:
>>> Thanks much to you both for your comments and suggestions. We had>>>
>>> already considered creating a temporary "staging" partition and
>>> shuffling mailboxes around, as Michael discussed, but have the same>>>
>>> reservations about it. Since we're dealing with nearly 6TB of data,>>>
>>> most of it old, this scheme would introduce considerable
>>> disruption to a>>> very active mail system. We have a hard time getting a
>>> two hour
>>> maintenance window, and this would take days!
>>>
>>> Bron, other Fastmailers, any thoughts??
>>> -nic
>>>
>>> On 11/03/2017 11:20 AM, Michael Menge wrote:
Hi,
Quoting Reinaldo Gil Lima de Carvalho :
> I think that singleinstancestore (message hard links) will not
> survive when
> moving from one partition to the other and storage total size will>
> increase
> significantly.
>
thanks for the hint. This was not a problem while migration to the
meta-data partition,
as the mails stayed on the same partition (as in file-system
and not cyrus-partition)
and only hardlinks where change at all.
So one more reason for an other migration path.
>
> 2017-11-03 12:22 GMT-03:00 Michael Menge
> > :
>
>> Hi Nic,
>>
>> Quoting Nic Bernstein :
>>
>> Friends,
>>> I have a client with Cyrus 2.5.10 installed. Last year we
>>> migrated>>> their
>>> old 2.3.18 system to 2.5.10, with an eye towards an eventual
>>> move to>>> 3.0.x. Based on Bron's most excellent email of last
>>> year,
>>> ([Subject: Cyrus
>>> database and file usage data] from Cyrus Devel of 8 January
>>> 2016)>>> we used a
>>> tiered layout for the storage:
>>>
>>> The main categories are:
>>>
>>> * Config directo
