Re: [lustre-discuss] backup zfs MDT or migrate from ZFS back to ldiskfs

2017-07-22 Thread Stu Midgley
Interesting, so my fears were well founded. Basically, once you choose ZFS, you are stuck with it. The only way to migrate off ZFS is to create a new Lustre file system and copy all the contents out. I'll get to the bottom of the slow send/receive. I am using ashift=12 for both file systems. I

Re: [lustre-discuss] backup zfs MDT or migrate from ZFS back to ldiskfs

2017-07-22 Thread Dilger, Andreas
Using rsync or tar to backup/restore a ZFS MDT is not supported, because this changes the dnode numbering, but ZFS OI Scrub is not yet implemented (there is a Jira ticket for this, and some work is underway there). Options include using zfs send/recv, as you were using, or just incrementally re

Re: [lustre-discuss] backup zfs MDT or migrate from ZFS back to ldiskfs

2017-07-22 Thread Raj
Stu, Is there a reason why you picked Raidz 3 rather than 4 way mirror across 4 disks? Raidz 3 parity calculation might take more cpu resources rather than mirroring across disks but also the latency may be higher in mirroring to sync across all the disks. Wondering if you did some testing before d

Re: [lustre-discuss] backup zfs MDT or migrate from ZFS back to ldiskfs

2017-07-20 Thread Stu Midgley
we have been happily using 2.9.52+0.7.0-rc3 for a while now. The MDT is a raidz3 across 4 disks. On Fri, Jul 21, 2017 at 1:19 PM, Isaac Huang wrote: > On Fri, Jul 21, 2017 at 12:54:15PM +0800, Stu Midgley wrote: > > Afternoon > > > > I have an MDS running on spinning media and wish to migrate i

Re: [lustre-discuss] backup zfs MDT or migrate from ZFS back to ldiskfs

2017-07-20 Thread Isaac Huang
On Fri, Jul 21, 2017 at 12:54:15PM +0800, Stu Midgley wrote: > Afternoon > > I have an MDS running on spinning media and wish to migrate it to SSD's. > > Lustre 2.9.52 > ZFS 0.7.0-rc3 This may not be a stable combination - I don't think Lustre officially supports 0.7.0-rc yet. Plus, ther

[lustre-discuss] backup zfs MDT or migrate from ZFS back to ldiskfs

2017-07-20 Thread Stu Midgley
Afternoon I have an MDS running on spinning media and wish to migrate it to SSD's. Lustre 2.9.52 ZFS 0.7.0-rc3 How do I do it? This is a loaded question :) The MDT is using ~2TB of space. I used the zfs send | zfs receive method to no avail. It was just too slow (I killed it after a