On 10/24/2016 08:03 PM, Bill Pappas wrote: > > Loric-> Hi. I was wondering, what version of GPFS where you running > on the home and cache clusters? I take it you broke up the prefetch list > into smaller (for example <2 million) file lists? If not, how? How much > capacity did you migfrate over and how long did this process take? Thanks. > Hello, We use GPFS 4.1.1-8. The migration was within a single cluster. The files lists were split in 2M files parts.
The amount of data migrated is rather small (~70 TB), but there are ~100M files, including many extremely small files: the median (not average) file size is 2705 bytes. The migration itself took about 3 weeks: I initially did the first "mmafmctl prefetches" one fileset at a time (& in slices of 2M files), to manage the various issues mentionned previously and minimize the impact on the "home" filesystem since it was still being used by the end users. Following "prefetches" were done a few filesets at a time (filesets chosen to spread the load on the gateway nodes). We chose to do an "incremental migration" instead of a "progressive migration", since we were unable to get a good understanding of the performance impact of the AFM migration during our tests on a toy cluster running on VMs. Loïc (Loic w/ a diaresis on the i, not Loric :-) -- | Loïc Tortay <tor...@cc.in2p3.fr> - IN2P3 Computing Centre | _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
