Hi,
Just wondering what the best recipe for migrating a user’s home directory
content from one GFPS file system to another which hosts a larger research GPFS
file system? I’m currently using rsync and it has maxed out the client system’s
IB interface.
Best,
Dwayne
—
Dwayne Hart | Systems Admi
Hi Dwayne,
I’m assuming you can’t just let an rsync run, possibly throttled in some way?
If not, and if you’re just tapping out your network, then would it be possible
to go old school? We have parts of the Medical Center here where their network
connections are … um, less than robust. So th
lf Of Buterbaugh, Kevin L
Sent: Thursday, October 18, 2018 2:26 PM
To: gpfsug main discussion list
Subject: Re: [gpfsug-discuss] Best way to migrate data
Hi Dwayne,
I’m assuming you can’t just let an rsync run, possibly throttled in some way?
If not, and if you’re just tapping out your network,
ion list
Subject: Re: [gpfsug-discuss] Best way to migrate data
Hi Dwayne,
I’m assuming you can’t just let an rsync run, possibly throttled in some
way? If not, and if you’re just tapping out your network, then would it be
possible to go old school? We have parts of the Medical
3 PM
To: gpfsug main discussion list
Subject: Re: [gpfsug-discuss] Best way to migrate data
Other tools and approaches that we've found helpful:
msrsync: handles parallelizing rsync within a dir tree and can greatly speed up
transfers on a single node with both filesystems mounted, especially when
w] [monq 0] [jq 62043]
>
> Best,
> Dwayne
>
> -Original Message-
> From: gpfsug-discuss-boun...@spectrumscale.org [mailto:
> gpfsug-discuss-boun...@spectrumscale.org] On Behalf Of Christopher Black
> Sent: Thursday, October 18, 2018 4:43 PM
> To: gpfsug main discuss
On 18/10/2018 18:19, dwayne.h...@med.mun.ca wrote:
Hi,
Just wondering what the best recipe for migrating a user’s home
directory content from one GFPS file system to another which hosts a
larger research GPFS file system? I’m currently using rsync and it
has maxed out the client system’s IB inte
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
We use parsyncfp. Our target is not GPFS, though. I was really hoping
to hear about something snazzier for GPFS-GPFS. Lenovo would probably
tell you that HSM is the way to go (we asked something similar for a
replacement for our current setup or for di
scuss@spectrumscale.org
Subject: Re: [gpfsug-discuss] Best way to migrate data
On 18/10/2018 18:19, dwayne.h...@med.mun.ca wrote:
> Hi,
>
> Just wondering what the best recipe for migrating a user’s home
> directory content from one GFPS file system to another which hosts a
> large
Thank you Ryan. I’ll have a more in-depth look at this application later today
and see how it deals with some of the large genetic files that are generated by
the sequencer. By copying it from GPFS fs to another GPFS fs.
Best,
Dwayne
—
Dwayne Hart | Systems Administrator IV
CHIA, Faculty of Med
It seems like the primary way that this helps us is that we transfer user home
directories and many of them have VERY large numbers of small files (in the
millions), so running multiple simultaneous rsyncs allows the transfer to
continue past that one slow area. I guess it balances the bandwidth
i am not sure if that was mentioned already but in some version of V5.0.X
based on my suggestion a tool was added by mark on a AS-IS basis (thanks
mark) to do what you want with one exception :
/usr/lpp/mmfs/samples/ilm/mmxcp -h
Usage: /usr/lpp/mmfs/samples/ilm/mmxcp -t target -p strip_count
sourc
Rather than hack rsync or cp ... I proposed a smallish utility that would
copy those extended attributes and ACLs that cp -a just skips over.
This can be done using the documented GPFS APIs that were designed for
backup and restore of files.
SMOP and then add it as an option to samples/ilm/mmxcp
t: Re: [gpfsug-discuss] Best way to migrate data : mmfind ... mmxcp
Rather than hack rsync or cp ... I proposed a smallish utility that would copy
those extended attributes and ACLs that cp -a just skips over.
This can be done using the documented GPFS APIs that were designed for backup
and restore of
additional attributes.
Sven
From: on behalf of Marc A Kaplan
Reply-To: gpfsug main discussion list
Date: Monday, October 22, 2018 at 1:09 PM
To: gpfsug main discussion list
Subject: Re: [gpfsug-discuss] Best way to migrate data : mmfind ... mmxcp
Rather than hack rsync or cp ... I
, you have "one tool".
From: Sven Oehme
To: gpfsug main discussion list
Date: 10/22/2018 04:33 PM
Subject: Re: [gpfsug-discuss] Best way to migrate data : mmfind ...
mmxcp
Sent by:gpfsug-discuss-boun...@spectrumscale.org
Marc,
The issue with that is th
Hi,
I agree, a tool with proper wrapping delivered in samples would be the
right approach.
No warranty, no support - below a prototype I documented 2 years ago (prior
to mmfind availability). The BP used an alternate approach, so its not
tested at scale, but the principle was tested and works.
;gpfsug-discuss@spectrumscale.org"
Date: Tuesday, 23 October 2018 at 06:52
To: "gpfsug-discuss@spectrumscale.org"
Subject: Re: [gpfsug-discuss] Best way to migrate data : Plan B: policy engine
+ rsync
# Use 'GPFS patched' rsync on both ends to keep GPFS attributes
https://github
18 matches
Mail list logo
