Re: [lustre-discuss] DNE2 settings are not propagated?

[Mannthey, Keith]

Thanks Andreas and Cory very helpful.   -D will help for what I am using it for.

Thanks!
  Keith

From: Spitz, Cory James 
Sent: Friday, March 20, 2020 12:57 PM
To: Andreas Dilger ; Mannthey, Keith 

Cc: lustre-discuss@lists.lustre.org; Koutoupis, Petros 

Subject: Re: [lustre-discuss] DNE2 settings are not propagated?

As Andreas says, it isn’t really recommended, but you can get sub-directories 
to inherit the striping by using the -D, --default option to `lfs mkdir` or 
`lfs setdirstripe`.   From the man page:

   -D, --default
  Set  the  default striping pattern of subdirectories. Newly cre-
  ated sub-directories will use the new default striping  pattern,
  but  existing  sub-directories  will not be affected.  The newly
  created sub-directories will also inherit the specified  default
  striping  pattern.  Only  default  stripe count is supported for
  now.

  Note that striping all directories across all MDTs by default is
  not  recommended  at  this  time, as the clients will have to do
  more RPCs to create and access each directory,  hurting  perfor-
  mance rather than improving it.  Default striped directories are
  preferred for cases where large subdirectories will  be  created
  (e.g. file-per-process job output directories).

-Cory


On 3/20/20, 2:33 PM, "lustre-discuss on behalf of Andreas Dilger" 
mailto:lustre-discuss-boun...@lists.lustre.org>
 on behalf of adil...@whamcloud.com<mailto:adil...@whamcloud.com>> wrote:

On Mar 20, 2020, at 13:02, Mannthey, Keith 
mailto:keith.mannt...@intel.com>> wrote:

I am doing some work with Lustre 2.13 (Downloaded from Whamcloud) and DNE 2 
there are 6 MDTs.


"
[root@colts52-clx01 for-test]# lfs getdirstripe TEST
lmv_stripe_count: 6 lmv_stripe_offset: 1 lmv_hash_type: fnv_1a_64
mdtidx   FID[seq:oid:ver]
1   [0x2c400:0x5:0x0]
5   [0x30403:0x5:0x0]
3   [0x34403:0x5:0x0]
0   [0x20403:0x5:0x0]
4   [0x24402:0x5:0x0]
2   [0x28401:0x5:0x0]
[root@colts52-clx01 for-test]# mkdir TEST/now
[root@colts52-clx01 for-test]# lfs getdirstripe TEST/now/
lmv_stripe_count: 0 lmv_stripe_offset: 2 lmv_hash_type: none
"

Directory stripping settings are not being propagated.  Is this expected?

Yes, this working as designed.  It isn't necessarily good for all directories 
to be striped, as that adds overhead without necessary improving performance.

The current recommendation is that DNE should be used for e.g. a top-level 
directory to distribute files and subdirectories across MDTs, or in the case of 
large directories with millions of files.

Cheers, Andreas
--
Andreas Dilger
Principal Lustre Architect
Whamcloud





___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] DNE2 settings are not propagated?

[Spitz, Cory James]

As Andreas says, it isn’t really recommended, but you can get sub-directories 
to inherit the striping by using the -D, --default option to `lfs mkdir` or 
`lfs setdirstripe`.   From the man page:

   -D, --default
  Set  the  default striping pattern of subdirectories. Newly cre-
  ated sub-directories will use the new default striping  pattern,
  but  existing  sub-directories  will not be affected.  The newly
  created sub-directories will also inherit the specified  default
  striping  pattern.  Only  default  stripe count is supported for
  now.

  Note that striping all directories across all MDTs by default is
  not  recommended  at  this  time, as the clients will have to do
  more RPCs to create and access each directory,  hurting  perfor-
  mance rather than improving it.  Default striped directories are
  preferred for cases where large subdirectories will  be  created
  (e.g. file-per-process job output directories).

-Cory


On 3/20/20, 2:33 PM, "lustre-discuss on behalf of Andreas Dilger" 
mailto:lustre-discuss-boun...@lists.lustre.org>
 on behalf of adil...@whamcloud.com> wrote:

On Mar 20, 2020, at 13:02, Mannthey, Keith 
mailto:keith.mannt...@intel.com>> wrote:

I am doing some work with Lustre 2.13 (Downloaded from Whamcloud) and DNE 2 
there are 6 MDTs.


"
[root@colts52-clx01 for-test]# lfs getdirstripe TEST
lmv_stripe_count: 6 lmv_stripe_offset: 1 lmv_hash_type: fnv_1a_64
mdtidx   FID[seq:oid:ver]
1   [0x2c400:0x5:0x0]
5   [0x30403:0x5:0x0]
3   [0x34403:0x5:0x0]
0   [0x20403:0x5:0x0]
4   [0x24402:0x5:0x0]
2   [0x28401:0x5:0x0]
[root@colts52-clx01 for-test]# mkdir TEST/now
[root@colts52-clx01 for-test]# lfs getdirstripe TEST/now/
lmv_stripe_count: 0 lmv_stripe_offset: 2 lmv_hash_type: none
"

Directory stripping settings are not being propagated.  Is this expected?

Yes, this working as designed.  It isn't necessarily good for all directories 
to be striped, as that adds overhead without necessary improving performance.

The current recommendation is that DNE should be used for e.g. a top-level 
directory to distribute files and subdirectories across MDTs, or in the case of 
large directories with millions of files.

Cheers, Andreas
--
Andreas Dilger
Principal Lustre Architect
Whamcloud






___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] DNE2 settings are not propagated?

[Andreas Dilger]

On Mar 20, 2020, at 13:02, Mannthey, Keith 
mailto:keith.mannt...@intel.com>> wrote:

I am doing some work with Lustre 2.13 (Downloaded from Whamcloud) and DNE 2 
there are 6 MDTs.


"
[root@colts52-clx01 for-test]# lfs getdirstripe TEST
lmv_stripe_count: 6 lmv_stripe_offset: 1 lmv_hash_type: fnv_1a_64
mdtidx   FID[seq:oid:ver]
1   [0x2c400:0x5:0x0]
5   [0x30403:0x5:0x0]
3   [0x34403:0x5:0x0]
0   [0x20403:0x5:0x0]
4   [0x24402:0x5:0x0]
2   [0x28401:0x5:0x0]
[root@colts52-clx01 for-test]# mkdir TEST/now
[root@colts52-clx01 for-test]# lfs getdirstripe TEST/now/
lmv_stripe_count: 0 lmv_stripe_offset: 2 lmv_hash_type: none
"

Directory stripping settings are not being propagated.  Is this expected?

Yes, this working as designed.  It isn't necessarily good for all directories 
to be striped, as that adds overhead without necessary improving performance.

The current recommendation is that DNE should be used for e.g. a top-level 
directory to distribute files and subdirectories across MDTs, or in the case of 
large directories with millions of files.

Cheers, Andreas
--
Andreas Dilger
Principal Lustre Architect
Whamcloud






___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

[lustre-discuss] DNE2 settings are not propagated?

[Mannthey, Keith]

I am doing some work with Lustre 2.13 (Downloaded from Whamcloud) and DNE 2 
there are 6 MDTs. 


"
[root@colts52-clx01 for-test]# lfs getdirstripe TEST
lmv_stripe_count: 6 lmv_stripe_offset: 1 lmv_hash_type: fnv_1a_64
mdtidx   FID[seq:oid:ver]
 1   [0x2c400:0x5:0x0]
 5   [0x30403:0x5:0x0]
 3   [0x34403:0x5:0x0]
 0   [0x20403:0x5:0x0]
 4   [0x24402:0x5:0x0]
 2   [0x28401:0x5:0x0]
[root@colts52-clx01 for-test]# mkdir TEST/now
[root@colts52-clx01 for-test]# lfs getdirstripe TEST/now/
lmv_stripe_count: 0 lmv_stripe_offset: 2 lmv_hash_type: none
 "

Directory stripping settings are not being propagated.  Is this expected?  


Thanks,
 Keith 
 
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org