[Lustre-discuss] Tar on Lustre

2012-02-13 Thread Ronald K Long
We have a setup that requires a lot of tar being used on lustre. We see a drop in tar performance when going to the lustre setup. I have searched and found the lustre tar 1.19. Unfortunate this did not provide any improvement in the tar times. Are there any suggestions for improving tar wit

Re: [Lustre-discuss] Tar on Lustre

2012-02-13 Thread Peter Grandi
> We have a setup that requires a lot of tar being used on > lustre. That's your choice, however audacious it may be. [ ... ] > tar to extract a 5.2GB file on to the lustre system - 3m22s > tar to extract the same 5.2GB file on to local disk - 1m06s > tat to create a 5.2GB tar file on the lustr

[Lustre-discuss] FW: [Lustre-community] Create directory where files get striped over the same set of OSTs

2012-02-13 Thread Kevin Canady
More appropriate mail list for this question. Cheers, Kevin On 2/9/12 3:24 PM, "Kshitij Mehta" wrote: >Hello, >My lustre installation has a total of 64 OSTs and I have created a >directory that has been configured over 8 OSTs. >However, files in this directory get striped over different OSTs. F

Re: [Lustre-discuss] FW: [Lustre-community] Create directory where files get striped over the same set of OSTs

2012-02-13 Thread Colin Faber
Hi Kshitij, If you're using a reasonably modern Lustre you can use the OST pools feature. This allows you to assign certain OST's to certain files/directories. see 'OST Pools' in the operations manual for more information. -cf On 02/13/2012 09:56 AM, Kevin Canady wrote: > More appropriate ma

Re: [Lustre-discuss] FW: [Lustre-community] Create directory where files get striped over the same set of OSTs

2012-02-13 Thread Kshitij Mehta
Hi Colin, We have Lustre 1.6.7, so OST pools is not available on it. I had the following suggestion from someone which I am going to try out: > In that case, you can set the striping on the directory in question to > start at OST index 0 with a stripe_count of 8 and create your files in > that dir

Re: [Lustre-discuss] IOR writing to a shared file, performance does not scale

2012-02-13 Thread Kshitij Mehta
> Details like OSS configuration, storage and network details would help 38 TByte Lustre filesystem, exported by 11 servers via Infiniband (64 OSTs) > What happens when you write, single client to a single OST, what figures are > you getting? ~ 280 MBps. I see similar results when I use multipl

Re: [Lustre-discuss] "obdidx" ordering in "lfs getstripe"

2012-02-13 Thread Jack David
On Thu, Feb 9, 2012 at 8:18 PM, Andreas Dilger wrote: > On 2012-02-09, at 6:20 AM, Jack David wrote: >> In the output of "lsf getstripe | ", the obdidx >> denotes the OST index (I assume). >> >> Consider the following output: >> >> lmm_stripe_count:   2 >> lmm_stripe_size:    1048576 >> lmm_strip