Re: [Lustre-discuss] OST - low MB/s

2009-09-10 Thread Dr. Hung-Sheng Tsao (LaoTsao) 
not sure I understand Ur setup
which one is the raid6 lun?
which are the individual HD?


Rafael David Tinoco wrote:
> 216MB/s using 8*128 (1024k) as bs. Too low for 8 active disks.. right ? 
> Arround 27MB/s.. from 50MB/s in the "real" disk.
>
> -Original Message-
> From: lustre-discuss-boun...@lists.lustre.org 
> [mailto:lustre-discuss-boun...@lists.lustre.org] On Behalf Of Dr. Hung-Sheng 
> Tsao
> (LaoTsao)
> Sent: Thursday, September 10, 2009 5:50 PM
> To: Rafael David Tinoco
> Cc: lustre-discuss@lists.lustre.org
> Subject: Re: [Lustre-discuss] OST - low MB/s
>
> raid 6 chunk size=128k the full strip size will be 128k*8 (for 10 disks 
> 8+2 raid 6)
> in Ur dd test one should use bs=128k*8, then each 8 HDD will be busy
> regards
>
>
> Rafael David Tinoco wrote:
>   
>> With this RAID5 configuration Im getting:
>>
>> r...@a02n00:~# dd if=/dev/zero of=/dev/md20 bs=128k count=1
>>
>> 1+0 records in
>>
>> 1+0 records out
>>
>> 131072 bytes (1.3 GB) copied, 5.20774 seconds, 252 MB/s
>>
>> r...@a02n00:~# dd if=/dev/zero of=/dev/md20 bs=128k count=1
>>
>> 1+0 records in
>>
>> 1+0 records out
>>
>> 131072 bytes (1.3 GB) copied, 5.12 seconds, 256 MB/s
>>
>> So, 80MB/s using these md20 as OSTs isnt quite right .
>>
>> *From:* lustre-discuss-boun...@lists.lustre.org 
>> [mailto:lustre-discuss-boun...@lists.lustre.org] *On Behalf Of *Rafael 
>> David Tinoco
>> *Sent:* Thursday, September 10, 2009 4:26 PM
>> *To:* lustre-discuss@lists.lustre.org
>> *Subject:* [Lustre-discuss] OST - low MB/s
>>
>> Hello,
>>
>> I'm having problems now with my "OSTs" throughput.
>>
>> I have 4 OSS each one with 2 OSTs. These OSTs are RAID6 with 10 disks, 
>> chunk size of 128k.
>>
>> These disks are from J4400 (JBOD) connected in multipath using multipathd.
>>
>> Each disk speed is giving me 50MB/s with dd.
>>
>> With lustre, using IOR or DD I can get only arround 80MB/s. I was 
>> expecting for 8 active disks in raid 8*50 = something between 300 and 
>> 400MB/s.
>>
>> avg-cpu: %user %nice %system %iowait %steal %idle
>>
>> 0.00 0.00 6.00 9.06 0.00 84.94
>>
>> Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await 
>> svctm %util
>>
>> md10 0.00 0.00 0.00 398.00 0.00 1.55 8.00 0.00 0.00 0.00 0.00
>>
>> md11 0.00 0.00 0.00 380.00 0.00 1.48 8.00 0.00 0.00 0.00 0.00
>>
>> md20 0.00 0.00 0.00 158.00 0.00 79.00 1024.00 0.00 0.00 0.00 0.00
>>
>> md21 0.00 0.00 0.00 159.00 0.00 79.50 1024.00 0.00 0.00 0.00 0.00
>>
>> avg-cpu: %user %nice %system %iowait %steal %idle
>>
>> 0.00 0.00 5.94 9.32 0.00 84.74
>>
>> Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await 
>> svctm %util
>>
>> md10 0.00 0.00 0.00 407.50 0.00 1.59 8.00 0.00 0.00 0.00 0.00
>>
>> md11 0.00 0.00 0.00 394.00 0.00 1.54 8.00 0.00 0.00 0.00 0.00
>>
>> md20 0.00 0.00 0.00 159.00 0.00 79.50 1024.00 0.00 0.00 0.00 0.00
>>
>> md21 0.00 0.00 0.00 158.00 0.00 79.00 1024.00 0.00 0.00 0.00 0.00
>>
>> avg-cpu: %user %nice %system %iowait %steal %idle
>>
>> 0.00 0.00 6.37 9.43 0.00 84.21
>>
>> Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await 
>> svctm %util
>>
>> md10 0.00 0.00 0.00 410.50 0.00 1.60 8.00 0.00 0.00 0.00 0.00
>>
>> md11 0.00 0.00 0.00 376.00 0.00 1.47 8.00 0.00 0.00 0.00 0.00
>>
>> md20 0.00 0.00 0.00 165.00 0.00 82.50 1024.00 0.00 0.00 0.00 0.00
>>
>> md21 0.00 0.00 0.00 165.00 0.00 82.50 1024.00 0.00 0.00 0.00 0.00
>>
>> Any clues ?
>>
>> Rafael David Tinoco - Sun Microsystems
>>
>> Systems Engineer - High Performance Computing
>>
>> rafael.tin...@sun.com - 55.11.5187.2194
>>
>> 
>>
>> ___
>> Lustre-discuss mailing list
>> Lustre-discuss@lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>>   
>> 
> ___
> Lustre-discuss mailing list
> Lustre-discuss@lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>
>   
___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

Re: [Lustre-discuss] OST - low MB/s

2009-09-10 Thread Dr. Hung-Sheng Tsao (LaoTsao) 
raid 6 chunk size=128k the full strip size will be 128k*8 (for 10 disks 
8+2 raid 6)
in Ur dd test one should use bs=128k*8, then each 8 HDD will be busy
regards


Rafael David Tinoco wrote:
>
> With this RAID5 configuration Im getting:
>
> r...@a02n00:~# dd if=/dev/zero of=/dev/md20 bs=128k count=1
>
> 1+0 records in
>
> 1+0 records out
>
> 131072 bytes (1.3 GB) copied, 5.20774 seconds, 252 MB/s
>
> r...@a02n00:~# dd if=/dev/zero of=/dev/md20 bs=128k count=1
>
> 1+0 records in
>
> 1+0 records out
>
> 131072 bytes (1.3 GB) copied, 5.12 seconds, 256 MB/s
>
> So, 80MB/s using these md20 as OSTs isnt quite right …
>
> *From:* lustre-discuss-boun...@lists.lustre.org 
> [mailto:lustre-discuss-boun...@lists.lustre.org] *On Behalf Of *Rafael 
> David Tinoco
> *Sent:* Thursday, September 10, 2009 4:26 PM
> *To:* lustre-discuss@lists.lustre.org
> *Subject:* [Lustre-discuss] OST - low MB/s
>
> Hello,
>
> I’m having problems now with my “OSTs” throughput.
>
> I have 4 OSS each one with 2 OSTs. These OSTs are RAID6 with 10 disks, 
> chunk size of 128k.
>
> These disks are from J4400 (JBOD) connected in multipath using multipathd.
>
> Each disk speed is giving me 50MB/s with dd.
>
> With lustre, using IOR or DD I can get only arround 80MB/s. I was 
> expecting for 8 active disks in raid 8*50 = something between 300 and 
> 400MB/s.
>
> avg-cpu: %user %nice %system %iowait %steal %idle
>
> 0.00 0.00 6.00 9.06 0.00 84.94
>
> Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await 
> svctm %util
>
> md10 0.00 0.00 0.00 398.00 0.00 1.55 8.00 0.00 0.00 0.00 0.00
>
> md11 0.00 0.00 0.00 380.00 0.00 1.48 8.00 0.00 0.00 0.00 0.00
>
> md20 0.00 0.00 0.00 158.00 0.00 79.00 1024.00 0.00 0.00 0.00 0.00
>
> md21 0.00 0.00 0.00 159.00 0.00 79.50 1024.00 0.00 0.00 0.00 0.00
>
> avg-cpu: %user %nice %system %iowait %steal %idle
>
> 0.00 0.00 5.94 9.32 0.00 84.74
>
> Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await 
> svctm %util
>
> md10 0.00 0.00 0.00 407.50 0.00 1.59 8.00 0.00 0.00 0.00 0.00
>
> md11 0.00 0.00 0.00 394.00 0.00 1.54 8.00 0.00 0.00 0.00 0.00
>
> md20 0.00 0.00 0.00 159.00 0.00 79.50 1024.00 0.00 0.00 0.00 0.00
>
> md21 0.00 0.00 0.00 158.00 0.00 79.00 1024.00 0.00 0.00 0.00 0.00
>
> avg-cpu: %user %nice %system %iowait %steal %idle
>
> 0.00 0.00 6.37 9.43 0.00 84.21
>
> Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await 
> svctm %util
>
> md10 0.00 0.00 0.00 410.50 0.00 1.60 8.00 0.00 0.00 0.00 0.00
>
> md11 0.00 0.00 0.00 376.00 0.00 1.47 8.00 0.00 0.00 0.00 0.00
>
> md20 0.00 0.00 0.00 165.00 0.00 82.50 1024.00 0.00 0.00 0.00 0.00
>
> md21 0.00 0.00 0.00 165.00 0.00 82.50 1024.00 0.00 0.00 0.00 0.00
>
> Any clues ?
>
> Rafael David Tinoco - Sun Microsystems
>
> Systems Engineer - High Performance Computing
>
> rafael.tin...@sun.com - 55.11.5187.2194
>
> 
>
> ___
> Lustre-discuss mailing list
> Lustre-discuss@lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>   
___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

[Lustre-discuss] modify client kernel support

2009-08-30 Thread Dr. Hung-Sheng Tsao (LaoTsao) 

hi
how do lustre handle some clients need to modify the kernel due to some 
local requirement?

TIA

<>___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

Re: [Lustre-discuss] Lustre playground in VirtualBox?

2009-08-10 Thread Dr. Hung-Sheng Tsao (LaoTsao) 


IMHO, one just need 64-bit support

Wolfgang Stief wrote:

Hi out there!

Before I start installing and fiddling around: Are there any reasons
AGAINST setting up a Lustre playground in a VirtualBox environment? I
just want to play around w/ recovery and debugging situations and
upgrades. No performance required at all.

Anybody already set up a virtual environment for such a purpose? Any
"best practice" or something?

Thanks for answers!
wolfgang
___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss
  
begin:vcard
fn:hung-seng tsao
n:tsao;hung-seng
org:;EH/East
adr:;;400 Atrium dr;sommerset;nj;08873;usa
email;internet:hung-sheng.t...@sun.com
title:SE
tel;work:1877 319 0460 (x67079)
tel;fax:1877 319 0460
tel;cell:19734950840
note;quoted-printable:ts amb=0D=0A=
	grid-aces=0D=0A=
	cluster-aces=0D=0A=
	blade-aces=0D=0A=
	hpc-aces
url:http://blogs.sun.com/hstsao/
version:2.1
end:vcard

___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

Re: [Lustre-discuss] OST redundancy between nodes?

2009-06-19 Thread Dr. Hung-Sheng Tsao (LaoTsao) 
it seems that one can use dual connected array to two nodes and use 
software mirroring between the two  array to give U double protection

1)HW-raid within array
2)SW raid between arrays
hth

Kevin Van Maren wrote:

Gary Gogick wrote:
  

Heya all,

I'm investigating potential solutions for a storage deployment.  
Lustre piqued my interest due to ease of scalability and awesome 
aggregate throughput potential. 

Wondering if there's any provision in Lustre for handling catastrophic 
loss of a node containing an OST; eg. replication/mirroring of OSTs to 
other nodes?


I'm gathering from the 1.8.0 documentation that there's no protection 
of this sort for data other than underlying RAID configs on any 
individual node, at least not without attempting to do some 
interesting stuff with DRDB.  Just started looking at Lustre over the 
past day though, so I'd totally appreciate an authoritative answer in 
case I'm misinterpreting the documentation. :)



Correct.

Lustre failover can be used to support catastrophic failure of a _node_, 
but not the _storage_.  If your configuration makes LUNs available to 
two nodes, it is possible to configure Lustre to operate across the 
failure of a server.


If your LUN fails catastrophically, all the data on that lun is gone.  
It is possible to bring Lustre up without it, but none of the files on 
that OST would be available.  If you are concerned about this case, then 
backups are your friend.


While drdb could be used to make a lun "available" to two nodes, it will 
have a significant impact on performance, and (AFAIK) does not do 
synchronous replication, so an fsck would be required prior to mounting 
the OST on the second node, and there would be some data loss.


Kevin

___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss
  
begin:vcard
fn:hung-seng tsao
n:tsao;hung-seng
org:;EH/East
adr:;;400 Atrium dr;sommerset;nj;08873;usa
email;internet:hung-sheng.t...@sun.com
title:SE
tel;work:1877 319 0460 (x67079)
tel;fax:1877 319 0460
tel;cell:19734950840
note;quoted-printable:ts amb=0D=0A=
	grid-aces=0D=0A=
	cluster-aces=0D=0A=
	blade-aces=0D=0A=
	hpc-aces
url:http://blogs.sun.com/hstsao/
version:2.1
end:vcard

___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

Re: [Lustre-discuss] MDT backup procedure

2009-06-17 Thread Dr. Hung-Sheng Tsao (LaoTsao) 

IMHO, may be one can do a HA-MDS using shared storage
all the datas are in the shared storage so U can do a failover


Ramiro Alba Queipo wrote:

HiDaniel,

By reading Chapter 15 of Lustre Operations Manual, it follows that an
MDT backup is only useful if you are changing hardwary or the like.
I am afraid that you can not pretend to replace with a previous image an
failed MDT, as data in OSTs and MDT is not matching any more, right?

Cheers

On Wed, 2009-06-17 at 09:41 -0600, Daniel Kulinski wrote:
  

As we move forward with our lustre testing I am wondering about MDT
backup.  

 


Is it feasible to unmount the MDT, create an image of it and remount
it after the backup.  Of course this wouldn’t happen but nightly.

 


From what I can identify, in the case of an MDT failure we would have
to do the following:

 


Restore from the last backup.

Run an lfsck across the filesystem.

 


Am I missing anything else at this point?  We will also be doing file
level backups of the filesystem as a whole but we are looking for
quick ways to recover from an MDT failure.

 


Thanks,

  Dan Kulinski



--
Aquest missatge ha estat analitzat per MailScanner 
a la cerca de virus i d'altres continguts perillosos, 
i es considera que está net. 
MailScanner agraeix a transtec Computers pel seu suport. 
___

Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss




___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

begin:vcard
fn:hung-seng tsao
n:tsao;hung-seng
org:;EH/East
adr:;;400 Atrium dr;sommerset;nj;08873;usa
email;internet:hung-sheng.t...@sun.com
title:SE
tel;work:1877 319 0460
tel;fax:1877 319 0460
tel;cell:19734950840
note;quoted-printable:ts amb=0D=0A=
	grid-aces=0D=0A=
	cluster-aces=0D=0A=
	blade-aces=0D=0A=
	hpc-aces
url:http://blogs.sun.com/hstsao/
version:2.1
end:vcard

___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

Re: [Lustre-discuss] lustre and OS file system

2009-06-17 Thread Dr. Hung-Sheng Tsao (LaoTsao) 



Carlos Santana wrote:
I am a newbie to the lustre world. I have a very basic and probably 
stupid question here. When we install a Lustre FS, i.e., install RPM 
packages for the server side then what happens to the operating 
system's file system? 

os fs still support os
lustre FS will be on separate disks

When and how does lustre come into the picture? Comments appreciated.

-
Neil.


___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss
  
begin:vcard
fn:hung-seng tsao
n:tsao;hung-seng
org:;EH/East
adr:;;400 Atrium dr;sommerset;nj;08873;usa
email;internet:hung-sheng.t...@sun.com
title:SE
tel;work:1877 319 0460
tel;fax:1877 319 0460
tel;cell:19734950840
note;quoted-printable:ts amb=0D=0A=
	grid-aces=0D=0A=
	cluster-aces=0D=0A=
	blade-aces=0D=0A=
	hpc-aces
url:http://blogs.sun.com/hstsao/
version:2.1
end:vcard

___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

Re: [Lustre-discuss] Lustre installation and configuration problems

2009-06-17 Thread Dr. Hung-Sheng Tsao (LaoTsao) 

http://blogs.sun.com/manoj/entry/lustre_demo_flash
http://blogs.sun.com/manoj/entry/lustre_installation_multi_node

Carlos Santana wrote:
And is there any specific installation order for patchless client? 
Could someone please share it with me?


-
CS.

On Wed, Jun 17, 2009 at 10:18 AM, Carlos Santana > wrote:


Huh... :( Sorry to bug you guys again...

I am planning to make a fresh start now as nothing seems to have
worked for me. If you have any comments/feedback please share them.

I would like to confirm installation order before I make a fresh
start. From Arden's experience:
http://lists.lustre.org/pipermail/lustre-discuss/2009-June/010710.html
, the lusre-module is installed last. As I was installing Lustre
1.8, I was referring 1.8 operations manual
http://manual.lustre.org/index.php?title=Main_Page . The
installation order in the manual is different than what Arden has
suggested.

Will it make a difference in configuration at later stage? Which
one should I follow now?
Any comments?

Thanks,
CS.


On Wed, Jun 17, 2009 at 12:35 AM, Carlos Santana mailto:neu...@gmail.com>> wrote:

Thanks Cliff.

The depmod -a was successful before as well. I am using CentOS 5.2
box. Following are the packages installed:
[r...@localhost tmp]# rpm -qa | grep -i lustre
lustre-modules-1.8.0-2.6.18_92.1.17.el5_lustre.1.8.0smp
lustre-1.8.0-2.6.18_92.1.17.el5_lustre.1.8.0smp

[r...@localhost tmp]# uname -a
Linux localhost.localdomain 2.6.18-92.el5 #1 SMP Tue Jun 10
18:49:47
EDT 2008 i686 i686 i386 GNU/Linux

And here is a output from strace for mount:
http://www.heypasteit.com/clip/8WT

Any further debugging hints?

Thanks,
CS.

On 6/16/09, Cliff White mailto:cliff.wh...@sun.com>> wrote:
> Carlos Santana wrote:
>> The '$ modprobe -l lustre*' did not show any module on a
patchless
>> client. modprobe -v returns 'FATAL: Module lustre not found'.
>>
>> How do I install a patchless client?
>> I have tried lustre-client-modules and lustre-client-ver
rpm packages in
>> both sequences. Am I missing anything?
>>
>
> Make sure the lustre-client-modules package matches your
running kernel.
> Run depmod -a to be sure
> cliffw
>
>> Thanks,
>> CS.
>>
>>
>>
>> On Tue, Jun 16, 2009 at 2:28 PM, Cliff White
mailto:cliff.wh...@sun.com>
>> >>
wrote:
>>
>> Carlos Santana wrote:
>>
>> The lctlt ping and 'net up' failed with the
following messages:
>> --- ---
>> [r...@localhost ~]# lctl ping 10.0.0.42
>> opening /dev/lnet failed: No such device
>> hint: the kernel modules may not be loaded
>> failed to ping 10.0.0...@tcp: No such device
>>
>> [r...@localhost ~]# lctl network up
>> opening /dev/lnet failed: No such device
>> hint: the kernel modules may not be loaded
>> LNET configure error 19: No such device
>>
>>
>> Make sure modules are unloaded, then try modprobe -v.
>> Looks like you have lnet mis-configured, if your module
options are
>> wrong, you will see an error during the modprobe.
>> cliffw
>>
>> --- ---
>>
>>
>> I tried lustre_rmmod and depmod commands and it did
not return
>> any error messages. Any further clues? Reinstall
patchless
>> client again?
>>
>> -
>> CS.
>>
>>
>> On Tue, Jun 16, 2009 at 1:32 PM, Cliff White
>> mailto:cliff.wh...@sun.com>
>
>>  >
>>Carlos Santana wrote:
>>
>>I was able to run lustre_rmmod and depmod
successfully. The
>>'$lctl list_nids' returned the server ip
address and
>> interface
>>(tcp0).
>>
>>I tried to mount the file system on a remote
client, but it
>>failed with the following message.
>>--- ---
>>[r...@localhost ~]# mount -t lustre
10.0.0...@tcp0:/lustre
>>/mnt/lustre