Re: [Lustre-discuss] CVS checkout of 1.8.1 fails

2009-07-01 Thread Johann Lombardi 
On Jun 29, 2009, at 7:27 PM, Robert LeBlanc wrote:
> We issued
> ./lustrecvs b1_8_1

The branch name is b_release_1_8_1.

Johann
___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

Re: [Lustre-discuss] quota problems in lustre 1.8

2009-07-01 Thread Giacinto Donvito 
I tried also using Lustre
1.6.7.2 with the official kernels and it works always in the same way:
the first joined server honours
the quota denying writing when quota is reached while the second server does
allow writing from the same user and from the same client.
I also tried to
use different kernel at the client side obtaining always the same behaviour.

Do anyone have any suggestion or hints?

In our installation the quota is absolutely needed.

Thank you in advance for any help you may provide.

Cheers,
Giacinto

~~
Giacinto DonvitoLIBI -- EGEE2 SA1 INFN - Bari ITALY
--
giacinto.donv...@ba.infn.it   | GTalk/GMail:
donvito.giaci...@gmail.com
tel. +39 080 5443244   Fax  +39 0805442470  VOIP:  +41225481596   | MSN:
donvito.giaci...@hotmail.it
Skype: giacinto_it | AIM/iChat: gdonvito1 | Yahoo: eric1_it
--
"A simple design always takes less time to finish than a complex one.
So always do the simplest thing that could possibly work."
Don we...@www.extremeprogramming.org

"To see what is in front of one's nose needs a constant struggle." - George
Orwell 

On Fri, Jun 26, 2009 at 17:59, Giacinto Donvito  wrote:

> Hi Nirmal,
>
> I've tried with both: " 2.6.18-128.1.6.el5_lustre.1.8.0.1smp x86_64" and
> "2.6.22.14 x86_64" clients.
>
> The results is the same.
>
> Cheers,
> Giacinto
>
> --
> --
> ~~
> Giacinto DonvitoLIBI -- EGEE3 SA1 INFN - Bari ITALY
> --
> giacinto.donv...@ba.infn.it   | GTalk/GMail:
> donvito.giaci...@gmail.com
>  tel. +39 080 5443244   Fax  +39 0805442470| Skype: giacinto_it
> VOIP:  +41225481596   | MSN: donvito.giaci...@hotmail.it
> AIM/iChat: gdonvito1  | Yahoo: eric1_it
> --
> “The Linux philosophy is 'Laugh in the face of danger'. Oops. Wrong One.
> 'Do it yourself'. Yes, that's it.”
> Linus Torvalds
>
>
>
>
>
> Il giorno 26/giu/09, alle ore 17:41, Nirmal Seenu ha scritto:
>
>
>  I have the quota working on one of our cluster while I am trying to
>> figure out what is wrong with the second cluster.
>>
>> The cluster where I have quota working, the Lustre servers are running
>> 2.6.18-128.1.6.el5_lustre.1.8.0.1smp kernel and the Lustre clients run
>> on RHEL5.3 distro with the 2.6.18-128.1.10.el5 kernel + Lustre patchless
>> client version 1.8.0.1.
>>
>> The other cluster that I have the same problem that you are facing, the
>> lustre servers are running 2.6.18-92.1.17.el5_lustre.1.6.7.1smp and the
>> clients run on RHEL4.4 with 2.6.21 kernel.org kernel + Lustre patchless
>> clients version 1.6.7.
>>
>> The quota problem seems to be related to the kernel on the Client side.
>>  I guess that there is no 64-bit quota support in the 2.6.21 kernel?
>>
>> What configuration do you have on the client side? I would be curious to
>> see if the quota works fine with the 2.6.18-128.1.10.el5 kernel on your
>> clients.
>>
>> Nirmal
>> ___
>> Lustre-discuss mailing list
>> Lustre-discuss@lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>>
>
>
___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

Re: [Lustre-discuss] CVS checkout of 1.8.1 fails

2009-07-01 Thread Johann Lombardi 
On Jun 30, 2009, at 3:09 PM, Peter Jones wrote:
> Work to support 2.6.30 is taking place under bug 19808. It does not  
> look
> like this will be ready in time for 1.8.1 but should make 1.8.2

Actually, 1.8.1 should support 2.6.27 patchless client.
As for server support,  2.6.30 won't indeed be available before 1.8.2,
although 1.8.1 introduces sles11 server support (and drops 2.6.22).

Johann
___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

Re: [Lustre-discuss] quota problems in lustre 1.8

2009-07-01 Thread Zhiyong Landen tian 

> Hi All,
>
> I'm experiencing some problem in a test installation of lustre 1.8.0.X
>
> The installation is composed by one server hosting the MDS, and two 
> servers hosting the OSTs.
> One of the servers has 12x2.7TB devices and the other has 16x2.3TB 
> devices.
>
> All the devices were configured with:
>
> "tunefs.lustre --ost --mgsnode=lustr...@tcp0 --param ost.quota_type=ug 
> --writeconf /dev/sdxx"
>
> on the admin node I issued the "lfs quotacheck -ug /lustre" (I see 
> read operation occurring on the both disk servers) that ends without 
> error.
>
> I was able to set-up quotas per user on the admin node and it seems 
> successfully registered by checking with: "lfs quota -u donvito /lustre"
>
> The problem that I see is that it is possible for a user to overfill 
> the quota as the two server behave differently: one of the two deny 
> writing while the other not.
> I tried with both lustre rpms and vanilla (2.6.22) patched kernel and 
> the result is the same. It is not related to the physical server as 
> both of them sometimes has the same behaviour (But only one of the 
> server at the time). I have tried with both 1.8.0 and 1.8.0.1 and the 
> same behaviour is observed.
>
> As you can see the system is correctly accounting the used space but 
> the server do not deny writing:
>
> [r...@lustre01 ~]# lfs quota -u donvito /lustre
> Disk quotas for user donvito (uid 501):
>  Filesystem  kbytes   quota   limit   grace   files   quota   
> limit   grace
> /lustre 124927244* 1200 1500  24 
> 100 100   

(hint)You can use "lfs quota -v ..." to get how much quota grant every 
quota slave has.

>
> The "messages" log on the MDS and both the OSS servers are clean and 
> nothing strange is noticeable.
> Do you have any ideas of where I can look in order to understand the 
> problem?
It is expected by the current lquota design of lustre.  For lustre 
quota, there are two kinds of roles: quota master(mds) and quota 
slaves(osts). When you set quota, the limitation is recorded on quota 
master. When data is written on osts,  osts will get some quota grant 
from quota master if remained quota on osts isn't enough. But, at the 
same time, osts will get some kinds of "quota grant cache" so that quota 
slaves won't ask quota from quota master every time when they write(if 
so, performance will be hurted). Then every quota slave will judge if 
the request it received will trigger out of quota based upon the grant 
quota it got from mds _respectively_. Then you get what you saw.
> Do you have any other suggestion of tests that I can do? 
For causes of performance, lquota of lustre is distributed and isn't 
exactly like local quota(e.g. ext3). So what you saw is normal to 
lquota, currently you can only change you application to adapt it if it 
hurt you
>
> Thank you very much.
>
> Best Regards,
> Giacinto
>
> ~~
> Giacinto DonvitoLIBI -- EGEE2 SA1 INFN - Bari ITALY
> --
> giacinto.donv...@ba.infn.it    
> | GTalk/GMail: donvito.giaci...@gmail.com 
> 
> tel. +39 080 5443244   Fax  +39 0805442470  VOIP:  +41225481596   | 
> MSN: donvito.giaci...@hotmail.it 
> Skype: giacinto_it | AIM/iChat: gdonvito1 | Yahoo: eric1_it
> --
> "A simple design always takes less time to finish than a complex one.
> So always do the simplest thing that could possibly work."
> Don we...@www.extremeprogramming.org 
> 
>
> "Writing about music is like dancing about architecture." - Frank 
> Zappa 
> 
>
> ___
> Lustre-discuss mailing list
> Lustre-discuss@lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>   

___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

Re: [Lustre-discuss] CVS checkout of 1.8.1 fails

2009-07-01 Thread Ralf Utermann 
Johann Lombardi wrote:
> On Jun 30, 2009, at 3:09 PM, Peter Jones wrote:
>> Work to support 2.6.30 is taking place under bug 19808. It does not  
>> look
>> like this will be ready in time for 1.8.1 but should make 1.8.2
> 
> Actually, 1.8.1 should support 2.6.27 patchless client.
> As for server support,  2.6.30 won't indeed be available before 1.8.2,
> although 1.8.1 introduces sles11 server support (and drops 2.6.22).

thanks for this info. Our question would be, whether 1.8.1 supports a
2.6.27 patched kernel. We did tests with patchless clients and were
not very happy with the performance. And we need it for the
serverside anyway.


Bye, Ralf
-- 
Ralf Utermann
_
Universität Augsburg, Institut für Physik   --   EDV-Betreuer
Universitätsstr.1 
D-86135 Augsburg Phone:  +49-821-598-3231
SMTP: ralf.uterm...@physik.uni-augsburg.de Fax: -3411
___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

Re: [Lustre-discuss] CVS checkout of 1.8.1 fails

2009-07-01 Thread Johann Lombardi 
On Jul 1, 2009, at 12:37 PM, Ralf Utermann wrote:
> thanks for this info. Our question would be, whether 1.8.1 supports a
> 2.6.27 patched kernel.

There is no patch series (for both kernel & ldiskfs) available for  
2.6.27
and we will likely not provide one before we support 2.6.30, since we
need a stable ext4 tree for ldiskfs. ext4 in SLES11 is quite up to
date, that's why we can afford to support servers on this kernel.

> We did tests with patchless clients and were not very happy with the
> performance. And we need it for the serverside anyway.

What kernel did you use to compare the performance?
FYI, we removed the intent patches (used on the client side)
for kernels >= 2.6.16. Basically, this means that patchless
client is the default now, even if you use the patched kernel.

Johann
___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

Re: [Lustre-discuss] quota problems in lustre 1.8

2009-07-01 Thread Giacinto Donvito 

Thank you Zhiyong,

with this hint I was able to find a way to solve the problem.

Cheers,
Giacinto

--  
--  
~~

Giacinto DonvitoLIBI -- EGEE3 SA1 INFN - Bari ITALY
--
giacinto.donv...@ba.infn.it   | GTalk/GMail: 
donvito.giaci...@gmail.com
tel. +39 080 5443244   Fax  +39 0805442470| Skype: giacinto_it
VOIP:  +41225481596   | MSN: donvito.giaci...@hotmail.it
AIM/iChat: gdonvito1  | Yahoo: eric1_it
--
Life is something that everyone should try at least once.
Henry J. Tillman





Il giorno 01/lug/09, alle ore 12:24, Zhiyong Landen tian ha scritto:




Hi All,

I'm experiencing some problem in a test installation of lustre  
1.8.0.X


The installation is composed by one server hosting the MDS, and two  
servers hosting the OSTs.
One of the servers has 12x2.7TB devices and the other has 16x2.3TB  
devices.


All the devices were configured with:

"tunefs.lustre --ost --mgsnode=lustr...@tcp0 --param  
ost.quota_type=ug --writeconf /dev/sdxx"


on the admin node I issued the "lfs quotacheck -ug /lustre" (I see  
read operation occurring on the both disk servers) that ends  
without error.


I was able to set-up quotas per user on the admin node and it seems  
successfully registered by checking with: "lfs quota -u donvito / 
lustre"


The problem that I see is that it is possible for a user to  
overfill the quota as the two server behave differently: one of the  
two deny writing while the other not.
I tried with both lustre rpms and vanilla (2.6.22) patched kernel  
and the result is the same. It is not related to the physical  
server as both of them sometimes has the same behaviour (But only  
one of the server at the time). I have tried with both 1.8.0 and  
1.8.0.1 and the same behaviour is observed.


As you can see the system is correctly accounting the used space  
but the server do not deny writing:


[r...@lustre01 ~]# lfs quota -u donvito /lustre
Disk quotas for user donvito (uid 501):
Filesystem  kbytes   quota   limit   grace   files   quota
limit   grace
   /lustre 124927244* 1200 1500  24  
100 100


(hint)You can use "lfs quota -v ..." to get how much quota grant  
every quota slave has.




The "messages" log on the MDS and both the OSS servers are clean  
and nothing strange is noticeable.
Do you have any ideas of where I can look in order to understand  
the problem?
It is expected by the current lquota design of lustre.  For lustre  
quota, there are two kinds of roles: quota master(mds) and quota  
slaves(osts). When you set quota, the limitation is recorded on  
quota master. When data is written on osts,  osts will get some  
quota grant from quota master if remained quota on osts isn't  
enough. But, at the same time, osts will get some kinds of "quota  
grant cache" so that quota slaves won't ask quota from quota master  
every time when they write(if so, performance will be hurted). Then  
every quota slave will judge if the request it received will trigger  
out of quota based upon the grant quota it got from mds  
_respectively_. Then you get what you saw.

Do you have any other suggestion of tests that I can do?
For causes of performance, lquota of lustre is distributed and isn't  
exactly like local quota(e.g. ext3). So what you saw is normal to  
lquota, currently you can only change you application to adapt it if  
it hurt you


Thank you very much.

Best Regards,
Giacinto

~~
Giacinto DonvitoLIBI -- EGEE2 SA1 INFN - Bari ITALY
--
giacinto.donv...@ba.infn.it  
   | GTalk/ 
GMail: donvito.giaci...@gmail.com 
tel. +39 080 5443244   Fax  +39 0805442470  VOIP:  +41225481596   |  
MSN: donvito.giaci...@hotmail.it 

Skype: giacinto_it | AIM/iChat: gdonvito1 | Yahoo: eric1_it
--
"A simple design always takes less time to finish than a complex one.
So always do the simplest thing that could possibly work."
Don we...@www.extremeprogramming.org 


"Writing about music is like dancing about architecture." - Frank  
Zappa 



___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss







smime.p7s
Description: S/MIME cryptographic signature
___
Lustre-discuss m

Re: [Lustre-discuss] OST max supported size?

2009-07-01 Thread Johann Lombardi 
On Jun 25, 2009, at 12:48 AM, Kevin Van Maren wrote:
> When ext4 (or ZFS) is used instead of ext3 as the basis for ldiskfs.
> ext4 will increase it to 16TB. Search bugzilla for more info.

For now, we limit the OST size to 8TB (can be forced with a mount  
option)
even on ext4-based ldiskfs, since this needs to be tested and  
validated first.

Cheers,
Johann
___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

Re: [Lustre-discuss] quota problems in lustre 1.8

2009-07-01 Thread Robert LeBlanc 
How did you solve this, we will be implementing quotas on our system soon
and don't want to fall into the same trap.

Thanks,

Robert LeBlanc
Life Sciences & Undergraduate Education Computer Support
Brigham Young University


On Wed, Jul 1, 2009 at 5:53 AM, Giacinto Donvito <
giacinto.donv...@ba.infn.it> wrote:

> Thank you Zhiyong,
>
> with this hint I was able to find a way to solve the problem.
>
> Cheers,
> Giacinto
>
> -- -- ~~
> Giacinto DonvitoLIBI -- EGEE3 SA1 INFN - Bari ITALY
> --
> giacinto.donv...@ba.infn.it   | GTalk/GMail:
> donvito.giaci...@gmail.com
> tel. +39 080 5443244   Fax  +39 0805442470| Skype: giacinto_it
> VOIP:  +41225481596   | MSN: donvito.giaci...@hotmail.it
> AIM/iChat: gdonvito1  | Yahoo: eric1_it
> --
> Life is something that everyone should try at least once.
>Henry J. Tillman
>
>
>
>
>
> Il giorno 01/lug/09, alle ore 12:24, Zhiyong Landen tian ha scritto:
>
>
>
>>  Hi All,
>>>
>>> I'm experiencing some problem in a test installation of lustre 1.8.0.X
>>>
>>> The installation is composed by one server hosting the MDS, and two
>>> servers hosting the OSTs.
>>> One of the servers has 12x2.7TB devices and the other has 16x2.3TB
>>> devices.
>>>
>>> All the devices were configured with:
>>>
>>> "tunefs.lustre --ost --mgsnode=lustr...@tcp0 --param ost.quota_type=ug
>>> --writeconf /dev/sdxx"
>>>
>>> on the admin node I issued the "lfs quotacheck -ug /lustre" (I see read
>>> operation occurring on the both disk servers) that ends without error.
>>>
>>> I was able to set-up quotas per user on the admin node and it seems
>>> successfully registered by checking with: "lfs quota -u donvito /lustre"
>>>
>>> The problem that I see is that it is possible for a user to overfill the
>>> quota as the two server behave differently: one of the two deny writing
>>> while the other not.
>>> I tried with both lustre rpms and vanilla (2.6.22) patched kernel and the
>>> result is the same. It is not related to the physical server as both of them
>>> sometimes has the same behaviour (But only one of the server at the time). I
>>> have tried with both 1.8.0 and 1.8.0.1 and the same behaviour is observed.
>>>
>>> As you can see the system is correctly accounting the used space but the
>>> server do not deny writing:
>>>
>>> [r...@lustre01 ~]# lfs quota -u donvito /lustre
>>> Disk quotas for user donvito (uid 501):
>>>Filesystem  kbytes   quota   limit   grace   files   quota   limit
>>> grace
>>>   /lustre 124927244* 1200 1500  24 100
>>> 100
>>>
>>
>> (hint)You can use "lfs quota -v ..." to get how much quota grant every
>> quota slave has.
>>
>>
>>> The "messages" log on the MDS and both the OSS servers are clean and
>>> nothing strange is noticeable.
>>> Do you have any ideas of where I can look in order to understand the
>>> problem?
>>>
>> It is expected by the current lquota design of lustre.  For lustre quota,
>> there are two kinds of roles: quota master(mds) and quota slaves(osts). When
>> you set quota, the limitation is recorded on quota master. When data is
>> written on osts,  osts will get some quota grant from quota master if
>> remained quota on osts isn't enough. But, at the same time, osts will get
>> some kinds of "quota grant cache" so that quota slaves won't ask quota from
>> quota master every time when they write(if so, performance will be hurted).
>> Then every quota slave will judge if the request it received will trigger
>> out of quota based upon the grant quota it got from mds _respectively_. Then
>> you get what you saw.
>>
>>> Do you have any other suggestion of tests that I can do?
>>>
>> For causes of performance, lquota of lustre is distributed and isn't
>> exactly like local quota(e.g. ext3). So what you saw is normal to lquota,
>> currently you can only change you application to adapt it if it hurt you
>>
>>>
>>> Thank you very much.
>>>
>>> Best Regards,
>>> Giacinto
>>>
>>> ~~
>>> Giacinto DonvitoLIBI -- EGEE2 SA1 INFN - Bari ITALY
>>> --
>>> giacinto.donv...@ba.infn.it 
>>>   | GTalk/GMail: donvito.giaci...@gmail.com >> donvito.giaci...@gmail.com>
>>> tel. +39 080 5443244   Fax  +39 0805442470  VOIP:  +41225481596   | MSN:
>>> donvito.giaci...@hotmail.it 
>>> Skype: giacinto_it | AIM/iChat: gdonvito1 | Yahoo: eric1_it
>>> --
>>> "A simple design always takes less time to finish than a complex one.
>>> So always do the simplest thing that could possibly work."
>>> Don we...@www.extremeprogramming.org >> we...@www.

[Lustre-discuss] Lustre-1.9.210: mkfs.ldiskfs?

2009-07-01 Thread Josephine Palencia 

OS: Centos 5.3, x86_64
Kernel: 2.6.18-128.1.6

[r...@attractor ~]# cat /proc/fs/lustre/version
lustre: 1.9.210
..

Looking for mkfs.ldiskfs during mkfs.lustre?
Did a find on previous working lustre-1.9 working and didn't find one.
Ps advice.

[r...@attractor ~]# mkfs.lustre --fsname=jwan0 --mdt --mgs /dev/hdc1

Permanent disk data:
Target: jwan0-MDT
Index:  unassigned
Lustre FS:  jwan0
Mount type: ldiskfs
Flags:  0x75
   (MDT MGS needs_index first_time update )
Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr
Parameters:

checking for existing Lustre data: not found
device size = 19085MB
2 6 18
formatting backing filesystem ldiskfs on /dev/hdc1
 target name  jwan0-MDT
 4k blocks 0
 options-J size=400 -i 4096 -I 512 -q -O 
dir_index,uninit_groups -F
mkfs_cmd = mkfs.ldiskfs -j -b 4096 -L jwan0-MDT  -J size=400 -i 4096 
-I 512 -q -O dir_index,uninit_groups -F /dev/hdc1
sh: mkfs.ldiskfs: command not found

mkfs.lustre FATAL: Unable to build fs /dev/hdc1 (32512)

mkfs.lustre FATAL: mkfs failed 32512


Thanks,
josephin
___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

Re: [Lustre-discuss] quota problems in lustre 1.8

2009-07-01 Thread Giacinto Donvito 
Basically, when a new OSS server was added, I followed this procedure  
(I'm not sure that all the steps are needed but this works with our  
installation):


quotainv
quotacheck
lfs setquota -u donvito --block-softlimit 0 --block-hardlimit 0 /lustre
lfs setquota -u donvito --block-softlimit 200 --block-hardlimit  
100 /lustre


after this all the OSTs have the quota showed correctly.

I hope that this will be useful for you too.

Cheers,
Giacinto

--
--
~~
Giacinto DonvitoLIBI -- EGEE3 SA1 INFN - Bari ITALY
--
giacinto.donv...@ba.infn.it   | GTalk/GMail: 
donvito.giaci...@gmail.com
tel. +39 080 5443244   Fax  +39 0805442470| Skype: giacinto_it
VOIP:  +41225481596   | MSN: donvito.giaci...@hotmail.it
AIM/iChat: gdonvito1  | Yahoo: eric1_it
--
"Not everything that can be counted counts, and not everything that  
counts can be counted."

- Albert Einstein (1879-1955)






Il giorno 01/lug/09, alle ore 16:59, Robert LeBlanc ha scritto:

How did you solve this, we will be implementing quotas on our system  
soon and don't want to fall into the same trap.


Thanks,

Robert LeBlanc
Life Sciences & Undergraduate Education Computer Support
Brigham Young University


On Wed, Jul 1, 2009 at 5:53 AM, Giacinto Donvito > wrote:

Thank you Zhiyong,

with this hint I was able to find a way to solve the problem.

Cheers,
Giacinto

-- --  
~~

Giacinto DonvitoLIBI -- EGEE3 SA1 INFN - Bari ITALY
--
giacinto.donv...@ba.infn.it   | GTalk/GMail: 
donvito.giaci...@gmail.com
tel. +39 080 5443244   Fax  +39 0805442470| Skype: giacinto_it

VOIP:  +41225481596   | MSN: donvito.giaci...@hotmail.it
AIM/iChat: gdonvito1  | Yahoo: eric1_it
--
Life is something that everyone should try at least once.
   Henry J. Tillman





Il giorno 01/lug/09, alle ore 12:24, Zhiyong Landen tian ha scritto:



Hi All,

I'm experiencing some problem in a test installation of lustre 1.8.0.X

The installation is composed by one server hosting the MDS, and two  
servers hosting the OSTs.
One of the servers has 12x2.7TB devices and the other has 16x2.3TB  
devices.


All the devices were configured with:

"tunefs.lustre --ost --mgsnode=lustr...@tcp0 --param  
ost.quota_type=ug --writeconf /dev/sdxx"


on the admin node I issued the "lfs quotacheck -ug /lustre" (I see  
read operation occurring on the both disk servers) that ends without  
error.


I was able to set-up quotas per user on the admin node and it seems  
successfully registered by checking with: "lfs quota -u donvito / 
lustre"


The problem that I see is that it is possible for a user to overfill  
the quota as the two server behave differently: one of the two deny  
writing while the other not.
I tried with both lustre rpms and vanilla (2.6.22) patched kernel  
and the result is the same. It is not related to the physical server  
as both of them sometimes has the same behaviour (But only one of  
the server at the time). I have tried with both 1.8.0 and 1.8.0.1  
and the same behaviour is observed.


As you can see the system is correctly accounting the used space but  
the server do not deny writing:


[r...@lustre01 ~]# lfs quota -u donvito /lustre
Disk quotas for user donvito (uid 501):
   Filesystem  kbytes   quota   limit   grace   files   quota
limit   grace
  /lustre 124927244* 1200 1500  24  
100 100


(hint)You can use "lfs quota -v ..." to get how much quota grant  
every quota slave has.



The "messages" log on the MDS and both the OSS servers are clean and  
nothing strange is noticeable.
Do you have any ideas of where I can look in order to understand the  
problem?
It is expected by the current lquota design of lustre.  For lustre  
quota, there are two kinds of roles: quota master(mds) and quota  
slaves(osts). When you set quota, the limitation is recorded on  
quota master. When data is written on osts,  osts will get some  
quota grant from quota master if remained quota on osts isn't  
enough. But, at the same time, osts will get some kinds of "quota  
grant cache" so that quota slaves won't ask quota from quota master  
every time when they write(if so, performance will be hurted). Then  
every quota slave will judge if the request it received will trigger  
out of quota based upon the grant quota it got from mds  
_respectively_. Then you get what you saw.

Do you have any other suggestion of tests that I can do?
For causes of performance, lquota of lustre is distributed and isn't  
exactly like local quota(e.g. ext3). So what you saw is n

Re: [Lustre-discuss] Lustre-1.9.210: mkfs.ldiskfs?

2009-07-01 Thread Andreas Dilger 
On Jul 01, 2009  13:03 -0400, Josephine Palencia wrote:
> [r...@attractor ~]# cat /proc/fs/lustre/version
> lustre: 1.9.210
> 
> Looking for mkfs.ldiskfs during mkfs.lustre?
> Did a find on previous working lustre-1.9 working and didn't find one.
> Ps advice.

Well, mkfs.ldiskfs doesn't exist, it should just be using mkfs.ext3
to create the backing filesystem.

Please file a bug against the 1.9.210 release (if it exists), or HEAD.

> [r...@attractor ~]# mkfs.lustre --fsname=jwan0 --mdt --mgs /dev/hdc1
> 
> Permanent disk data:
> Target: jwan0-MDT
> Index:  unassigned
> Lustre FS:  jwan0
> Mount type: ldiskfs
> Flags:  0x75
>(MDT MGS needs_index first_time update )
> Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr
> Parameters:
> 
> checking for existing Lustre data: not found
> device size = 19085MB
> 2 6 18
> formatting backing filesystem ldiskfs on /dev/hdc1
>  target name  jwan0-MDT
>  4k blocks 0
>  options-J size=400 -i 4096 -I 512 -q -O 
> dir_index,uninit_groups -F
> mkfs_cmd = mkfs.ldiskfs -j -b 4096 -L jwan0-MDT  -J size=400 -i 4096 
> -I 512 -q -O dir_index,uninit_groups -F /dev/hdc1
> sh: mkfs.ldiskfs: command not found
> 
> mkfs.lustre FATAL: Unable to build fs /dev/hdc1 (32512)
> 
> mkfs.lustre FATAL: mkfs failed 32512

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss

Re: [Lustre-discuss] Lustre-1.9.210: mkfs.ldiskfs?

2009-07-01 Thread Robert Read 

On Jul 1, 2009, at 10:03 , Josephine Palencia wrote:

>
> OS: Centos 5.3, x86_64
> Kernel: 2.6.18-128.1.6
>
> [r...@attractor ~]# cat /proc/fs/lustre/version
> lustre: 1.9.210
> ..
>
> Looking for mkfs.ldiskfs during mkfs.lustre?
> Did a find on previous working lustre-1.9 working and didn't find one.
> Ps advice.
>


What configure options did you use when you built lustre?  I think "-- 
with-ldiskfsprogs" will configure lustre to use these names, however  
the standard version the e2fs utils doesn't use this naming scheme.   
This would be used, for instance, if you needed to have both the  
original and ldiskfs version of e2fsutils installed at the same time.

robert


> [r...@attractor ~]# mkfs.lustre --fsname=jwan0 --mdt --mgs /dev/hdc1
>
>Permanent disk data:
> Target: jwan0-MDT
> Index:  unassigned
> Lustre FS:  jwan0
> Mount type: ldiskfs
> Flags:  0x75
>   (MDT MGS needs_index first_time update )
> Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr
> Parameters:
>
> checking for existing Lustre data: not found
> device size = 19085MB
> 2 6 18
> formatting backing filesystem ldiskfs on /dev/hdc1
> target name  jwan0-MDT
> 4k blocks 0
> options-J size=400 -i 4096 -I 512 -q -O
> dir_index,uninit_groups -F
> mkfs_cmd = mkfs.ldiskfs -j -b 4096 -L jwan0-MDT  -J size=400 -i  
> 4096
> -I 512 -q -O dir_index,uninit_groups -F /dev/hdc1
>sh: mkfs.ldiskfs: command not found

>
> mkfs.lustre FATAL: Unable to build fs /dev/hdc1 (32512)
>
> mkfs.lustre FATAL: mkfs failed 32512
>
>
> Thanks,
> josephin
> ___
> Lustre-discuss mailing list
> Lustre-discuss@lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss

___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss