Volume configuration:
yval1000:/ # gluster volume info
Volume Name: venus
Type: Distributed-Replicate
Status: Started
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Bricks:
Brick1: ylal3020:/soft/venus
Brick2: ylal3030:/soft/venus
Brick3: yval1000:/soft/venus
Brick4: yval1010:/soft/venus
Options Reconfigured:
features.quota-timeout: 120
features.limit-usage: /test:100MB,/psa:200MB,/:7GB,/soft:5GB
features.quota: on
geo-replication.indexing: on
network.ping-timeout: 10
performance.cache-size: 1GB
performance.cache-refresh-timeout: 60
nfs.port: 2049
Indeed values didn't converged automaticaly for big folder, for small folder
like /test and /psa it was fine (I waited 1 day before checking the value). I
was forced to run ls -lRa several times, but even with this workaround, values
for big folder like /soft are not rightfully calculated :
gluster volume quota venus list
path limit_set size
----------------------------------------------------------------------------------
/test 100MB 4.5MB <= OK
/psa 200MB 167.0MB <=OK
/ 7GB 5.2GB <=Strange
/soft 5GB 4.7GB <= Strange
=> the size of the entire filesystem is 1.4GB
Anthony
From: [email protected]
Date: Mon, 5 Sep 2011 14:38:06 +0530
Subject: Re: [Gluster-users] quota calculation gluster 3.2.3
To: [email protected]
CC: [email protected]
Hi Anthony,
By volume configuration, I meant 'is the volume setup dht, afr, or dht-afr', I
would like to know the output of
volume info volume-name
When data is already present before the activation of quota, it takes some time
for quota values to converge because it has to crawl through the entire
file-system to calculate the sizes. This is automatically done when you run
volume quota vol-name enable
The output of volume quota vol-name list
will reflect the aggregated value at that point in time. If the values are not
converging then its a issue. A simple work-around is to run ls -lR on the mount
point. Please confirm if the values were not converging automatically, we will
have a look into it.
Junaid
On Mon, Sep 5, 2011 at 12:51 PM, anthony garnier <[email protected]> wrote:
Junaid,
What do you mean by " the volume configuration on other two servers" ?
Configuration is the same on all the server..
I'm creating data with "dd" cmd, and yes, data was already present before
activation of quota.
From: [email protected]
Date: Fri, 2 Sep 2011 22:55:02 +0530
Subject: Re: [Gluster-users] quota calculation gluster 3.2.3
To: [email protected]
CC: [email protected]
Hi Anthony,
Thanks for providing the input. What is the volume configuration on other two
servers (yval1000 and yval1010)? My question is same, what data where you
creating on these two machines? Was the data already present before quota was
enabled?
On Fri, Sep 2, 2011 at 1:49 PM, anthony garnier <[email protected]> wrote:
I've done a ls -lRa throught the mount point and now i get :
/soft/venus # gluster volume quota venus list
path limit_set size
----------------------------------------------------------------------------------
/ 2GB 1.3GB
/test 100MB 8.0KB
/psa 200MB 151.0MB
/soft 1500MB 826.3MB
=>Seems to be consistent
and now command getfattr on yval1000 and yval1010 show :
yval1000 :
getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/
trusted.glusterfs.quota.size=0x0000000028f9ca00
getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/soft
trusted.glusterfs.quota.size=0x000000001421d400
getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/psa
trusted.glusterfs.quota.size=0x00000000053d6000
yval1010 :
getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/
trusted.glusterfs.quota.size=0x000000002904ea00
getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/soft
trusted.glusterfs.quota.size=0x000000002292ba00
getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/psa
trusted.glusterfs.quota.size=0x00000000053d6000
Regards,
Anthony
From: [email protected]
To: [email protected]
CC: [email protected]
Subject: RE: [Gluster-users] quota calculation gluster 3.2.3
Date: Fri, 2 Sep 2011 07:57:19 +0000
I forgot to check on the 2 other server.
So ylal3020 and ylal3030 show similar output send previously.
But yval1000 and yval1010 show with all command :
trusted.glusterfs.quota.size=0x0000000000000000
Is it normal ?
Regards,
Anthony
From: [email protected]
To: [email protected]
CC: [email protected]
Subject: RE: [Gluster-users] quota calculation gluster 3.2.3
Date: Fri, 2 Sep 2011 07:51:52 +0000
Hi Junaid,
Here is the output of the diffrent command executed on both server.
Command show identical output on both server
# getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus
getfattr: Removing leading '/' from absolute path names
# file: soft/venus
trusted.glusterfs.quota.size=0x0000000022ccac00
# getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/test
getfattr: Removing leading '/' from absolute path names
# file: soft/venus/test
trusted.glusterfs.quota.size=0x0000000000000000
# getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/psa
getfattr: Removing leading '/' from absolute path names
# file: soft/venus/psa
trusted.glusterfs.quota.size=0x000000000013d000
# getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/soft
getfattr: Removing leading '/' from absolute path names
# file: soft/venus/soft
trusted.glusterfs.quota.size=0x000000000d15ce00
# gluster volume quota venus list
path limit_set size
----------------------------------------------------------------------------------
/ 2GB 556.8MB
/test 100MB 0Bytes
/psa 200MB 1.2MB
/soft 1500MB 209.4MB
Indeed there is 90% of small file (~60000 files).
Thx
Anthony
From: [email protected]
Date: Fri, 2 Sep 2011 06:18:30 +0530
Subject: Re: [Gluster-users] quota calculation gluster 3.2.3
To: [email protected]
CC: [email protected]
Hi Anthony,
To debug this further, can you send the output of
getfattr -n trusted.glusterfs.quota.size -e hex /soft/venusgetfattr -n
trusted.glusterfs.quota.size -e hex /soft/venus/test
getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/psagetfattr -n
trusted.glusterfs.quota.size -e hex /soft/venus/soft
from both the machine. Also check the output of
gluster volume quota Replicate list
Sometimes it takes a small amount of time to bring up the sizes. Also, what
kind of data where you creating (I mean large amount of small files or large
files because in case of small files, the directory sizes are not accounted by
quota to calculate the size unlike the du -h command which uses the directory
size as well).
Junaid
On Thu, Sep 1, 2011 at 1:37 PM, anthony garnier <[email protected]> wrote:
Hi all,
I've enable quota but I'm a bit confused by values displayed by GlusterFS
Here is my volume :
Volume Name: venus
Type: Replicate
Status: Started
Number of Bricks: 2
Transport-type: tcp
Bricks:
Brick1: ylal3020:/soft/venus
Brick2: ylal3030:/soft/venus
Options Reconfigured:
features.limit-usage: /test:100MB,/psa:200MB,/soft:1500MB,/:2GB
features.quota: on
diagnostics.client-log-level: ERROR
diagnostics.brick-log-level: ERROR
network.ping-timeout: 10
performance.cache-size: 2GB
nfs.port: 2049
I've got 3 folders in the backend (/soft/venus) :
psa 160MB (with du -sh)
soft 1.2GB
test 12KB
Total 1.4GB
But when I list the quota with gluster I got :
# gluster volume quota venus list
path limit_set size
----------------------------------------------------------------------------------
/test 100MB 12.0KB <= This
one is OK
/psa 200MB 64.4MB <= not OK
/soft 1500MB 281.8MB <= not OK
/ 2GB 346.2MB <= not OK
Any idea ?
Regards,
Anthony Garnier
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
