Thank you Phil and Rob. I could see the file size properly with pvfs2-statfs and karma gui. Phil you mentioned in detail how pvfs2 calculates free space. Now if one of the drives becomes full(say one of the 60 GB drives) and there is space on other servers, will pvfs2 distribute data among them evenly or the pvfs2 volume will beunusable?
On Fri, Feb 6, 2009 at 11:16 AM, Phil Carns <[email protected]> wrote: > Hi Abhinav, > > To summarize, in total it sounds like you have PVFS servers running on 14 > nodes. Most of those nodes only have 60 GB hard drives, but one of them has > 640 GB of storage. > > PVFS doesn't have any mechanism to account for unbalanced storage on each > of the servers. If any one of your servers runs out of space then the PVFS > volume is pretty much out of space as well. So when it calculates the total > space on the file system (for df) it uses this algorithm: > > <capacity of smallest server> * <number of servers> > > ... which gets you to the 850 GB value that df is reporting. > > As far as the 52% being allocated already, this probably means that on at > least some of your servers, the server's storage directory is on the same > partition as your root file system (or some other data). That means that > things external to PVFS are counting against the capacity on some servers. > There is nothing wrong with that necessarily. Most likely one or more of > your machines has used up half of its 60 GB already and PVFS therefore only > has maybe 30 GB * 14 available out of the 60 GB * 14 total. > > If you want more detail about how much space is available on each > individual server, you can use the pvfs2-statfs command. > > There isn't any way to set a limit on each server from the PVFS > perspective. You could do it at the node level by using a separate > partition for the PVFS data, though. > > -Phil > > Abhinav Chawade wrote: > >> I have a 14 node cluster with 1 storage server 1 head node and 12 >> computational node running centos 5.2 >> I deployed pvfs2 on all of them and head and storage node are metadata >> servers. Each computational node has 60GB hard drive and the storage server >> has 640 GB storage. >> When i mount the file system and ping it, all servers are running. I try >> to see how much free space is left using df command. df output shows all in >> all 850 GB storage out of which 52% is allocated already. There isn't a >> single file in pvfs2 mounted partition. How can i check size of pvfs2 space >> i created? Is there a way to set limit on storage on each server? >> >> Abhinav >> >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> Pvfs2-users mailing list >> [email protected] >> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users >> > >
_______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
