Re: [Gluster-devel] DHT xlator, read and write a file during creating a new file
Do you have quota enabled ? If you did may be the crawl has not happened yet on the directory (can take a while on a deep hierarchy), On Wed, May 3, 2017 at 9:57 PM, Tahereh Fattahi wrote: > Hi > > I test getfattr -d -m. -e hex ' but i get different result: > getfattr: Removing leading '/' from absolute path names > # file: data/brick1/gv2/c > trusted.gfid=0xae8c02978686440dac3754880dde5b93 > trusted.glusterfs.dht=0x0001 > > maybe it is better to create a new question a bout detail of directory > divided by brik! > > On Wed, May 3, 2017 at 4:15 PM, Tahereh Fattahi > wrote: > >> Yes, it is ok >> Is it possible to see this attribute in client side? In which xlator? >> >> On Wed, May 3, 2017 at 2:58 PM, Sanoj Unnikrishnan >> wrote: >> >>> I am not sure, if i got your requirement correctly. >>> If you need to have the file count and directory count of each brick >>> individually (without aggregation), you could get that by a getxattr of >>> trusted.glusterfs.quota.size on the brick. >>> The size xattr value can be interpreted as follows: >>> getfattr -d -m. -e hex ' >>> size: first 16 bytes >>> filecount: Next 16 bytes >>> dircount: Last 16 bytes >>> >>> Regards, >>> Sanoj >>> >>> On Wed, May 3, 2017 at 12:20 PM, Tahereh Fattahi >>> wrote: >>> Thank you I need this information divided to bricks. On Tue, May 2, 2017 at 11:07 PM, Vijay Bellur wrote: > > > On Tue, May 2, 2017 at 8:00 AM, Tahereh Fattahi > wrote: > >> Hi >> >> I want to use a file as a counter when I create a file in dht xlator. >> I mean, after creating a new file, I want open a file in the same >> directory with a special name, read that, update the counter and write >> back. >> I think for this purpose I should open in dht_create_cbk, read in >> dht_open_cbk and write in dht_readv_cbk. >> I think I should use dht_open , dht_readv and dht_writev. Maybe I >> could create inputs for these function expect frame! is it correct to use >> the frame fro dht_create function? >> >> Is this scenario correct or there is better way? >> >> > Have you tried the object count feature [1] ? > > Regards, > Vijay > > [1] http://gluster-documentations.readthedocs.io/en/latest/Featu > res/quota-object-count/ > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel >>> >>> >> > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] DHT xlator, read and write a file during creating a new file
Hi I test getfattr -d -m. -e hex ' but i get different result: getfattr: Removing leading '/' from absolute path names # file: data/brick1/gv2/c trusted.gfid=0xae8c02978686440dac3754880dde5b93 trusted.glusterfs.dht=0x0001 maybe it is better to create a new question a bout detail of directory divided by brik! On Wed, May 3, 2017 at 4:15 PM, Tahereh Fattahi wrote: > Yes, it is ok > Is it possible to see this attribute in client side? In which xlator? > > On Wed, May 3, 2017 at 2:58 PM, Sanoj Unnikrishnan > wrote: > >> I am not sure, if i got your requirement correctly. >> If you need to have the file count and directory count of each brick >> individually (without aggregation), you could get that by a getxattr of >> trusted.glusterfs.quota.size on the brick. >> The size xattr value can be interpreted as follows: >> getfattr -d -m. -e hex ' >> size: first 16 bytes >> filecount: Next 16 bytes >> dircount: Last 16 bytes >> >> Regards, >> Sanoj >> >> On Wed, May 3, 2017 at 12:20 PM, Tahereh Fattahi >> wrote: >> >>> Thank you >>> I need this information divided to bricks. >>> >>> On Tue, May 2, 2017 at 11:07 PM, Vijay Bellur >>> wrote: >>> On Tue, May 2, 2017 at 8:00 AM, Tahereh Fattahi wrote: > Hi > > I want to use a file as a counter when I create a file in dht xlator. > I mean, after creating a new file, I want open a file in the same > directory with a special name, read that, update the counter and write > back. > I think for this purpose I should open in dht_create_cbk, read in > dht_open_cbk and write in dht_readv_cbk. > I think I should use dht_open , dht_readv and dht_writev. Maybe I > could create inputs for these function expect frame! is it correct to use > the frame fro dht_create function? > > Is this scenario correct or there is better way? > > Have you tried the object count feature [1] ? Regards, Vijay [1] http://gluster-documentations.readthedocs.io/en/latest/Featu res/quota-object-count/ >>> >>> >>> ___ >>> Gluster-devel mailing list >>> Gluster-devel@gluster.org >>> http://lists.gluster.org/mailman/listinfo/gluster-devel >>> >> >> > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] DHT xlator, read and write a file during creating a new file
Yes, it is ok Is it possible to see this attribute in client side? In which xlator? On Wed, May 3, 2017 at 2:58 PM, Sanoj Unnikrishnan wrote: > I am not sure, if i got your requirement correctly. > If you need to have the file count and directory count of each brick > individually (without aggregation), you could get that by a getxattr of > trusted.glusterfs.quota.size on the brick. > The size xattr value can be interpreted as follows: > getfattr -d -m. -e hex ' > size: first 16 bytes > filecount: Next 16 bytes > dircount: Last 16 bytes > > Regards, > Sanoj > > On Wed, May 3, 2017 at 12:20 PM, Tahereh Fattahi > wrote: > >> Thank you >> I need this information divided to bricks. >> >> On Tue, May 2, 2017 at 11:07 PM, Vijay Bellur wrote: >> >>> >>> >>> On Tue, May 2, 2017 at 8:00 AM, Tahereh Fattahi >>> wrote: >>> Hi I want to use a file as a counter when I create a file in dht xlator. I mean, after creating a new file, I want open a file in the same directory with a special name, read that, update the counter and write back. I think for this purpose I should open in dht_create_cbk, read in dht_open_cbk and write in dht_readv_cbk. I think I should use dht_open , dht_readv and dht_writev. Maybe I could create inputs for these function expect frame! is it correct to use the frame fro dht_create function? Is this scenario correct or there is better way? >>> Have you tried the object count feature [1] ? >>> >>> Regards, >>> Vijay >>> >>> [1] http://gluster-documentations.readthedocs.io/en/latest/Featu >>> res/quota-object-count/ >>> >> >> >> ___ >> Gluster-devel mailing list >> Gluster-devel@gluster.org >> http://lists.gluster.org/mailman/listinfo/gluster-devel >> > > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] DHT xlator, read and write a file during creating a new file
I am not sure, if i got your requirement correctly. If you need to have the file count and directory count of each brick individually (without aggregation), you could get that by a getxattr of trusted.glusterfs.quota.size on the brick. The size xattr value can be interpreted as follows: getfattr -d -m. -e hex ' size: first 16 bytes filecount: Next 16 bytes dircount: Last 16 bytes Regards, Sanoj On Wed, May 3, 2017 at 12:20 PM, Tahereh Fattahi wrote: > Thank you > I need this information divided to bricks. > > On Tue, May 2, 2017 at 11:07 PM, Vijay Bellur wrote: > >> >> >> On Tue, May 2, 2017 at 8:00 AM, Tahereh Fattahi >> wrote: >> >>> Hi >>> >>> I want to use a file as a counter when I create a file in dht xlator. >>> I mean, after creating a new file, I want open a file in the same >>> directory with a special name, read that, update the counter and write >>> back. >>> I think for this purpose I should open in dht_create_cbk, read in >>> dht_open_cbk and write in dht_readv_cbk. >>> I think I should use dht_open , dht_readv and dht_writev. Maybe I could >>> create inputs for these function expect frame! is it correct to use the >>> frame fro dht_create function? >>> >>> Is this scenario correct or there is better way? >>> >>> >> Have you tried the object count feature [1] ? >> >> Regards, >> Vijay >> >> [1] http://gluster-documentations.readthedocs.io/en/latest/Featu >> res/quota-object-count/ >> > > > ___ > Gluster-devel mailing list > Gluster-devel@gluster.org > http://lists.gluster.org/mailman/listinfo/gluster-devel > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] DHT xlator, read and write a file during creating a new file
Thank you I need this information divided to bricks. On Tue, May 2, 2017 at 11:07 PM, Vijay Bellur wrote: > > > On Tue, May 2, 2017 at 8:00 AM, Tahereh Fattahi > wrote: > >> Hi >> >> I want to use a file as a counter when I create a file in dht xlator. >> I mean, after creating a new file, I want open a file in the same >> directory with a special name, read that, update the counter and write >> back. >> I think for this purpose I should open in dht_create_cbk, read in >> dht_open_cbk and write in dht_readv_cbk. >> I think I should use dht_open , dht_readv and dht_writev. Maybe I could >> create inputs for these function expect frame! is it correct to use the >> frame fro dht_create function? >> >> Is this scenario correct or there is better way? >> >> > Have you tried the object count feature [1] ? > > Regards, > Vijay > > [1] http://gluster-documentations.readthedocs.io/en/latest/ > Features/quota-object-count/ > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] DHT xlator, read and write a file during creating a new file
On Tue, May 2, 2017 at 8:00 AM, Tahereh Fattahi wrote: > Hi > > I want to use a file as a counter when I create a file in dht xlator. > I mean, after creating a new file, I want open a file in the same > directory with a special name, read that, update the counter and write > back. > I think for this purpose I should open in dht_create_cbk, read in > dht_open_cbk and write in dht_readv_cbk. > I think I should use dht_open , dht_readv and dht_writev. Maybe I could > create inputs for these function expect frame! is it correct to use the > frame fro dht_create function? > > Is this scenario correct or there is better way? > > Have you tried the object count feature [1] ? Regards, Vijay [1] http://gluster-documentations.readthedocs.io/en/latest/Features/quota-object-count/ ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] DHT xlator, read and write a file during creating a new file
Thank you very much I had test this before. I want all the client can see this counter and can update that, so I add an attribute for directory. setxattr for directory was very in-efficient (I saw a lot of lookup request to server because of setxattr) for every create file when our workload generate a lot of file. So which one is better in your opinion? On Tue, May 2, 2017 at 5:06 PM, Amar Tumballi wrote: > > > On Tue, May 2, 2017 at 5:30 PM, Tahereh Fattahi > wrote: > >> Hi >> >> I want to use a file as a counter when I create a file in dht xlator. >> I mean, after creating a new file, I want open a file in the same >> directory with a special name, read that, update the counter and write >> back. >> I think for this purpose I should open in dht_create_cbk, read in >> dht_open_cbk and write in dht_readv_cbk. >> I think I should use dht_open , dht_readv and dht_writev. Maybe I could >> create inputs for these function expect frame! is it correct to use the >> frame fro dht_create function? >> >> Is this scenario correct or there is better way? >> >> This is correct. But very in-efficient (for so many different fops for > one fop from user). See if you can keep another extended attribute itself > which you can update. That way, you can just handle the counter management > using 'xattrop' or `setxattr()' fops > > Regards, > Amar > >> ___ >> Gluster-devel mailing list >> Gluster-devel@gluster.org >> http://lists.gluster.org/mailman/listinfo/gluster-devel >> > > > > -- > Amar Tumballi (amarts) > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] DHT xlator, read and write a file during creating a new file
On Tue, May 2, 2017 at 5:30 PM, Tahereh Fattahi wrote: > Hi > > I want to use a file as a counter when I create a file in dht xlator. > I mean, after creating a new file, I want open a file in the same > directory with a special name, read that, update the counter and write > back. > I think for this purpose I should open in dht_create_cbk, read in > dht_open_cbk and write in dht_readv_cbk. > I think I should use dht_open , dht_readv and dht_writev. Maybe I could > create inputs for these function expect frame! is it correct to use the > frame fro dht_create function? > > Is this scenario correct or there is better way? > > This is correct. But very in-efficient (for so many different fops for one fop from user). See if you can keep another extended attribute itself which you can update. That way, you can just handle the counter management using 'xattrop' or `setxattr()' fops Regards, Amar > ___ > Gluster-devel mailing list > Gluster-devel@gluster.org > http://lists.gluster.org/mailman/listinfo/gluster-devel > -- Amar Tumballi (amarts) ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
[Gluster-devel] DHT xlator, read and write a file during creating a new file
Hi I want to use a file as a counter when I create a file in dht xlator. I mean, after creating a new file, I want open a file in the same directory with a special name, read that, update the counter and write back. I think for this purpose I should open in dht_create_cbk, read in dht_open_cbk and write in dht_readv_cbk. I think I should use dht_open , dht_readv and dht_writev. Maybe I could create inputs for these function expect frame! is it correct to use the frame fro dht_create function? Is this scenario correct or there is better way? ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel