Re: [Gluster-devel] About spread count, lookup and layout

2017-04-29 Thread Raghavendra Gowdappa


- Original Message -
> From: "Tahereh Fattahi" 
> To: "Raghavendra Gowdappa" 
> Cc: "Gluster Devel" 
> Sent: Sunday, April 30, 2017 10:04:25 AM
> Subject: Re: [Gluster-devel] About spread count, lookup and layout
> 
> Thanks a lot
> I read in document that when we want to create a file, first we should be
> sure that this file does not exist in the directory. So first look at the
> hashed subvolume and if it does not exist then broad cast this lookup to
> other subvolume (when lookup optimize is off) to ensure that this file was
> not created before.

lookup-everywhere was done to handle cases where layout might be wrong, not to 
handle racing creates.

> I had thought that lookup everywhere is for searching file not for layout.

lookup everywhere searches for file.

> I want to reduce the lookup for file (not layout) when lookup optimize is
> off and compare it when lookup optimize is on. (of course lookup optimize
> on is better , I want do this)

For the cases where file is not present (negative-lookup scenario), lookup 
optimize will always result in smaller number of lookups.

> 
> On Sun, Apr 30, 2017 at 8:12 AM, Raghavendra Gowdappa 
> wrote:
> 
> >
> >
> > - Original Message -
> > > From: "Tahereh Fattahi" 
> > > To: "Gluster Devel" 
> > > Sent: Sunday, April 30, 2017 8:03:15 AM
> > > Subject: [Gluster-devel] About spread count, lookup and layout
> > >
> > > Hi
> > > I have some question about spread count:
> > >
> > > 1. When I set spread count less than subvolume count, I see the hash is
> > > assigned to one directory layout for some subvolumes are zero (that is
> > ok) ,
> > > but I expect these subvolumes should not bee looked up in case
> > > lookup_everywhere for creating a file. Unfortunately I see these lookups
> > in
> > > server profile.
> > > Am I correct?
> >
> > Yes. lookup-everywhere is a fall back mechanism for the scenarios where
> > layout can't be trusted (like in the window b/w fix layout after add/remove
> > brick etc and healing of the immediate children of directory as per the new
> > layout). Since layout itself can't be trusted, it doesn't matter whether it
> > has zero ranges or not. A better way of solving this problem (than
> > identifying 0 ranges etc) is lookup-optimize, which identifies scenarios
> > (predominantly add/remove brick) and decides whether to use
> > lookup-everywhere before saying that a file/directory is not present.
> >
> > > is there any way for this case to reduce the amount of lookups that is
> > send
> > > to the servers? (lookup optimize is off and I want this)
> >
> > Is there any reason why you want lookup-optimize to be off?
> >
> > >
> > > 2. I see the option for changing the spread count and fixing the layout
> > in
> > > dht xlaor (dht_setxattr), but when I tried to set this attribute and
> > change
> > > the layout I dont see any change!! while there is no error and crash (I
> > call
> > > setfattr in client side because I dont see dht xlator in server graph)
> >
> > Probably there might be bugs in that codepath. Can you please file a bug?
> >
> > regards,
> > Raghavendra
> >
> > >
> > > ___
> > > Gluster-devel mailing list
> > > Gluster-devel@gluster.org
> > > http://lists.gluster.org/mailman/listinfo/gluster-devel
> >
> 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel


Re: [Gluster-devel] About spread count, lookup and layout

2017-04-29 Thread Tahereh Fattahi
Thanks a lot
I read in document that when we want to create a file, first we should be
sure that this file does not exist in the directory. So first look at the
hashed subvolume and if it does not exist then broad cast this lookup to
other subvolume (when lookup optimize is off) to ensure that this file was
not created before.
I had thought that lookup everywhere is for searching file not for layout.
I want to reduce the lookup for file (not layout) when lookup optimize is
off and compare it when lookup optimize is on. (of course lookup optimize
on is better , I want do this)

On Sun, Apr 30, 2017 at 8:12 AM, Raghavendra Gowdappa 
wrote:

>
>
> - Original Message -
> > From: "Tahereh Fattahi" 
> > To: "Gluster Devel" 
> > Sent: Sunday, April 30, 2017 8:03:15 AM
> > Subject: [Gluster-devel] About spread count, lookup and layout
> >
> > Hi
> > I have some question about spread count:
> >
> > 1. When I set spread count less than subvolume count, I see the hash is
> > assigned to one directory layout for some subvolumes are zero (that is
> ok) ,
> > but I expect these subvolumes should not bee looked up in case
> > lookup_everywhere for creating a file. Unfortunately I see these lookups
> in
> > server profile.
> > Am I correct?
>
> Yes. lookup-everywhere is a fall back mechanism for the scenarios where
> layout can't be trusted (like in the window b/w fix layout after add/remove
> brick etc and healing of the immediate children of directory as per the new
> layout). Since layout itself can't be trusted, it doesn't matter whether it
> has zero ranges or not. A better way of solving this problem (than
> identifying 0 ranges etc) is lookup-optimize, which identifies scenarios
> (predominantly add/remove brick) and decides whether to use
> lookup-everywhere before saying that a file/directory is not present.
>
> > is there any way for this case to reduce the amount of lookups that is
> send
> > to the servers? (lookup optimize is off and I want this)
>
> Is there any reason why you want lookup-optimize to be off?
>
> >
> > 2. I see the option for changing the spread count and fixing the layout
> in
> > dht xlaor (dht_setxattr), but when I tried to set this attribute and
> change
> > the layout I dont see any change!! while there is no error and crash (I
> call
> > setfattr in client side because I dont see dht xlator in server graph)
>
> Probably there might be bugs in that codepath. Can you please file a bug?
>
> regards,
> Raghavendra
>
> >
> > ___
> > Gluster-devel mailing list
> > Gluster-devel@gluster.org
> > http://lists.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] About spread count, lookup and layout

2017-04-29 Thread Raghavendra Gowdappa


- Original Message -
> From: "Tahereh Fattahi" 
> To: "Gluster Devel" 
> Sent: Sunday, April 30, 2017 8:03:15 AM
> Subject: [Gluster-devel] About spread count, lookup and layout
> 
> Hi
> I have some question about spread count:
> 
> 1. When I set spread count less than subvolume count, I see the hash is
> assigned to one directory layout for some subvolumes are zero (that is ok) ,
> but I expect these subvolumes should not bee looked up in case
> lookup_everywhere for creating a file. Unfortunately I see these lookups in
> server profile.
> Am I correct?

Yes. lookup-everywhere is a fall back mechanism for the scenarios where layout 
can't be trusted (like in the window b/w fix layout after add/remove brick etc 
and healing of the immediate children of directory as per the new layout). 
Since layout itself can't be trusted, it doesn't matter whether it has zero 
ranges or not. A better way of solving this problem (than identifying 0 ranges 
etc) is lookup-optimize, which identifies scenarios (predominantly add/remove 
brick) and decides whether to use lookup-everywhere before saying that a 
file/directory is not present. 

> is there any way for this case to reduce the amount of lookups that is send
> to the servers? (lookup optimize is off and I want this)

Is there any reason why you want lookup-optimize to be off?

> 
> 2. I see the option for changing the spread count and fixing the layout in
> dht xlaor (dht_setxattr), but when I tried to set this attribute and change
> the layout I dont see any change!! while there is no error and crash (I call
> setfattr in client side because I dont see dht xlator in server graph)

Probably there might be bugs in that codepath. Can you please file a bug?

regards,
Raghavendra

> 
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel


[Gluster-devel] About spread count, lookup and layout

2017-04-29 Thread Tahereh Fattahi
Hi
I have some question about spread count:

1. When I set spread count less than subvolume count, I see the hash is
assigned to one directory layout for some subvolumes are zero (that is ok)
, but I expect these subvolumes should not bee looked up in case
lookup_everywhere for creating a file. Unfortunately I see these lookups in
server profile.
Am I correct?
is there any way for this case to reduce the amount of lookups that is send
to the servers? (lookup optimize is off and I want this)

2. I see the option for changing the spread count and fixing the layout in
dht xlaor (dht_setxattr), but when I tried to set this attribute and change
the layout I dont see any change!! while there is no error and crash (I
call setfattr in client side because I dont see dht xlator in server graph)
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel