Re: [Gluster-devel] Persistent AFR changelog attributes
On 02/12/2014 12:15 PM, Pranith Kumar Karampuri wrote: - Original Message - From: "Pranith Kumar Karampuri" To: "Anand Avati" Cc: "Ravishankar N" , "Gluster Devel" Sent: Wednesday, February 12, 2014 11:44:08 AM Subject: Re: [Gluster-devel] Persistent AFR changelog attributes - Original Message - From: "Pranith Kumar Karampuri" To: "Anand Avati" Cc: "Ravishankar N" , "Gluster Devel" Sent: Wednesday, February 12, 2014 11:16:57 AM Subject: Re: [Gluster-devel] Persistent AFR changelog attributes - Original Message - From: "Anand Avati" To: "Ravishankar N" Cc: "Gluster Devel" Sent: Wednesday, February 12, 2014 10:30:29 AM Subject: Re: [Gluster-devel] Persistent AFR changelog attributes Ravi, We had earlier discussed a solution for this (sometime last year) by making volgen remember xlator names and not reassign them. Copying KP with who I had discussed this to quite a level of detail. Have you guys spoken about this? IIRC the solution KP and I discussed was more generic and could also support retaining user made changes/customizations to the volfiles. How will new machines that are joining the cluster will know about this modified graph? One way to achieve it is to send the skeleton structure of the graph to other machines. I wonder how this will address snapshot volumes' graph. May be even in the case of snapshot volumes, the skeleton has to be copied over to snapshot volume. So instead of storing just the client xlator names, the generic solution will have to store the entire skeleton and should keep it in sync at the time of probe, snapshot. Is that the rough algo you guys discussed? Did I miss anything? If this indeed is the approach, I don't see an easy way out for generating brick volfiles according to versions. Brick processes don't have graph switching capability yet. Because of this, new versions may need to generate new xlators (Ex: barrier xlator that is going to come in for 3.6) in new versions but not for old versions. In essence the skeleton that is stored for one version could be incompatible for other versions. Then we will have to start having some sort of conversion mechanisms of the skeletons from one version to other. It seems a bit complicated :-(. Any suggestions? After speaking to avati, he said KP knows more about this design. CC kp for inputs. Pranith Hi Avati, After discussing with KP and others, we feel it is better to go with the current approach (of persisting only client xlator names) for glusterFS 3.6, solving the immediate problem of self-heals and snapshot support while not having to do much change for backward compatibility. The more generic solution of reading the volfile and modifying the graph in-place (for all xlator names) could be done for glusterFS 4 . Does this sound okay? Thanks, Ravi Pranith Thanks, Avati On Tue, Feb 11, 2014 at 8:33 PM, Ravishankar N < ravishan...@redhat.com > wrote: Hello, As you might perhaps be aware, AFR translator currently uses the client translator names as the name of it's changelog extended attributes. i)This can be a problem when a remove-brick operation is performed when there are pending heals happening because remove-brick causes a graph change wherein the client translator names become different. ii) Also, for gluster snapshot volumes to work correctly, there needs to be a persistent mapping of AFR changelog attributes to the bricks. After some internal discussions, we have come up with a new naming mechanism that ensures backward compatibility. Details on the aforementioned problems and the proposed solution are detailed in a feature page [1] for GlusterFS 3.6. Please feel free to go through it and give your comments/ critiques. Thanks and regards, Ravi [1] https://www.gluster.org/ community/documentation/index. php/Features/persistent-AFR- changelog-xattributes __ _ Gluster-devel mailing list Gluster-devel@nongnu.org https://lists.nongnu.org/ mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Persistent AFR changelog attributes
CCed now :-) Pranith - Original Message - > From: "Pranith Kumar Karampuri" > To: "Anand Avati" > Cc: "Gluster Devel" > Sent: Wednesday, February 12, 2014 12:15:13 PM > Subject: Re: [Gluster-devel] Persistent AFR changelog attributes > > > > - Original Message - > > From: "Pranith Kumar Karampuri" > > To: "Anand Avati" > > Cc: "Ravishankar N" , "Gluster Devel" > > > > Sent: Wednesday, February 12, 2014 11:44:08 AM > > Subject: Re: [Gluster-devel] Persistent AFR changelog attributes > > > > > > > > - Original Message - > > > From: "Pranith Kumar Karampuri" > > > To: "Anand Avati" > > > Cc: "Ravishankar N" , "Gluster Devel" > > > > > > Sent: Wednesday, February 12, 2014 11:16:57 AM > > > Subject: Re: [Gluster-devel] Persistent AFR changelog attributes > > > > > > > > > > > > - Original Message - > > > > From: "Anand Avati" > > > > To: "Ravishankar N" > > > > Cc: "Gluster Devel" > > > > Sent: Wednesday, February 12, 2014 10:30:29 AM > > > > Subject: Re: [Gluster-devel] Persistent AFR changelog attributes > > > > > > > > Ravi, > > > > We had earlier discussed a solution for this (sometime last year) by > > > > making > > > > volgen remember xlator names and not reassign them. Copying KP with who > > > > I > > > > had discussed this to quite a level of detail. Have you guys spoken > > > > about > > > > this? IIRC the solution KP and I discussed was more generic and could > > > > also > > > > support retaining user made changes/customizations to the volfiles. > > > > > > How will new machines that are joining the cluster will know about this > > > modified graph? One way to achieve it is to send the skeleton structure > > > of > > > the graph to other machines. I wonder how this will address snapshot > > > volumes' graph. May be even in the case of snapshot volumes, the skeleton > > > has to be copied over to snapshot volume. So instead of storing just the > > > client xlator names, the generic solution will have to store the entire > > > skeleton and should keep it in sync at the time of probe, snapshot. Is > > > that > > > the rough algo you guys discussed? Did I miss anything? > > > > If this indeed is the approach, I don't see an easy way out for generating > > brick volfiles according to versions. Brick processes don't have graph > > switching capability yet. Because of this, new versions may need to > > generate > > new xlators (Ex: barrier xlator that is going to come in for 3.6) in new > > versions but not for old versions. In essence the skeleton that is stored > > for one version could be incompatible for other versions. Then we will have > > to start having some sort of conversion mechanisms of the skeletons from > > one > > version to other. It seems a bit complicated :-(. Any suggestions? > > After speaking to avati, he said KP knows more about this design. CC kp for > inputs. > > Pranith > > > > > > > > Pranith > > > > > > > > > > > Thanks, > > > > Avati > > > > > > > > > > > > On Tue, Feb 11, 2014 at 8:33 PM, Ravishankar N < ravishan...@redhat.com > > > > > > > > > wrote: > > > > > > > > > > > > Hello, > > > > > > > > As you might perhaps be aware, AFR translator currently uses the client > > > > translator names as the name of it's changelog extended attributes. > > > > > > > > i)This can be a problem when a remove-brick operation is performed when > > > > there > > > > are pending heals happening because remove-brick causes a graph change > > > > wherein the client translator names become different. > > > > > > > > ii) Also, for gluster snapshot volumes to work correctly, there needs > > > > to > > > > be > > > > a > > > > persistent mapping of AFR changelog attributes to the bricks. > > > > > > > > After some internal discussions, we have come up with a new naming > > > > mechanism > > > > that ensures backward compatibility. Details on the afore
Re: [Gluster-devel] Persistent AFR changelog attributes
- Original Message - > From: "Pranith Kumar Karampuri" > To: "Anand Avati" > Cc: "Ravishankar N" , "Gluster Devel" > > Sent: Wednesday, February 12, 2014 11:44:08 AM > Subject: Re: [Gluster-devel] Persistent AFR changelog attributes > > > > - Original Message - > > From: "Pranith Kumar Karampuri" > > To: "Anand Avati" > > Cc: "Ravishankar N" , "Gluster Devel" > > > > Sent: Wednesday, February 12, 2014 11:16:57 AM > > Subject: Re: [Gluster-devel] Persistent AFR changelog attributes > > > > > > > > - Original Message ----- > > > From: "Anand Avati" > > > To: "Ravishankar N" > > > Cc: "Gluster Devel" > > > Sent: Wednesday, February 12, 2014 10:30:29 AM > > > Subject: Re: [Gluster-devel] Persistent AFR changelog attributes > > > > > > Ravi, > > > We had earlier discussed a solution for this (sometime last year) by > > > making > > > volgen remember xlator names and not reassign them. Copying KP with who I > > > had discussed this to quite a level of detail. Have you guys spoken about > > > this? IIRC the solution KP and I discussed was more generic and could > > > also > > > support retaining user made changes/customizations to the volfiles. > > > > How will new machines that are joining the cluster will know about this > > modified graph? One way to achieve it is to send the skeleton structure of > > the graph to other machines. I wonder how this will address snapshot > > volumes' graph. May be even in the case of snapshot volumes, the skeleton > > has to be copied over to snapshot volume. So instead of storing just the > > client xlator names, the generic solution will have to store the entire > > skeleton and should keep it in sync at the time of probe, snapshot. Is that > > the rough algo you guys discussed? Did I miss anything? > > If this indeed is the approach, I don't see an easy way out for generating > brick volfiles according to versions. Brick processes don't have graph > switching capability yet. Because of this, new versions may need to generate > new xlators (Ex: barrier xlator that is going to come in for 3.6) in new > versions but not for old versions. In essence the skeleton that is stored > for one version could be incompatible for other versions. Then we will have > to start having some sort of conversion mechanisms of the skeletons from one > version to other. It seems a bit complicated :-(. Any suggestions? After speaking to avati, he said KP knows more about this design. CC kp for inputs. Pranith > > > > > Pranith > > > > > > > > Thanks, > > > Avati > > > > > > > > > On Tue, Feb 11, 2014 at 8:33 PM, Ravishankar N < ravishan...@redhat.com > > > > wrote: > > > > > > > > > Hello, > > > > > > As you might perhaps be aware, AFR translator currently uses the client > > > translator names as the name of it's changelog extended attributes. > > > > > > i)This can be a problem when a remove-brick operation is performed when > > > there > > > are pending heals happening because remove-brick causes a graph change > > > wherein the client translator names become different. > > > > > > ii) Also, for gluster snapshot volumes to work correctly, there needs to > > > be > > > a > > > persistent mapping of AFR changelog attributes to the bricks. > > > > > > After some internal discussions, we have come up with a new naming > > > mechanism > > > that ensures backward compatibility. Details on the aforementioned > > > problems > > > and the proposed solution are detailed in a feature page [1] for > > > GlusterFS > > > 3.6. > > > Please feel free to go through it and give your comments/ critiques. > > > > > > Thanks and regards, > > > Ravi > > > > > > [1] https://www.gluster.org/ community/documentation/index. > > > php/Features/persistent-AFR- changelog-xattributes > > > > > > __ _ > > > Gluster-devel mailing list > > > Gluster-devel@nongnu.org > > > https://lists.nongnu.org/ mailman/listinfo/gluster-devel > > > > > > > > > ___ > > > Gluster-devel mailing list > > > Gluster-devel@nongnu.org > > > https://lists.nongnu.org/mailman/listinfo/gluster-devel > > > > > > ___ Gluster-devel mailing list Gluster-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Persistent AFR changelog attributes
- Original Message - > From: "Pranith Kumar Karampuri" > To: "Anand Avati" > Cc: "Ravishankar N" , "Gluster Devel" > > Sent: Wednesday, February 12, 2014 11:16:57 AM > Subject: Re: [Gluster-devel] Persistent AFR changelog attributes > > > > - Original Message - > > From: "Anand Avati" > > To: "Ravishankar N" > > Cc: "Gluster Devel" > > Sent: Wednesday, February 12, 2014 10:30:29 AM > > Subject: Re: [Gluster-devel] Persistent AFR changelog attributes > > > > Ravi, > > We had earlier discussed a solution for this (sometime last year) by making > > volgen remember xlator names and not reassign them. Copying KP with who I > > had discussed this to quite a level of detail. Have you guys spoken about > > this? IIRC the solution KP and I discussed was more generic and could also > > support retaining user made changes/customizations to the volfiles. > > How will new machines that are joining the cluster will know about this > modified graph? One way to achieve it is to send the skeleton structure of > the graph to other machines. I wonder how this will address snapshot > volumes' graph. May be even in the case of snapshot volumes, the skeleton > has to be copied over to snapshot volume. So instead of storing just the > client xlator names, the generic solution will have to store the entire > skeleton and should keep it in sync at the time of probe, snapshot. Is that > the rough algo you guys discussed? Did I miss anything? If this indeed is the approach, I don't see an easy way out for generating brick volfiles according to versions. Brick processes don't have graph switching capability yet. Because of this, new versions may need to generate new xlators (Ex: barrier xlator that is going to come in for 3.6) in new versions but not for old versions. In essence the skeleton that is stored for one version could be incompatible for other versions. Then we will have to start having some sort of conversion mechanisms of the skeletons from one version to other. It seems a bit complicated :-(. Any suggestions? > > Pranith > > > > > Thanks, > > Avati > > > > > > On Tue, Feb 11, 2014 at 8:33 PM, Ravishankar N < ravishan...@redhat.com > > > wrote: > > > > > > Hello, > > > > As you might perhaps be aware, AFR translator currently uses the client > > translator names as the name of it's changelog extended attributes. > > > > i)This can be a problem when a remove-brick operation is performed when > > there > > are pending heals happening because remove-brick causes a graph change > > wherein the client translator names become different. > > > > ii) Also, for gluster snapshot volumes to work correctly, there needs to be > > a > > persistent mapping of AFR changelog attributes to the bricks. > > > > After some internal discussions, we have come up with a new naming > > mechanism > > that ensures backward compatibility. Details on the aforementioned problems > > and the proposed solution are detailed in a feature page [1] for GlusterFS > > 3.6. > > Please feel free to go through it and give your comments/ critiques. > > > > Thanks and regards, > > Ravi > > > > [1] https://www.gluster.org/ community/documentation/index. > > php/Features/persistent-AFR- changelog-xattributes > > > > __ _ > > Gluster-devel mailing list > > Gluster-devel@nongnu.org > > https://lists.nongnu.org/ mailman/listinfo/gluster-devel > > > > > > ___ > > Gluster-devel mailing list > > Gluster-devel@nongnu.org > > https://lists.nongnu.org/mailman/listinfo/gluster-devel > > > ___ Gluster-devel mailing list Gluster-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Persistent AFR changelog attributes
- Original Message - > From: "Anand Avati" > To: "Ravishankar N" > Cc: "Gluster Devel" > Sent: Wednesday, February 12, 2014 10:30:29 AM > Subject: Re: [Gluster-devel] Persistent AFR changelog attributes > > Ravi, > We had earlier discussed a solution for this (sometime last year) by making > volgen remember xlator names and not reassign them. Copying KP with who I > had discussed this to quite a level of detail. Have you guys spoken about > this? IIRC the solution KP and I discussed was more generic and could also > support retaining user made changes/customizations to the volfiles. How will new machines that are joining the cluster will know about this modified graph? One way to achieve it is to send the skeleton structure of the graph to other machines. I wonder how this will address snapshot volumes' graph. May be even in the case of snapshot volumes, the skeleton has to be copied over to snapshot volume. So instead of storing just the client xlator names, the generic solution will have to store the entire skeleton and should keep it in sync at the time of probe, snapshot. Is that the rough algo you guys discussed? Did I miss anything? Pranith > > Thanks, > Avati > > > On Tue, Feb 11, 2014 at 8:33 PM, Ravishankar N < ravishan...@redhat.com > > wrote: > > > Hello, > > As you might perhaps be aware, AFR translator currently uses the client > translator names as the name of it's changelog extended attributes. > > i)This can be a problem when a remove-brick operation is performed when there > are pending heals happening because remove-brick causes a graph change > wherein the client translator names become different. > > ii) Also, for gluster snapshot volumes to work correctly, there needs to be a > persistent mapping of AFR changelog attributes to the bricks. > > After some internal discussions, we have come up with a new naming mechanism > that ensures backward compatibility. Details on the aforementioned problems > and the proposed solution are detailed in a feature page [1] for GlusterFS > 3.6. > Please feel free to go through it and give your comments/ critiques. > > Thanks and regards, > Ravi > > [1] https://www.gluster.org/ community/documentation/index. > php/Features/persistent-AFR- changelog-xattributes > > __ _ > Gluster-devel mailing list > Gluster-devel@nongnu.org > https://lists.nongnu.org/ mailman/listinfo/gluster-devel > > > ___ > Gluster-devel mailing list > Gluster-devel@nongnu.org > https://lists.nongnu.org/mailman/listinfo/gluster-devel > ___ Gluster-devel mailing list Gluster-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Persistent AFR changelog attributes
Ravi, We had earlier discussed a solution for this (sometime last year) by making volgen remember xlator names and not reassign them. Copying KP with who I had discussed this to quite a level of detail. Have you guys spoken about this? IIRC the solution KP and I discussed was more generic and could also support retaining user made changes/customizations to the volfiles. Thanks, Avati On Tue, Feb 11, 2014 at 8:33 PM, Ravishankar N wrote: > Hello, > > As you might perhaps be aware, AFR translator currently uses the client > translator names as the name of it's changelog extended attributes. > > i)This can be a problem when a remove-brick operation is performed when > there are pending heals happening because remove-brick causes a graph > change wherein the client translator names become different. > > ii) Also, for gluster snapshot volumes to work correctly, there needs to > be a persistent mapping of AFR changelog attributes to the bricks. > > After some internal discussions, we have come up with a new naming > mechanism that ensures backward compatibility. Details on the > aforementioned problems and the proposed solution are detailed in a feature > page [1] for GlusterFS 3.6. > Please feel free to go through it and give your comments/ critiques. > > Thanks and regards, > Ravi > > [1] https://www.gluster.org/community/documentation/index. > php/Features/persistent-AFR-changelog-xattributes > > ___ > Gluster-devel mailing list > Gluster-devel@nongnu.org > https://lists.nongnu.org/mailman/listinfo/gluster-devel > ___ Gluster-devel mailing list Gluster-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/gluster-devel
