Re: wip-crush
On 08/22/2012 03:10 AM, Sage Weil wrote: > I pushed a branch that changes some of the crush terminology. Instead of > having a crush type called "pool" that requires you to say things like > "pool=default" in the "ceph osd crush set ..." command, it uses "root" > instead. That hopefully reinforces that it is a tree/hierarchy. > > There is also a patch that changes "bucket" to "node" throughout, since > bucket is a term also used by radosgw. > > Thoughts? I think the main pain in making this transition is that old > clusters have maps that have a type 'pool' and new ones won't, and the > docs will need to walk people through both... "pool" in a crushmap being completely unrelated to a RADOS pool is something that I've heard customers/users report as confusing, as well. So changing that is probably a good thing. Naming it "root" is probably a good choice as well, as it happens to match http://ceph.com/wiki/Custom_data_placement_with_CRUSH. As for changing "bucket" to node... a "node" is normally simply a physical server (at least in HA terminology, which many potential Ceph users will be familiar with), and CRUSH uses "host" for that. So that's another recipe for confusion. How about using something super-generic, like "element" or "item"? Cheers, Florian -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: wip-crush
On Aug 22, 2012, at 10:46 AM, Florian Haas wrote: > On 08/22/2012 03:10 AM, Sage Weil wrote: >> I pushed a branch that changes some of the crush terminology. Instead of >> having a crush type called "pool" that requires you to say things like >> "pool=default" in the "ceph osd crush set ..." command, it uses "root" >> instead. That hopefully reinforces that it is a tree/hierarchy. >> >> There is also a patch that changes "bucket" to "node" throughout, since >> bucket is a term also used by radosgw. >> >> Thoughts? I think the main pain in making this transition is that old >> clusters have maps that have a type 'pool' and new ones won't, and the >> docs will need to walk people through both... > > "pool" in a crushmap being completely unrelated to a RADOS pool is > something that I've heard customers/users report as confusing, as well. > So changing that is probably a good thing. Naming it "root" is probably > a good choice as well, as it happens to match > http://ceph.com/wiki/Custom_data_placement_with_CRUSH. > > As for changing "bucket" to node... a "node" is normally simply a > physical server (at least in HA terminology, which many potential Ceph > users will be familiar with), and CRUSH uses "host" for that. So that's > another recipe for confusion. How about using something super-generic, > like "element" or "item"? > > Cheers, > Florian My guess is that he is trying to use data structure tree nomenclature (root, node, leaf). I agree that node is an overloaded term (as is pool). As for an alternative to bucket which indicates the item is a collection, what about subtree or branch? Scott-- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: wip-crush
On Wed, 22 Aug 2012, Atchley, Scott wrote: > On Aug 22, 2012, at 10:46 AM, Florian Haas wrote: > > > On 08/22/2012 03:10 AM, Sage Weil wrote: > >> I pushed a branch that changes some of the crush terminology. Instead of > >> having a crush type called "pool" that requires you to say things like > >> "pool=default" in the "ceph osd crush set ..." command, it uses "root" > >> instead. That hopefully reinforces that it is a tree/hierarchy. > >> > >> There is also a patch that changes "bucket" to "node" throughout, since > >> bucket is a term also used by radosgw. > >> > >> Thoughts? I think the main pain in making this transition is that old > >> clusters have maps that have a type 'pool' and new ones won't, and the > >> docs will need to walk people through both... > > > > "pool" in a crushmap being completely unrelated to a RADOS pool is > > something that I've heard customers/users report as confusing, as well. > > So changing that is probably a good thing. Naming it "root" is probably > > a good choice as well, as it happens to match > > http://ceph.com/wiki/Custom_data_placement_with_CRUSH. > > > > As for changing "bucket" to node... a "node" is normally simply a > > physical server (at least in HA terminology, which many potential Ceph > > users will be familiar with), and CRUSH uses "host" for that. So that's > > another recipe for confusion. How about using something super-generic, > > like "element" or "item"? > > > > Cheers, > > Florian > > My guess is that he is trying to use data structure tree nomenclature > (root, node, leaf). I agree that node is an overloaded term (as is > pool). Yeah... > As for an alternative to bucket which indicates the item is a > collection, what about subtree or branch? I think fixing the overloading of 'pool' in the default crush map is the biggest pain point. I can live with crush 'buckets' staying the same (esp since that's what the papers and code use pervasively) if we can't come up with a better option. On the pool part, though, the challenge is how to transition. Existing clusters have maps that use 'pool', and new clusters will use 'root' (or whatever). Some options: - document both. this kills much of the benefit of switching, but is probably inevitable since people will be running different versions. - make the upgrade process transparently rename the type. this lets all the tools use the new names. - make the tools silently translate old names to new names. this is kludgey in that it makes the code make assumptions about the names of the data it is working with, but would cover everyone except those who created their own crush maps from scratch. - ? sage -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: wip-crush
On Wed, Aug 22, 2012 at 9:33 AM, Sage Weil wrote: > On Wed, 22 Aug 2012, Atchley, Scott wrote: >> On Aug 22, 2012, at 10:46 AM, Florian Haas wrote: >> >> > On 08/22/2012 03:10 AM, Sage Weil wrote: >> >> I pushed a branch that changes some of the crush terminology. Instead of >> >> having a crush type called "pool" that requires you to say things like >> >> "pool=default" in the "ceph osd crush set ..." command, it uses "root" >> >> instead. That hopefully reinforces that it is a tree/hierarchy. >> >> >> >> There is also a patch that changes "bucket" to "node" throughout, since >> >> bucket is a term also used by radosgw. >> >> >> >> Thoughts? I think the main pain in making this transition is that old >> >> clusters have maps that have a type 'pool' and new ones won't, and the >> >> docs will need to walk people through both... >> > >> > "pool" in a crushmap being completely unrelated to a RADOS pool is >> > something that I've heard customers/users report as confusing, as well. >> > So changing that is probably a good thing. Naming it "root" is probably >> > a good choice as well, as it happens to match >> > http://ceph.com/wiki/Custom_data_placement_with_CRUSH. >> > >> > As for changing "bucket" to node... a "node" is normally simply a >> > physical server (at least in HA terminology, which many potential Ceph >> > users will be familiar with), and CRUSH uses "host" for that. So that's >> > another recipe for confusion. How about using something super-generic, >> > like "element" or "item"? >> > >> > Cheers, >> > Florian >> >> My guess is that he is trying to use data structure tree nomenclature >> (root, node, leaf). I agree that node is an overloaded term (as is >> pool). > > Yeah... > >> As for an alternative to bucket which indicates the item is a >> collection, what about subtree or branch? > > I think fixing the overloading of 'pool' in the default crush map is the > biggest pain point. I can live with crush 'buckets' staying the same (esp > since that's what the papers and code use pervasively) if we can't come up > with a better option. I'm definitely most interested in replacing "pool", and "root" works for that in my mind. RGW buckets live at a sufficiently different level that I think people are unlikely to be confused — and "bucket" is actually a good name for what they are (I'm open to better ones, but I don't think that "node" qualifies). > On the pool part, though, the challenge is how to transition. Existing > clusters have maps that use 'pool', and new clusters will use 'root' (or > whatever). Some options: > > - document both. this kills much of the benefit of switching, but is >probably inevitable since people will be running different versions. > - make the upgrade process transparently rename the type. this lets >all the tools use the new names. > - make the tools silently translate old names to new names. this is >kludgey in that it makes the code make assumptions about the names of >the data it is working with, but would cover everyone except those who >created their own crush maps from scratch. > - ? I would go with option two, and only document the new options — I wouldn't be surprised if the number of people who had changed those was zero. Anybody who has done so can certainly be counted on to pay enough attention that a line note "changed CRUSH names (see here if you customized your map)" would be sufficient, right? -Greg -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: wip-crush
On Wed, 22 Aug 2012, Gregory Farnum wrote: > On Wed, Aug 22, 2012 at 9:33 AM, Sage Weil wrote: > > On Wed, 22 Aug 2012, Atchley, Scott wrote: > >> On Aug 22, 2012, at 10:46 AM, Florian Haas wrote: > >> > >> > On 08/22/2012 03:10 AM, Sage Weil wrote: > >> >> I pushed a branch that changes some of the crush terminology. Instead > >> >> of > >> >> having a crush type called "pool" that requires you to say things like > >> >> "pool=default" in the "ceph osd crush set ..." command, it uses "root" > >> >> instead. That hopefully reinforces that it is a tree/hierarchy. > >> >> > >> >> There is also a patch that changes "bucket" to "node" throughout, since > >> >> bucket is a term also used by radosgw. > >> >> > >> >> Thoughts? I think the main pain in making this transition is that old > >> >> clusters have maps that have a type 'pool' and new ones won't, and the > >> >> docs will need to walk people through both... > >> > > >> > "pool" in a crushmap being completely unrelated to a RADOS pool is > >> > something that I've heard customers/users report as confusing, as well. > >> > So changing that is probably a good thing. Naming it "root" is probably > >> > a good choice as well, as it happens to match > >> > http://ceph.com/wiki/Custom_data_placement_with_CRUSH. > >> > > >> > As for changing "bucket" to node... a "node" is normally simply a > >> > physical server (at least in HA terminology, which many potential Ceph > >> > users will be familiar with), and CRUSH uses "host" for that. So that's > >> > another recipe for confusion. How about using something super-generic, > >> > like "element" or "item"? > >> > > >> > Cheers, > >> > Florian > >> > >> My guess is that he is trying to use data structure tree nomenclature > >> (root, node, leaf). I agree that node is an overloaded term (as is > >> pool). > > > > Yeah... > > > >> As for an alternative to bucket which indicates the item is a > >> collection, what about subtree or branch? > > > > I think fixing the overloading of 'pool' in the default crush map is the > > biggest pain point. I can live with crush 'buckets' staying the same (esp > > since that's what the papers and code use pervasively) if we can't come up > > with a better option. > > I'm definitely most interested in replacing "pool", and "root" works > for that in my mind. RGW buckets live at a sufficiently different > level that I think people are unlikely to be confused ? and "bucket" > is actually a good name for what they are (I'm open to better ones, > but I don't think that "node" qualifies). Yeah, sounds good to me. > > On the pool part, though, the challenge is how to transition. Existing > > clusters have maps that use 'pool', and new clusters will use 'root' (or > > whatever). Some options: > > > > - document both. this kills much of the benefit of switching, but is > >probably inevitable since people will be running different versions. > > - make the upgrade process transparently rename the type. this lets > >all the tools use the new names. > > - make the tools silently translate old names to new names. this is > >kludgey in that it makes the code make assumptions about the names of > >the data it is working with, but would cover everyone except those who > >created their own crush maps from scratch. > > - ? > > I would go with option two, and only document the new options ? I > wouldn't be surprised if the number of people who had changed those > was zero. Anybody who has done so can certainly be counted on to pay > enough attention that a line note "changed CRUSH names (see here if > you customized your map)" would be sufficient, right? Yeah. The one wrinkle is that people running old code (e.g., argonaut) reading the latest docs will see commands that don't quite work. At some point we need to fork the docs for each stable release... maybe now is the time to do that. sage
Re: wip-crush
Okay, it's trivial to change 'pool' to 'root' in the default generated crush map, and update all the docs accordingly. The problem is that some stuff built on top of ceph has 'pool=default' in there, including our chef cookbooks and those that other people are building. I see a few options: 1) - Document that pool=default should be used for clusters created pre-0.52, and root=default for clusters created after that (at http://ceph.com/docs/...). - Tell new tools to use root=default - Encourage existing tools to use *both* root=default pool=default (if the type is not present, it will be ignored). They will break on newly created clusters with new code, but it will at least happen quickly (on cluster creation). - Make noise in the release notes+announcement. 2) Make 'pool' be silently translated to 'root' (and the other way around) if the type is not present. That way new tools with old clusters will work, and new clusters with old automation will work. Other ideas? I don't much like #2. sage -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: wip-crush
On Fri, Aug 31, 2012 at 1:36 PM, Sage Weil wrote: > Okay, it's trivial to change 'pool' to 'root' in the default generated > crush map, and update all the docs accordingly. The problem is that some > stuff built on top of ceph has 'pool=default' in there, including our chef > cookbooks and those that other people are building. That's actually in ceph.git src/upstart/ceph-osd.conf -- it can change exactly when the code changes! -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: wip-crush
On Fri, 31 Aug 2012, Sage Weil wrote: > Okay, it's trivial to change 'pool' to 'root' in the default generated > crush map, and update all the docs accordingly. The problem is that some > stuff built on top of ceph has 'pool=default' in there, including our chef > cookbooks and those that other people are building. > > I see a few options: > > 1) - Document that pool=default should be used for clusters created > pre-0.52, and root=default for clusters created after that (at > http://ceph.com/docs/...). >- Tell new tools to use root=default >- Encourage existing tools to use *both* root=default pool=default (if > the type is not present, it will be ignored). They will break on > newly created clusters with new code, but it will at least happen > quickly (on cluster creation). >- Make noise in the release notes+announcement. Okay, #1 it is! I'll make some more noise when v0.52 is released. Merging this into master. sage > > 2) Make 'pool' be silently translated to 'root' (and the other way around) >if the type is not present. That way new tools with old clusters will >work, and new clusters with old automation will work. > > Other ideas? I don't much like #2. > > sage > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
