Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
Lets use https://etherpad.openstack.org/p/Icehouse-nova-oslo-sync to keep track of things. On Wed, Feb 26, 2014 at 5:10 PM, Joe Gordon wrote: > GCB, Ben, > > Thanks for volunteering to help. > > GCB, reminded me that we should be doing this for python-novaclient in > addition to nova itself. So that being said, as I see it here are the > steps moving forward: > > Note: as previously mentionedin this thread there already is a team > working on syncing oslo.db so we can ignore that for now (and once its > ready they will propose patches, so we just have to do reviews). > > 1) Review all outstanding nova/python-novaclient sync patches. > https://review.openstack.org/#/c/72596/ > https://review.openstack.org/#/c/74560/ > https://review.openstack.org/#/c/75644/ > 2) Using update.sh sync all low hanging fruit in both repos all at > once. Low hanging fruit is anything that doesn't require a change > outside of */openstack/common. As usual when doing these syncs we > should list all patches being synced across, as well as document which > modules we aren't syncing accross >./update.sh --base novaclient --config-file > ../python-novaclient/openstack-common.conf --dest-dir > ../python-novaclient/ > https://review.openstack.org/#/c/76642/ > ./update.sh --base nova --config-file ../nova/openstack-common.conf > --dest-dir ../nova/ > 3) At this point we should have a list of modules that are non-trivial > to sync, now we can triage them and decide if they are oslo bugs or if > nova/python-novaclient code needs updating. > > > So for now we need reviews on the patches listed in 1, and someone to > work on the low hanging fruit sync for nova. Followed by triaging of > the non-low hanging fruit. > > Once we have the low hanging fruit out of the way lets sync up about > how to handle the rest. > > best, > Joe > > > On Fri, Feb 21, 2014 at 6:26 PM, ChangBo Guo wrote: >> >> >> >> 2014-02-22 5:09 GMT+08:00 Ben Nemec : >> >>> /me finally catches up on -dev list traffic... >>> >>> On 2014-02-19 20:27, Doug Hellmann wrote: >>> >>> >>> >>> >>> On Wed, Feb 19, 2014 at 8:13 PM, Joe Gordon wrote: Hi All, As many of you know most oslo-incubator code is wildly out of sync. Assuming we consider it a good idea to sync up oslo-incubator code before cutting Icehouse, then we have a problem. Today oslo-incubator code is synced in ad-hoc manor, resulting in duplicated efforts and wildly out of date code. Part of the challenges today are backwards incompatible changes and new oslo bugs. I expect that once we get a single project to have an up to date oslo-incubator copy it will make syncing a second project significantly easier. So because I (hopefully) have some karma built up in nova, I would like to volunteer nova to be the guinea pig. >>> >>> >>> Thank you for volunteering to spear-head this, Joe. >>> >>> +1 To fix this I would like to propose starting an oslo-incubator/nova sync team. They would be responsible for getting nova's oslo code up to date. I expect this work to involve: * Reviewing lots of oslo sync patches * Tracking the current sync patches * Syncing over the low hanging fruit, modules that work without changing nova. * Reporting bugs to oslo team * Working with oslo team to figure out how to deal with backwards incompatible changes * Update nova code or make oslo module backwards compatible * Track all this * Create a roadmap for other projects to follow (re: documentation) I am looking for volunteers to help with this effort, any takers? >>> >>> >>> I will help, especially with reviews and tracking. >>> >>> I'm happy to help as well. I always try to help with oslo syncs any time >>> I'm made aware of problems anyway. >>> >>> What is our first step here? Get the low-hanging fruit syncs proposed all >>> at once? Do them individually (taking into consideration the module deps, >>> of course)? If we're going to try to get this done for Icehouse then we >>> probably need to start ASAP. >>> >>> -Ben >> >> I also would like to be volunteer of the new team :) >> -gcb >> >> >> -- >> ChangBo Guo(gcb) >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
GCB, Ben, Thanks for volunteering to help. GCB, reminded me that we should be doing this for python-novaclient in addition to nova itself. So that being said, as I see it here are the steps moving forward: Note: as previously mentionedin this thread there already is a team working on syncing oslo.db so we can ignore that for now (and once its ready they will propose patches, so we just have to do reviews). 1) Review all outstanding nova/python-novaclient sync patches. https://review.openstack.org/#/c/72596/ https://review.openstack.org/#/c/74560/ https://review.openstack.org/#/c/75644/ 2) Using update.sh sync all low hanging fruit in both repos all at once. Low hanging fruit is anything that doesn't require a change outside of */openstack/common. As usual when doing these syncs we should list all patches being synced across, as well as document which modules we aren't syncing accross ./update.sh --base novaclient --config-file ../python-novaclient/openstack-common.conf --dest-dir ../python-novaclient/ https://review.openstack.org/#/c/76642/ ./update.sh --base nova --config-file ../nova/openstack-common.conf --dest-dir ../nova/ 3) At this point we should have a list of modules that are non-trivial to sync, now we can triage them and decide if they are oslo bugs or if nova/python-novaclient code needs updating. So for now we need reviews on the patches listed in 1, and someone to work on the low hanging fruit sync for nova. Followed by triaging of the non-low hanging fruit. Once we have the low hanging fruit out of the way lets sync up about how to handle the rest. best, Joe On Fri, Feb 21, 2014 at 6:26 PM, ChangBo Guo wrote: > > > > 2014-02-22 5:09 GMT+08:00 Ben Nemec : > >> /me finally catches up on -dev list traffic... >> >> On 2014-02-19 20:27, Doug Hellmann wrote: >> >> >> >> >> On Wed, Feb 19, 2014 at 8:13 PM, Joe Gordon wrote: >>> >>> Hi All, >>> >>> As many of you know most oslo-incubator code is wildly out of sync. >>> Assuming we consider it a good idea to sync up oslo-incubator code >>> before cutting Icehouse, then we have a problem. >>> >>> Today oslo-incubator code is synced in ad-hoc manor, resulting in >>> duplicated efforts and wildly out of date code. Part of the challenges >>> today are backwards incompatible changes and new oslo bugs. I expect >>> that once we get a single project to have an up to date oslo-incubator >>> copy it will make syncing a second project significantly easier. So >>> because I (hopefully) have some karma built up in nova, I would like >>> to volunteer nova to be the guinea pig. >> >> >> Thank you for volunteering to spear-head this, Joe. >> >> +1 >>> >>> To fix this I would like to propose starting an oslo-incubator/nova >>> sync team. They would be responsible for getting nova's oslo code up >>> to date. I expect this work to involve: >>> * Reviewing lots of oslo sync patches >>> * Tracking the current sync patches >>> * Syncing over the low hanging fruit, modules that work without changing >>> nova. >>> * Reporting bugs to oslo team >>> * Working with oslo team to figure out how to deal with backwards >>> incompatible changes >>> * Update nova code or make oslo module backwards compatible >>> * Track all this >>> * Create a roadmap for other projects to follow (re: documentation) >>> >>> I am looking for volunteers to help with this effort, any takers? >> >> >> I will help, especially with reviews and tracking. >> >> I'm happy to help as well. I always try to help with oslo syncs any time >> I'm made aware of problems anyway. >> >> What is our first step here? Get the low-hanging fruit syncs proposed all >> at once? Do them individually (taking into consideration the module deps, >> of course)? If we're going to try to get this done for Icehouse then we >> probably need to start ASAP. >> >> -Ben > > I also would like to be volunteer of the new team :) > -gcb > > > -- > ChangBo Guo(gcb) > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
2014-02-22 5:09 GMT+08:00 Ben Nemec : > /me finally catches up on -dev list traffic... > > On 2014-02-19 20:27, Doug Hellmann wrote: > > > > > On Wed, Feb 19, 2014 at 8:13 PM, Joe Gordon wrote: > >> Hi All, >> >> As many of you know most oslo-incubator code is wildly out of sync. >> Assuming we consider it a good idea to sync up oslo-incubator code >> before cutting Icehouse, then we have a problem. >> >> Today oslo-incubator code is synced in ad-hoc manor, resulting in >> duplicated efforts and wildly out of date code. Part of the challenges >> today are backwards incompatible changes and new oslo bugs. I expect >> that once we get a single project to have an up to date oslo-incubator >> copy it will make syncing a second project significantly easier. So >> because I (hopefully) have some karma built up in nova, I would like >> to volunteer nova to be the guinea pig. > > > Thank you for volunteering to spear-head this, Joe. > > +1 > > To fix this I would like to propose starting an oslo-incubator/nova >> sync team. They would be responsible for getting nova's oslo code up >> to date. I expect this work to involve: >> * Reviewing lots of oslo sync patches >> * Tracking the current sync patches >> * Syncing over the low hanging fruit, modules that work without changing >> nova. >> * Reporting bugs to oslo team >> * Working with oslo team to figure out how to deal with backwards >> incompatible changes >> * Update nova code or make oslo module backwards compatible >> * Track all this >> * Create a roadmap for other projects to follow (re: documentation) >> >> I am looking for volunteers to help with this effort, any takers? > > > I will help, especially with reviews and tracking. > > I'm happy to help as well. I always try to help with oslo syncs any > time I'm made aware of problems anyway. > > What is our first step here? Get the low-hanging fruit syncs proposed all > at once? Do them individually (taking into consideration the module deps, > of course)? If we're going to try to get this done for Icehouse then we > probably need to start ASAP. > > -Ben > I also would like to be volunteer of the new team :) -gcb > -- ChangBo Guo(gcb) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
/me finally catches up on -dev list traffic... On 2014-02-19 20:27, Doug Hellmann wrote: > On Wed, Feb 19, 2014 at 8:13 PM, Joe Gordon wrote: > >> Hi All, >> >> As many of you know most oslo-incubator code is wildly out of sync. >> Assuming we consider it a good idea to sync up oslo-incubator code >> before cutting Icehouse, then we have a problem. >> >> Today oslo-incubator code is synced in ad-hoc manor, resulting in >> duplicated efforts and wildly out of date code. Part of the challenges >> today are backwards incompatible changes and new oslo bugs. I expect >> that once we get a single project to have an up to date oslo-incubator >> copy it will make syncing a second project significantly easier. So >> because I (hopefully) have some karma built up in nova, I would like >> to volunteer nova to be the guinea pig. > > Thank you for volunteering to spear-head this, Joe. +1 >> To fix this I would like to propose starting an oslo-incubator/nova >> sync team. They would be responsible for getting nova's oslo code up >> to date. I expect this work to involve: >> * Reviewing lots of oslo sync patches >> * Tracking the current sync patches >> * Syncing over the low hanging fruit, modules that work without changing >> nova. >> * Reporting bugs to oslo team >> * Working with oslo team to figure out how to deal with backwards >> incompatible changes >> * Update nova code or make oslo module backwards compatible >> * Track all this >> * Create a roadmap for other projects to follow (re: documentation) >> >> I am looking for volunteers to help with this effort, any takers? > > I will help, especially with reviews and tracking. I'm happy to help as well. I always try to help with oslo syncs any time I'm made aware of problems anyway. What is our first step here? Get the low-hanging fruit syncs proposed all at once? Do them individually (taking into consideration the module deps, of course)? If we're going to try to get this done for Icehouse then we probably need to start ASAP. -Ben > We are going to want someone from the team working on the db modules to > participate as well, since we know that's one area where the API has diverged > some (although we did take backwards compatibility into account). Victor, can > you help find us a volunteer? > > Doug > >> best, >> Joe Gordon >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev [1] > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > [1] Links: -- [1] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
Matt Riedemann wrote:
> This is kind of an aside, but I'm kind of confused now about how the
> syncs work with things that fall under oslo.rootwrap or oslo.messaging,
> like this patch [2]. It doesn't completely match the o-i patch, i.e.
> it's not syncing over openstack/common/rootwrap/wrapper.py, and I'm
> assuming because that's in oslo.rootwrap now? But then why does the
> code still exist in oslo-incubator?
FWIW the code was recently removed from the oslo-incubator, once Neutron
(the last of the rootwrap-consuming projects) got migrated to using
oslo.rootwrap.
> [2] https://review.openstack.org/#/c/73340/
This one syncs changes from https://review.openstack.org/#/c/63094
63094 should never have been approved, since rootwrap in oslo-incubator
was frozen ("graduating"). Now the changes are lost, since they were
never proposed to oslo.rootwrap, and the code in the incubator was
cleaned up.
I'll comment on the 73340 review to try to solve this mess.
--
Thierry Carrez (ttx)
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
Hello All I and Roman Podoliaka are familiar with the changes made to common db code, so we are ready to help with syncing it to OS projects. But we want to ask you for more activity in reviewing of these patches. Thanks, Victor On Thu, Feb 20, 2014 at 4:27 AM, Doug Hellmann wrote: > > > > On Wed, Feb 19, 2014 at 8:13 PM, Joe Gordon wrote: > >> Hi All, >> >> As many of you know most oslo-incubator code is wildly out of sync. >> Assuming we consider it a good idea to sync up oslo-incubator code >> before cutting Icehouse, then we have a problem. >> >> Today oslo-incubator code is synced in ad-hoc manor, resulting in >> duplicated efforts and wildly out of date code. Part of the challenges >> today are backwards incompatible changes and new oslo bugs. I expect >> that once we get a single project to have an up to date oslo-incubator >> copy it will make syncing a second project significantly easier. So >> because I (hopefully) have some karma built up in nova, I would like >> to volunteer nova to be the guinea pig. >> > > Thank you for volunteering to spear-head this, Joe. > > >> To fix this I would like to propose starting an oslo-incubator/nova >> sync team. They would be responsible for getting nova's oslo code up >> to date. I expect this work to involve: >> * Reviewing lots of oslo sync patches >> * Tracking the current sync patches >> * Syncing over the low hanging fruit, modules that work without changing >> nova. >> * Reporting bugs to oslo team >> * Working with oslo team to figure out how to deal with backwards >> incompatible changes >> * Update nova code or make oslo module backwards compatible >> * Track all this >> * Create a roadmap for other projects to follow (re: documentation) >> >> I am looking for volunteers to help with this effort, any takers? >> > > I will help, especially with reviews and tracking. > > We are going to want someone from the team working on the db modules to > participate as well, since we know that's one area where the API has > diverged some (although we did take backwards compatibility into account). > Victor, can you help find us a volunteer? > > Doug > > > >> >> >> best, >> Joe Gordon >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
Hi all, I'm ready to help with syncing of DB code. But we'll need reviewers attention in both oslo-incubator in nova :) Thanks, Roman On Thu, Feb 20, 2014 at 5:37 AM, Lance D Bragstad wrote: > Shed a little bit of light on Matt's comment about Keystone removing > oslo-incubator code and the issues we hit. Comments below. > > > Best Regards, > > Lance Bragstad > ldbra...@us.ibm.com > > Doug Hellmann wrote on 02/19/2014 09:00:29 PM: > >> From: Doug Hellmann >> To: "OpenStack Development Mailing List (not for usage questions)" >> , >> Date: 02/19/2014 09:12 PM >> Subject: Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator >> sync workflow > > >> >> > >> On Wed, Feb 19, 2014 at 9:52 PM, Joe Gordon wrote: >> As a side to this, as an exercise I tried a oslo sync in cinder to see >> what kind of issues would arise and here are my findings so far: >> https://review.openstack.org/#/c/74786/ >> >> On Wed, Feb 19, 2014 at 6:20 PM, Matt Riedemann >> wrote: >> > >> > >> > On 2/19/2014 7:13 PM, Joe Gordon wrote: >> >> >> >> Hi All, >> >> >> >> As many of you know most oslo-incubator code is wildly out of sync. >> >> Assuming we consider it a good idea to sync up oslo-incubator code >> >> before cutting Icehouse, then we have a problem. >> >> >> >> Today oslo-incubator code is synced in ad-hoc manor, resulting in >> >> duplicated efforts and wildly out of date code. Part of the challenges >> >> today are backwards incompatible changes and new oslo bugs. I expect >> >> that once we get a single project to have an up to date oslo-incubator >> >> copy it will make syncing a second project significantly easier. So >> >> because I (hopefully) have some karma built up in nova, I would like >> >> to volunteer nova to be the guinea pig. >> >> >> >> >> >> To fix this I would like to propose starting an oslo-incubator/nova >> >> sync team. They would be responsible for getting nova's oslo code up >> >> to date. I expect this work to involve: >> >> * Reviewing lots of oslo sync patches >> >> * Tracking the current sync patches >> >> * Syncing over the low hanging fruit, modules that work without >> >> changing >> >> nova. >> >> * Reporting bugs to oslo team >> >> * Working with oslo team to figure out how to deal with backwards >> >> incompatible changes >> >>* Update nova code or make oslo module backwards compatible >> >> * Track all this >> >> * Create a roadmap for other projects to follow (re: documentation) >> >> >> >> I am looking for volunteers to help with this effort, any takers? >> >> >> >> >> >> best, >> >> Joe Gordon >> >> >> >> ___ >> >> OpenStack-dev mailing list >> >> OpenStack-dev@lists.openstack.org >> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> > >> > Well I'll get the ball rolling... >> > >> > In the past when this has come up there is always a debate over should >> > be >> > just sync to sync because we should always be up to date, or is that >> > dangerous and we should only sync when there is a need (which is what >> > the >> > review guidelines say now [1]). There are pros and cons: >> > >> > pros: >> > >> > - we get bug fixes that we didn't know existed >> > - it should be less painful to sync if we do it more often >> > >> > cons: >> > >> > - it's more review overhead and some crazy guy thinks we need a special >> > team >> > dedicated to reviewing those changes :) >> > - there are some changes in o-i that would break nova; I'm specifically >> > thinking of the oslo RequestContext which has domain support now (or >> > some >> > other keystone thingy) and nova has it's own RequestContext - so if we >> > did >> > sync that from o-i it would change nova's logging context and break on >> > us >> > since we didn't use oslo context. >> > >> > For that last con, I'd argue that we should move to the oslo >> > RequestContext, >> > I'm not sure why we aren't. Would that module then not fall under &g
Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
Shed a little bit of light on Matt's comment about Keystone removing oslo-incubator code and the issues we hit. Comments below. Best Regards, Lance Bragstad ldbra...@us.ibm.com Doug Hellmann wrote on 02/19/2014 09:00:29 PM: > From: Doug Hellmann > To: "OpenStack Development Mailing List (not for usage questions)" > , > Date: 02/19/2014 09:12 PM > Subject: Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator > sync workflow > > > On Wed, Feb 19, 2014 at 9:52 PM, Joe Gordon wrote: > As a side to this, as an exercise I tried a oslo sync in cinder to see > what kind of issues would arise and here are my findings so far: > https://review.openstack.org/#/c/74786/ > > On Wed, Feb 19, 2014 at 6:20 PM, Matt Riedemann > wrote: > > > > > > On 2/19/2014 7:13 PM, Joe Gordon wrote: > >> > >> Hi All, > >> > >> As many of you know most oslo-incubator code is wildly out of sync. > >> Assuming we consider it a good idea to sync up oslo-incubator code > >> before cutting Icehouse, then we have a problem. > >> > >> Today oslo-incubator code is synced in ad-hoc manor, resulting in > >> duplicated efforts and wildly out of date code. Part of the challenges > >> today are backwards incompatible changes and new oslo bugs. I expect > >> that once we get a single project to have an up to date oslo-incubator > >> copy it will make syncing a second project significantly easier. So > >> because I (hopefully) have some karma built up in nova, I would like > >> to volunteer nova to be the guinea pig. > >> > >> > >> To fix this I would like to propose starting an oslo-incubator/nova > >> sync team. They would be responsible for getting nova's oslo code up > >> to date. I expect this work to involve: > >> * Reviewing lots of oslo sync patches > >> * Tracking the current sync patches > >> * Syncing over the low hanging fruit, modules that work without changing > >> nova. > >> * Reporting bugs to oslo team > >> * Working with oslo team to figure out how to deal with backwards > >> incompatible changes > >> * Update nova code or make oslo module backwards compatible > >> * Track all this > >> * Create a roadmap for other projects to follow (re: documentation) > >> > >> I am looking for volunteers to help with this effort, any takers? > >> > >> > >> best, > >> Joe Gordon > >> > >> ___ > >> OpenStack-dev mailing list > >> OpenStack-dev@lists.openstack.org > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >> > > > > Well I'll get the ball rolling... > > > > In the past when this has come up there is always a debate over should be > > just sync to sync because we should always be up to date, or is that > > dangerous and we should only sync when there is a need (which is what the > > review guidelines say now [1]). There are pros and cons: > > > > pros: > > > > - we get bug fixes that we didn't know existed > > - it should be less painful to sync if we do it more often > > > > cons: > > > > - it's more review overhead and some crazy guy thinks we need a special team > > dedicated to reviewing those changes :) > > - there are some changes in o-i that would break nova; I'm specifically > > thinking of the oslo RequestContext which has domain support now (or some > > other keystone thingy) and nova has it's own RequestContext - so if we did > > sync that from o-i it would change nova's logging context and break on us > > since we didn't use oslo context. > > > > For that last con, I'd argue that we should move to the oslo RequestContext, > > I'm not sure why we aren't. Would that module then not fall under > > low-hanging-fruit? > I am classifying low hanging fruit as anything that doesn't require > any nova changes to work. > > +1 > > I think the DB API modules have been a concern for auto-syncing before too > > but I can't remember why now...something about possibly changing the > > behavior of how the nova migrations would work? But if they are already > > using the common code, I don't see the issue. > AFAIK there is already a team working on db api syncing, so I was > thinking of let them deal with it. > > +1 > > Doug > > > > > This is kind of an aside, but I'm kind of confused now about how the syncs > > work w
Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
On Wed, Feb 19, 2014 at 9:52 PM, Joe Gordon wrote: > As a side to this, as an exercise I tried a oslo sync in cinder to see > what kind of issues would arise and here are my findings so far: > https://review.openstack.org/#/c/74786/ > > On Wed, Feb 19, 2014 at 6:20 PM, Matt Riedemann > wrote: > > > > > > On 2/19/2014 7:13 PM, Joe Gordon wrote: > >> > >> Hi All, > >> > >> As many of you know most oslo-incubator code is wildly out of sync. > >> Assuming we consider it a good idea to sync up oslo-incubator code > >> before cutting Icehouse, then we have a problem. > >> > >> Today oslo-incubator code is synced in ad-hoc manor, resulting in > >> duplicated efforts and wildly out of date code. Part of the challenges > >> today are backwards incompatible changes and new oslo bugs. I expect > >> that once we get a single project to have an up to date oslo-incubator > >> copy it will make syncing a second project significantly easier. So > >> because I (hopefully) have some karma built up in nova, I would like > >> to volunteer nova to be the guinea pig. > >> > >> > >> To fix this I would like to propose starting an oslo-incubator/nova > >> sync team. They would be responsible for getting nova's oslo code up > >> to date. I expect this work to involve: > >> * Reviewing lots of oslo sync patches > >> * Tracking the current sync patches > >> * Syncing over the low hanging fruit, modules that work without changing > >> nova. > >> * Reporting bugs to oslo team > >> * Working with oslo team to figure out how to deal with backwards > >> incompatible changes > >>* Update nova code or make oslo module backwards compatible > >> * Track all this > >> * Create a roadmap for other projects to follow (re: documentation) > >> > >> I am looking for volunteers to help with this effort, any takers? > >> > >> > >> best, > >> Joe Gordon > >> > >> ___ > >> OpenStack-dev mailing list > >> OpenStack-dev@lists.openstack.org > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >> > > > > Well I'll get the ball rolling... > > > > In the past when this has come up there is always a debate over should be > > just sync to sync because we should always be up to date, or is that > > dangerous and we should only sync when there is a need (which is what the > > review guidelines say now [1]). There are pros and cons: > > > > pros: > > > > - we get bug fixes that we didn't know existed > > - it should be less painful to sync if we do it more often > > > > cons: > > > > - it's more review overhead and some crazy guy thinks we need a special > team > > dedicated to reviewing those changes :) > > - there are some changes in o-i that would break nova; I'm specifically > > thinking of the oslo RequestContext which has domain support now (or some > > other keystone thingy) and nova has it's own RequestContext - so if we > did > > sync that from o-i it would change nova's logging context and break on us > > since we didn't use oslo context. > > > > For that last con, I'd argue that we should move to the oslo > RequestContext, > > I'm not sure why we aren't. Would that module then not fall under > > low-hanging-fruit? > > I am classifying low hanging fruit as anything that doesn't require > any nova changes to work. > +1 > > I think the DB API modules have been a concern for auto-syncing before > too > > but I can't remember why now...something about possibly changing the > > behavior of how the nova migrations would work? But if they are already > > using the common code, I don't see the issue. > > AFAIK there is already a team working on db api syncing, so I was > thinking of let them deal with it. > +1 Doug > > > > > This is kind of an aside, but I'm kind of confused now about how the > syncs > > work with things that fall under oslo.rootwrap or oslo.messaging, like > this > > patch [2]. It doesn't completely match the o-i patch, i.e. it's not > syncing > > over openstack/common/rootwrap/wrapper.py, and I'm assuming because > that's > > in oslo.rootwrap now? But then why does the code still exist in > > oslo-incubator? > > > > I think the keystone guys are running into a similar issue where they > want > > to remove a bunch of now-dead messaging code from keystone but can't > because > > there are still some things in oslo-incubator using oslo.messaging code, > or > > something weird like that. So maybe those modules are considered out of > > scope for this effort until the o-r/o-m code is completely out of o-i? > > > > Finally, just like we'd like to have cores for each virt driver in nova > and > > the neutron API in nova, I think this kind of thing, at least initially, > > would benefit from having some oslo cores involved in a team that are > also > > familiar to a degree with nova, e.g. bnemec or dims. > > > > [1] > https://wiki.openstack.org/wiki/ReviewChecklist#Oslo_Syncing_Checklist > > [2] https://review.openstack.org/#/c/73340/ > > > > -- > > > > Thanks, > > >
Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
As a side to this, as an exercise I tried a oslo sync in cinder to see what kind of issues would arise and here are my findings so far: https://review.openstack.org/#/c/74786/ On Wed, Feb 19, 2014 at 6:20 PM, Matt Riedemann wrote: > > > On 2/19/2014 7:13 PM, Joe Gordon wrote: >> >> Hi All, >> >> As many of you know most oslo-incubator code is wildly out of sync. >> Assuming we consider it a good idea to sync up oslo-incubator code >> before cutting Icehouse, then we have a problem. >> >> Today oslo-incubator code is synced in ad-hoc manor, resulting in >> duplicated efforts and wildly out of date code. Part of the challenges >> today are backwards incompatible changes and new oslo bugs. I expect >> that once we get a single project to have an up to date oslo-incubator >> copy it will make syncing a second project significantly easier. So >> because I (hopefully) have some karma built up in nova, I would like >> to volunteer nova to be the guinea pig. >> >> >> To fix this I would like to propose starting an oslo-incubator/nova >> sync team. They would be responsible for getting nova's oslo code up >> to date. I expect this work to involve: >> * Reviewing lots of oslo sync patches >> * Tracking the current sync patches >> * Syncing over the low hanging fruit, modules that work without changing >> nova. >> * Reporting bugs to oslo team >> * Working with oslo team to figure out how to deal with backwards >> incompatible changes >>* Update nova code or make oslo module backwards compatible >> * Track all this >> * Create a roadmap for other projects to follow (re: documentation) >> >> I am looking for volunteers to help with this effort, any takers? >> >> >> best, >> Joe Gordon >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > Well I'll get the ball rolling... > > In the past when this has come up there is always a debate over should be > just sync to sync because we should always be up to date, or is that > dangerous and we should only sync when there is a need (which is what the > review guidelines say now [1]). There are pros and cons: > > pros: > > - we get bug fixes that we didn't know existed > - it should be less painful to sync if we do it more often > > cons: > > - it's more review overhead and some crazy guy thinks we need a special team > dedicated to reviewing those changes :) > - there are some changes in o-i that would break nova; I'm specifically > thinking of the oslo RequestContext which has domain support now (or some > other keystone thingy) and nova has it's own RequestContext - so if we did > sync that from o-i it would change nova's logging context and break on us > since we didn't use oslo context. > > For that last con, I'd argue that we should move to the oslo RequestContext, > I'm not sure why we aren't. Would that module then not fall under > low-hanging-fruit? I am classifying low hanging fruit as anything that doesn't require any nova changes to work. > > I think the DB API modules have been a concern for auto-syncing before too > but I can't remember why now...something about possibly changing the > behavior of how the nova migrations would work? But if they are already > using the common code, I don't see the issue. AFAIK there is already a team working on db api syncing, so I was thinking of let them deal with it. > > This is kind of an aside, but I'm kind of confused now about how the syncs > work with things that fall under oslo.rootwrap or oslo.messaging, like this > patch [2]. It doesn't completely match the o-i patch, i.e. it's not syncing > over openstack/common/rootwrap/wrapper.py, and I'm assuming because that's > in oslo.rootwrap now? But then why does the code still exist in > oslo-incubator? > > I think the keystone guys are running into a similar issue where they want > to remove a bunch of now-dead messaging code from keystone but can't because > there are still some things in oslo-incubator using oslo.messaging code, or > something weird like that. So maybe those modules are considered out of > scope for this effort until the o-r/o-m code is completely out of o-i? > > Finally, just like we'd like to have cores for each virt driver in nova and > the neutron API in nova, I think this kind of thing, at least initially, > would benefit from having some oslo cores involved in a team that are also > familiar to a degree with nova, e.g. bnemec or dims. > > [1] https://wiki.openstack.org/wiki/ReviewChecklist#Oslo_Syncing_Checklist > [2] https://review.openstack.org/#/c/73340/ > > -- > > Thanks, > > Matt Riedemann > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://li
Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
On Wed, Feb 19, 2014 at 9:20 PM, Matt Riedemann wrote: > > > On 2/19/2014 7:13 PM, Joe Gordon wrote: > >> Hi All, >> >> As many of you know most oslo-incubator code is wildly out of sync. >> Assuming we consider it a good idea to sync up oslo-incubator code >> before cutting Icehouse, then we have a problem. >> >> Today oslo-incubator code is synced in ad-hoc manor, resulting in >> duplicated efforts and wildly out of date code. Part of the challenges >> today are backwards incompatible changes and new oslo bugs. I expect >> that once we get a single project to have an up to date oslo-incubator >> copy it will make syncing a second project significantly easier. So >> because I (hopefully) have some karma built up in nova, I would like >> to volunteer nova to be the guinea pig. >> >> >> To fix this I would like to propose starting an oslo-incubator/nova >> sync team. They would be responsible for getting nova's oslo code up >> to date. I expect this work to involve: >> * Reviewing lots of oslo sync patches >> * Tracking the current sync patches >> * Syncing over the low hanging fruit, modules that work without changing >> nova. >> * Reporting bugs to oslo team >> * Working with oslo team to figure out how to deal with backwards >> incompatible changes >>* Update nova code or make oslo module backwards compatible >> * Track all this >> * Create a roadmap for other projects to follow (re: documentation) >> >> I am looking for volunteers to help with this effort, any takers? >> >> >> best, >> Joe Gordon >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > Well I'll get the ball rolling... > > In the past when this has come up there is always a debate over should be > just sync to sync because we should always be up to date, or is that > dangerous and we should only sync when there is a need (which is what the > review guidelines say now [1]). There are pros and cons: > > pros: > > - we get bug fixes that we didn't know existed > - it should be less painful to sync if we do it more often > > cons: > > - it's more review overhead and some crazy guy thinks we need a special > team dedicated to reviewing those changes :) > - there are some changes in o-i that would break nova; I'm specifically > thinking of the oslo RequestContext which has domain support now (or some > other keystone thingy) and nova has it's own RequestContext - so if we did > sync that from o-i it would change nova's logging context and break on us > since we didn't use oslo context. > Another con is that if we do find a critical bug in an incubator module, and a project that uses that module is far out of date, applying the fix may be more difficult. (This is also another motivation for moving code out of the incubator entirely, but as Joe pointed out earlier today, that's not really a short-term solution.) > > For that last con, I'd argue that we should move to the oslo > RequestContext, I'm not sure why we aren't. Would that module then not > fall under low-hanging-fruit? > > I think the DB API modules have been a concern for auto-syncing before too > but I can't remember why now...something about possibly changing the > behavior of how the nova migrations would work? But if they are already > using the common code, I don't see the issue. > There has been some recent work on the db code to make it more suitable for use in some of the other projects that don't have a single global session pool. There's a compatibility shim, which should make the update painless, but it's not just a simple file copy. > > This is kind of an aside, but I'm kind of confused now about how the syncs > work with things that fall under oslo.rootwrap or oslo.messaging, like this > patch [2]. It doesn't completely match the o-i patch, i.e. it's not > syncing over openstack/common/rootwrap/wrapper.py, and I'm assuming > because that's in oslo.rootwrap now? But then why does the code still > exist in oslo-incubator? > After a module graduates to a library, we treat the incubator copy as the "stable branch" until all of the integrated projects that consume the module have migrated to the new library. That way if bugs are found, the fixes can be applied to a project without having to also migrate to the library. So, the best action is to port to the library. As a fall back, at least update to the most current version from in the incubator now. I believe all projects are already updated to use oslo.rootwrap. I think the keystone guys are running into a similar issue where they want > to remove a bunch of now-dead messaging code from keystone but can't > because there are still some things in oslo-incubator using oslo.messaging > code, or something weird like that. So maybe those modules are considered > out of scope for this effort until the o-r/o-m code is completely out of > o-i? > There's a notifier middlewa
Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
On Wed, Feb 19, 2014 at 8:13 PM, Joe Gordon wrote: > Hi All, > > As many of you know most oslo-incubator code is wildly out of sync. > Assuming we consider it a good idea to sync up oslo-incubator code > before cutting Icehouse, then we have a problem. > > Today oslo-incubator code is synced in ad-hoc manor, resulting in > duplicated efforts and wildly out of date code. Part of the challenges > today are backwards incompatible changes and new oslo bugs. I expect > that once we get a single project to have an up to date oslo-incubator > copy it will make syncing a second project significantly easier. So > because I (hopefully) have some karma built up in nova, I would like > to volunteer nova to be the guinea pig. > Thank you for volunteering to spear-head this, Joe. > To fix this I would like to propose starting an oslo-incubator/nova > sync team. They would be responsible for getting nova's oslo code up > to date. I expect this work to involve: > * Reviewing lots of oslo sync patches > * Tracking the current sync patches > * Syncing over the low hanging fruit, modules that work without changing > nova. > * Reporting bugs to oslo team > * Working with oslo team to figure out how to deal with backwards > incompatible changes > * Update nova code or make oslo module backwards compatible > * Track all this > * Create a roadmap for other projects to follow (re: documentation) > > I am looking for volunteers to help with this effort, any takers? > I will help, especially with reviews and tracking. We are going to want someone from the team working on the db modules to participate as well, since we know that's one area where the API has diverged some (although we did take backwards compatibility into account). Victor, can you help find us a volunteer? Doug > > > best, > Joe Gordon > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
On 2/19/2014 7:13 PM, Joe Gordon wrote: Hi All, As many of you know most oslo-incubator code is wildly out of sync. Assuming we consider it a good idea to sync up oslo-incubator code before cutting Icehouse, then we have a problem. Today oslo-incubator code is synced in ad-hoc manor, resulting in duplicated efforts and wildly out of date code. Part of the challenges today are backwards incompatible changes and new oslo bugs. I expect that once we get a single project to have an up to date oslo-incubator copy it will make syncing a second project significantly easier. So because I (hopefully) have some karma built up in nova, I would like to volunteer nova to be the guinea pig. To fix this I would like to propose starting an oslo-incubator/nova sync team. They would be responsible for getting nova's oslo code up to date. I expect this work to involve: * Reviewing lots of oslo sync patches * Tracking the current sync patches * Syncing over the low hanging fruit, modules that work without changing nova. * Reporting bugs to oslo team * Working with oslo team to figure out how to deal with backwards incompatible changes * Update nova code or make oslo module backwards compatible * Track all this * Create a roadmap for other projects to follow (re: documentation) I am looking for volunteers to help with this effort, any takers? best, Joe Gordon ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Well I'll get the ball rolling... In the past when this has come up there is always a debate over should be just sync to sync because we should always be up to date, or is that dangerous and we should only sync when there is a need (which is what the review guidelines say now [1]). There are pros and cons: pros: - we get bug fixes that we didn't know existed - it should be less painful to sync if we do it more often cons: - it's more review overhead and some crazy guy thinks we need a special team dedicated to reviewing those changes :) - there are some changes in o-i that would break nova; I'm specifically thinking of the oslo RequestContext which has domain support now (or some other keystone thingy) and nova has it's own RequestContext - so if we did sync that from o-i it would change nova's logging context and break on us since we didn't use oslo context. For that last con, I'd argue that we should move to the oslo RequestContext, I'm not sure why we aren't. Would that module then not fall under low-hanging-fruit? I think the DB API modules have been a concern for auto-syncing before too but I can't remember why now...something about possibly changing the behavior of how the nova migrations would work? But if they are already using the common code, I don't see the issue. This is kind of an aside, but I'm kind of confused now about how the syncs work with things that fall under oslo.rootwrap or oslo.messaging, like this patch [2]. It doesn't completely match the o-i patch, i.e. it's not syncing over openstack/common/rootwrap/wrapper.py, and I'm assuming because that's in oslo.rootwrap now? But then why does the code still exist in oslo-incubator? I think the keystone guys are running into a similar issue where they want to remove a bunch of now-dead messaging code from keystone but can't because there are still some things in oslo-incubator using oslo.messaging code, or something weird like that. So maybe those modules are considered out of scope for this effort until the o-r/o-m code is completely out of o-i? Finally, just like we'd like to have cores for each virt driver in nova and the neutron API in nova, I think this kind of thing, at least initially, would benefit from having some oslo cores involved in a team that are also familiar to a degree with nova, e.g. bnemec or dims. [1] https://wiki.openstack.org/wiki/ReviewChecklist#Oslo_Syncing_Checklist [2] https://review.openstack.org/#/c/73340/ -- Thanks, Matt Riedemann ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
Hi All, As many of you know most oslo-incubator code is wildly out of sync. Assuming we consider it a good idea to sync up oslo-incubator code before cutting Icehouse, then we have a problem. Today oslo-incubator code is synced in ad-hoc manor, resulting in duplicated efforts and wildly out of date code. Part of the challenges today are backwards incompatible changes and new oslo bugs. I expect that once we get a single project to have an up to date oslo-incubator copy it will make syncing a second project significantly easier. So because I (hopefully) have some karma built up in nova, I would like to volunteer nova to be the guinea pig. To fix this I would like to propose starting an oslo-incubator/nova sync team. They would be responsible for getting nova's oslo code up to date. I expect this work to involve: * Reviewing lots of oslo sync patches * Tracking the current sync patches * Syncing over the low hanging fruit, modules that work without changing nova. * Reporting bugs to oslo team * Working with oslo team to figure out how to deal with backwards incompatible changes * Update nova code or make oslo module backwards compatible * Track all this * Create a roadmap for other projects to follow (re: documentation) I am looking for volunteers to help with this effort, any takers? best, Joe Gordon ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
