Re: [openstack-dev] [Neutron] Gap 0 (database migrations) closed!
Carl Baldwin wrote: > Anna, > > Your second point is going to be a bit of a maintenance headache. I > was reviewing a patch [1] where you caught the lack of an import in > head.py. I'm not sure that I can be trusted to check consistently for > this in future reviews. I'm not sure that we can rely on you catching > them all either. Jenkins didn't seem to have any problem with the > patch without the import. > > So, how can we guarantee good maintenance of this head.py file? We have discussed moving all models out of their current diverse locations to one directory, like maybe neutron/db/models/*.py The idea is to move just the model classes (not the entire modules that they currently reside in) here. Then head.py would be able to from neutron.db.models import * # noqa and this would have much less baggage than importing all the current modules. I think the convention of putting all models in one directory will be quite easy to follow and maintain. I don't yet have a timeline for getting this done, but it will be discussed in the Neutron DB meeting[1] on Monday. [1] https://wiki.openstack.org/wiki/Meetings/NeutronDB -- Henry > > Carl > > [1] > https://review.openstack.org/#/c/102101/40/neutron/db/migration/models/head.py > > On Wed, Jul 16, 2014 at 2:14 AM, Anna Kamyshnikova > wrote: >> Hello everyone! >> >> I would like to bring the next two points to everybody's attention: >> >> 1) As Henry mentioned if you add new migration you should make it >> unconditional. Conditional migrations should not be merged since now. >> >> 2) If you add some new models you should ensure that module containing it is >> imported in /neutron/db/migration/models/head.py. >> >> The second point in important for testing which I hope will be merged soon: >> https://review.openstack.org/76520. >> >> Regards, >> Ann >> >> >> >> On Wed, Jul 16, 2014 at 5:54 AM, Kyle Mestery wrote: >>> >>> On Tue, Jul 15, 2014 at 5:49 PM, Henry Gessau wrote: I am happy to announce that the first (zero'th?) item in the Neutron Gap Coverage[1] has merged[2]. The Neutron database now contains all tables for all plugins, and database migrations are no longer conditional on the configuration. In the short term, Neutron developers who write migration scripts need to set migration_for_plugins = ['*'] but we will soon clean up the template for migration scripts so that this will be unnecessary. I would like to say special thanks to Ann Kamyshnikova and Jakub Libosvar for their great work on this solution. Also thanks to Salvatore Orlando and Mark McClain for mentoring this through to the finish. [1] https://wiki.openstack.org/wiki/Governance/TechnicalCommittee/Neutron_Gap_Coverage [2] https://review.openstack.org/96438 >>> This is great news! Thanks to everyone who worked on this particular >>> gap. We're making progress on the other gaps identified in that plan, >>> I'll send an email out once Juno-2 closes with where we're at. >>> >>> Thanks, >>> Kyle >>> ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> ___ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Gap 0 (database migrations) closed!
Anna, Your second point is going to be a bit of a maintenance headache. I was reviewing a patch [1] where you caught the lack of an import in head.py. I'm not sure that I can be trusted to check consistently for this in future reviews. I'm not sure that we can rely on you catching them all either. Jenkins didn't seem to have any problem with the patch without the import. So, how can we guarantee good maintenance of this head.py file? Carl [1] https://review.openstack.org/#/c/102101/40/neutron/db/migration/models/head.py On Wed, Jul 16, 2014 at 2:14 AM, Anna Kamyshnikova wrote: > Hello everyone! > > I would like to bring the next two points to everybody's attention: > > 1) As Henry mentioned if you add new migration you should make it > unconditional. Conditional migrations should not be merged since now. > > 2) If you add some new models you should ensure that module containing it is > imported in /neutron/db/migration/models/head.py. > > The second point in important for testing which I hope will be merged soon: > https://review.openstack.org/76520. > > Regards, > Ann > > > > On Wed, Jul 16, 2014 at 5:54 AM, Kyle Mestery wrote: >> >> On Tue, Jul 15, 2014 at 5:49 PM, Henry Gessau wrote: >> > I am happy to announce that the first (zero'th?) item in the Neutron Gap >> > Coverage[1] has merged[2]. The Neutron database now contains all tables >> > for >> > all plugins, and database migrations are no longer conditional on the >> > configuration. >> > >> > In the short term, Neutron developers who write migration scripts need >> > to set >> > migration_for_plugins = ['*'] >> > but we will soon clean up the template for migration scripts so that >> > this will >> > be unnecessary. >> > >> > I would like to say special thanks to Ann Kamyshnikova and Jakub >> > Libosvar for >> > their great work on this solution. Also thanks to Salvatore Orlando and >> > Mark >> > McClain for mentoring this through to the finish. >> > >> > [1] >> > >> > https://wiki.openstack.org/wiki/Governance/TechnicalCommittee/Neutron_Gap_Coverage >> > [2] https://review.openstack.org/96438 >> > >> This is great news! Thanks to everyone who worked on this particular >> gap. We're making progress on the other gaps identified in that plan, >> I'll send an email out once Juno-2 closes with where we're at. >> >> Thanks, >> Kyle >> >> > ___ >> > OpenStack-dev mailing list >> > OpenStack-dev@lists.openstack.org >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Gap 0 (database migrations) closed!
Thanks Jakub. Certainly helps, appreciate it. -Paddu On 7/16/14 7:41 AM, "Jakub Libosvar" wrote: >On 07/16/2014 04:29 PM, Paddu Krishnan (padkrish) wrote: >> Hello, >> A follow-up development question related to this: >> >> As a part of https://review.openstack.org/#/c/105563/, which was >> introducing a new table in Neutron DB, I was trying to send for review a >> new file in neutron/db/migration/alembic_migrations/versions/ >> >><https://review.openstack.org/#/c/105563/4/neutron/db/migration/alembic_m >>igrations/versions/1be5bdeb1d9a_ml2_network_overlay_type_driver.py> which >> got generated through script "neutron-db-manage". This also >> updated neutron/db/migration/alembic_migrations/versions/ >> >><https://review.openstack.org/#/c/105563/4/neutron/db/migration/alembic_m >>igrations/versions/1be5bdeb1d9a_ml2_network_overlay_type_driver.py>HEAD. >> I was trying to send this file for review as well. >> >> "git review" failed and I saw merge errors >> in neutron/db/migration/alembic_migrations/versions/ >> >><https://review.openstack.org/#/c/105563/4/neutron/db/migration/alembic_m >>igrations/versions/1be5bdeb1d9a_ml2_network_overlay_type_driver.py>HEAD. >> >> W/O HEAD modified, jenkins was failing. I am working to fix this and saw >> this e-mail. >> >> I had to go through all the links in detail in this thread. But, >> meanwhile, the two points mentioned below looks related to the >> patch/issues I am facing. >> So, if I add a new table, I don't need to run the "neutron-db-manage" >> script to generate the file and modify the HEAD anymore? Is (2) below >> need to be done manually? >Hi Paddu, > >the process is the same (create migration script, update HEAD file), but >all migrations should have > >migration_for_plugins = ['*'] > > >Because you created a new DB model in new module, you also need to add > >from neutron.plugins.ml2.drivers import type_network_overlay > >to neutron/db/migration/models/head.py module. > >I hope it helps. > >Kuba > >> >> Thanks, >> Paddu > > > >> >> From: Anna Kamyshnikova > <mailto:akamyshnik...@mirantis.com>> >> Reply-To: "OpenStack Development Mailing List (not for usage questions)" >> > <mailto:openstack-dev@lists.openstack.org>> >> Date: Wednesday, July 16, 2014 1:14 AM >> To: "OpenStack Development Mailing List (not for usage questions)" >> > <mailto:openstack-dev@lists.openstack.org>> >> Subject: Re: [openstack-dev] [Neutron] Gap 0 (database migrations) >>closed! >> >> Hello everyone! >> >> I would like to bring the next two points to everybody's attention: >> >> 1) As Henry mentioned if you add new migration you should make it >> unconditional. Conditional migrations should not be merged since now. >> >> 2) If you add some new models you should ensure that module containing >> it is imported in /neutron/db/migration/models/head.py. >> >> The second point in important for testing which I hope will be merged >> soon: https://review.openstack.org/76520. >> >> Regards, >> Ann >> >> >> >> On Wed, Jul 16, 2014 at 5:54 AM, Kyle Mestery > <mailto:mest...@mestery.com>> wrote: >> >> On Tue, Jul 15, 2014 at 5:49 PM, Henry Gessau > <mailto:ges...@cisco.com>> wrote: >> > I am happy to announce that the first (zero'th?) item in the >>Neutron Gap >> > Coverage[1] has merged[2]. The Neutron database now contains all >>tables for >> > all plugins, and database migrations are no longer conditional on >>the >> > configuration. >> > >> > In the short term, Neutron developers who write migration scripts >>need to set >> > migration_for_plugins = ['*'] >> > but we will soon clean up the template for migration scripts so >>that this will >> > be unnecessary. >> > >> > I would like to say special thanks to Ann Kamyshnikova and Jakub >>Libosvar for >> > their great work on this solution. Also thanks to Salvatore >>Orlando and Mark >> > McClain for mentoring this through to the finish. >> > >> > [1] >> > >>https://wiki.openstack.org/wiki/Governance/TechnicalCommittee/Neutron_Gap >>_Coverage >>
Re: [openstack-dev] [Neutron] Gap 0 (database migrations) closed!
On 07/16/2014 04:29 PM, Paddu Krishnan (padkrish) wrote: > Hello, > A follow-up development question related to this: > > As a part of https://review.openstack.org/#/c/105563/, which was > introducing a new table in Neutron DB, I was trying to send for review a > new file in neutron/db/migration/alembic_migrations/versions/ > <https://review.openstack.org/#/c/105563/4/neutron/db/migration/alembic_migrations/versions/1be5bdeb1d9a_ml2_network_overlay_type_driver.py> > which > got generated through script "neutron-db-manage". This also > updated neutron/db/migration/alembic_migrations/versions/ > <https://review.openstack.org/#/c/105563/4/neutron/db/migration/alembic_migrations/versions/1be5bdeb1d9a_ml2_network_overlay_type_driver.py>HEAD. > I was trying to send this file for review as well. > > "git review" failed and I saw merge errors > in neutron/db/migration/alembic_migrations/versions/ > <https://review.openstack.org/#/c/105563/4/neutron/db/migration/alembic_migrations/versions/1be5bdeb1d9a_ml2_network_overlay_type_driver.py>HEAD. > > > W/O HEAD modified, jenkins was failing. I am working to fix this and saw > this e-mail. > > I had to go through all the links in detail in this thread. But, > meanwhile, the two points mentioned below looks related to the > patch/issues I am facing. > So, if I add a new table, I don't need to run the "neutron-db-manage" > script to generate the file and modify the HEAD anymore? Is (2) below > need to be done manually? Hi Paddu, the process is the same (create migration script, update HEAD file), but all migrations should have migration_for_plugins = ['*'] Because you created a new DB model in new module, you also need to add from neutron.plugins.ml2.drivers import type_network_overlay to neutron/db/migration/models/head.py module. I hope it helps. Kuba > > Thanks, > Paddu > > From: Anna Kamyshnikova <mailto:akamyshnik...@mirantis.com>> > Reply-To: "OpenStack Development Mailing List (not for usage questions)" > <mailto:openstack-dev@lists.openstack.org>> > Date: Wednesday, July 16, 2014 1:14 AM > To: "OpenStack Development Mailing List (not for usage questions)" > <mailto:openstack-dev@lists.openstack.org>> > Subject: Re: [openstack-dev] [Neutron] Gap 0 (database migrations) closed! > > Hello everyone! > > I would like to bring the next two points to everybody's attention: > > 1) As Henry mentioned if you add new migration you should make it > unconditional. Conditional migrations should not be merged since now. > > 2) If you add some new models you should ensure that module containing > it is imported in /neutron/db/migration/models/head.py. > > The second point in important for testing which I hope will be merged > soon: https://review.openstack.org/76520. > > Regards, > Ann > > > > On Wed, Jul 16, 2014 at 5:54 AM, Kyle Mestery <mailto:mest...@mestery.com>> wrote: > > On Tue, Jul 15, 2014 at 5:49 PM, Henry Gessau <mailto:ges...@cisco.com>> wrote: > > I am happy to announce that the first (zero'th?) item in the Neutron Gap > > Coverage[1] has merged[2]. The Neutron database now contains all tables > for > > all plugins, and database migrations are no longer conditional on the > > configuration. > > > > In the short term, Neutron developers who write migration scripts need > to set > > migration_for_plugins = ['*'] > > but we will soon clean up the template for migration scripts so that > this will > > be unnecessary. > > > > I would like to say special thanks to Ann Kamyshnikova and Jakub > Libosvar for > > their great work on this solution. Also thanks to Salvatore Orlando and > Mark > > McClain for mentoring this through to the finish. > > > > [1] > > > https://wiki.openstack.org/wiki/Governance/TechnicalCommittee/Neutron_Gap_Coverage > > [2] https://review.openstack.org/96438 > > > This is great news! Thanks to everyone who worked on this particular > gap. We're making progress on the other gaps identified in that plan, > I'll send an email out once Juno-2 closes with where we're at. > > Thanks, > Kyle > > > ___ > > OpenStack-dev mailing list > > OpenStack-dev@lists.openstack.org > <mailto:OpenStack-dev@lists.openstack.org> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __
Re: [openstack-dev] [Neutron] Gap 0 (database migrations) closed!
Hello, A follow-up development question related to this: As a part of https://review.openstack.org/#/c/105563/, which was introducing a new table in Neutron DB, I was trying to send for review a new file in neutron/db/migration/alembic_migrations/versions/<https://review.openstack.org/#/c/105563/4/neutron/db/migration/alembic_migrations/versions/1be5bdeb1d9a_ml2_network_overlay_type_driver.py> which got generated through script "neutron-db-manage". This also updated neutron/db/migration/alembic_migrations/versions/<https://review.openstack.org/#/c/105563/4/neutron/db/migration/alembic_migrations/versions/1be5bdeb1d9a_ml2_network_overlay_type_driver.py>HEAD. I was trying to send this file for review as well. "git review" failed and I saw merge errors in neutron/db/migration/alembic_migrations/versions/<https://review.openstack.org/#/c/105563/4/neutron/db/migration/alembic_migrations/versions/1be5bdeb1d9a_ml2_network_overlay_type_driver.py>HEAD. W/O HEAD modified, jenkins was failing. I am working to fix this and saw this e-mail. I had to go through all the links in detail in this thread. But, meanwhile, the two points mentioned below looks related to the patch/issues I am facing. So, if I add a new table, I don't need to run the "neutron-db-manage" script to generate the file and modify the HEAD anymore? Is (2) below need to be done manually? Thanks, Paddu From: Anna Kamyshnikova mailto:akamyshnik...@mirantis.com>> Reply-To: "OpenStack Development Mailing List (not for usage questions)" mailto:openstack-dev@lists.openstack.org>> Date: Wednesday, July 16, 2014 1:14 AM To: "OpenStack Development Mailing List (not for usage questions)" mailto:openstack-dev@lists.openstack.org>> Subject: Re: [openstack-dev] [Neutron] Gap 0 (database migrations) closed! Hello everyone! I would like to bring the next two points to everybody's attention: 1) As Henry mentioned if you add new migration you should make it unconditional. Conditional migrations should not be merged since now. 2) If you add some new models you should ensure that module containing it is imported in /neutron/db/migration/models/head.py. The second point in important for testing which I hope will be merged soon: https://review.openstack.org/76520. Regards, Ann On Wed, Jul 16, 2014 at 5:54 AM, Kyle Mestery mailto:mest...@mestery.com>> wrote: On Tue, Jul 15, 2014 at 5:49 PM, Henry Gessau mailto:ges...@cisco.com>> wrote: > I am happy to announce that the first (zero'th?) item in the Neutron Gap > Coverage[1] has merged[2]. The Neutron database now contains all tables for > all plugins, and database migrations are no longer conditional on the > configuration. > > In the short term, Neutron developers who write migration scripts need to set > migration_for_plugins = ['*'] > but we will soon clean up the template for migration scripts so that this will > be unnecessary. > > I would like to say special thanks to Ann Kamyshnikova and Jakub Libosvar for > their great work on this solution. Also thanks to Salvatore Orlando and Mark > McClain for mentoring this through to the finish. > > [1] > https://wiki.openstack.org/wiki/Governance/TechnicalCommittee/Neutron_Gap_Coverage > [2] https://review.openstack.org/96438 > This is great news! Thanks to everyone who worked on this particular gap. We're making progress on the other gaps identified in that plan, I'll send an email out once Juno-2 closes with where we're at. Thanks, Kyle > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Gap 0 (database migrations) closed!
Hello everyone! I would like to bring the next two points to everybody's attention: 1) As Henry mentioned if you add new migration you should make it unconditional. Conditional migrations should not be merged since now. 2) If you add some new models you should ensure that module containing it is imported in /neutron/db/migration/models/head.py. The second point in important for testing which I hope will be merged soon: https://review.openstack.org/76520. Regards, Ann On Wed, Jul 16, 2014 at 5:54 AM, Kyle Mestery wrote: > On Tue, Jul 15, 2014 at 5:49 PM, Henry Gessau wrote: > > I am happy to announce that the first (zero'th?) item in the Neutron Gap > > Coverage[1] has merged[2]. The Neutron database now contains all tables > for > > all plugins, and database migrations are no longer conditional on the > > configuration. > > > > In the short term, Neutron developers who write migration scripts need > to set > > migration_for_plugins = ['*'] > > but we will soon clean up the template for migration scripts so that > this will > > be unnecessary. > > > > I would like to say special thanks to Ann Kamyshnikova and Jakub > Libosvar for > > their great work on this solution. Also thanks to Salvatore Orlando and > Mark > > McClain for mentoring this through to the finish. > > > > [1] > > > https://wiki.openstack.org/wiki/Governance/TechnicalCommittee/Neutron_Gap_Coverage > > [2] https://review.openstack.org/96438 > > > This is great news! Thanks to everyone who worked on this particular > gap. We're making progress on the other gaps identified in that plan, > I'll send an email out once Juno-2 closes with where we're at. > > Thanks, > Kyle > > > ___ > > OpenStack-dev mailing list > > OpenStack-dev@lists.openstack.org > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Gap 0 (database migrations) closed!
On Tue, Jul 15, 2014 at 5:49 PM, Henry Gessau wrote: > I am happy to announce that the first (zero'th?) item in the Neutron Gap > Coverage[1] has merged[2]. The Neutron database now contains all tables for > all plugins, and database migrations are no longer conditional on the > configuration. > > In the short term, Neutron developers who write migration scripts need to set > migration_for_plugins = ['*'] > but we will soon clean up the template for migration scripts so that this will > be unnecessary. > > I would like to say special thanks to Ann Kamyshnikova and Jakub Libosvar for > their great work on this solution. Also thanks to Salvatore Orlando and Mark > McClain for mentoring this through to the finish. > > [1] > https://wiki.openstack.org/wiki/Governance/TechnicalCommittee/Neutron_Gap_Coverage > [2] https://review.openstack.org/96438 > This is great news! Thanks to everyone who worked on this particular gap. We're making progress on the other gaps identified in that plan, I'll send an email out once Juno-2 closes with where we're at. Thanks, Kyle > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
