[openstack-dev] Mike Bayer 20141205
1. Alembic release - I worked through some regressions introduced by Alembic 0.7.0 and the subsequent 0.7.1 with the Neutron folks. This started on Monday with https://review.openstack.org/#/c/137989/, and by Wednesday I had identified enough small regressions in 0.7.0 that I had to put 0.7.1 out, so that review got expedited with https://review.openstack.org/#/c/138998/ following from Neutron devs to continue fixing. Version 0.7.1 includes the foreign key autogenerate support first proposed by Ann Kamyshnikova. Changelog at http://alembic.readthedocs.org/en/latest/changelog.html#change-0.7.1. 2. MySQL driver stuff. I have a SQLAlchemy user who is running some kind of heavy load with gevent and PyMySQL. While this user is not openstack-specific, the thing he is doing is a lot like what we might be doing if and when we move our MySQL drivers to MySQL-connector-Python, which is compatible with eventlet in that it is pure Python and can be monkeypatched.The issue observed by this user applies to both PyMySQL and MySQL-connector, and I can reproduce it *without* using SQLAlchemy, though it does use a very makeshift connection pool designed to approximate what SQLAlchemy’s does. The issue is scary because it illustrates Python code that should have been killed being invoked on a database connection that should have been dead, calling commit(), and then actually *succeeding* in committing only *part* of the data. This is not an issue that impacts Openstack right now but if the same thing applies to eventlet, then this would definitely be something we’d need to worry about if we start using MySQL-connector in a high load scenario (which has been the plan) so I’ve forwarded my findings onto openstack-dev to see if anyone can help me understand it. The intro + test case for this issue starts at http://lists.openstack.org/pipermail/openstack-dev/2014-December/052344.html. 3. enginefacade - The engine facade as I described in https://review.openstack.org/#/c/125181/, which we also talked about on the Nova compute call this week, is now built! I spent monday and tuesday on the buildout for this, and that can be seen and reviewed here: https://review.openstack.org/#/c/138215/ As of today I’m still nursing it through CI, as even with projects using the “legacy” APIs, they are still finding lots of little silly things that I keep having to fix (people calling the old EngineFacade with arguments I didn’t expect, people importing from oslo.db in an order I did not expect, etc). While these consuming projects could be fixed to not have these little issues, for now I am trying to push everything to work as identically as possible to how it was earlier, when the new API is not explicitly invoked. I’ll be continuing to get this to pass all tempest runs through next week. For enginefacade I’d like the folks from the call to take a look, and in particular if Matthew Booth wants to look into it, this is ready to start being used for prototyping Nova with it. 4. Connectivity stuff - today I worked a bunch with Viktor Sergeyev who has been trying to fix an issue with MySQL OperationalErrors that are raised when the database is shut off entirely; in oslo.db we have logic that wraps all exceptions unconditionally, including that it identifies disconnect exceptions. In the case where the DB throws a disconnect, and we loop around to “retry” this query in order to get it to reconnect, then that reconnect continues to fail, the second run doesn’t get wrapped. So today I’ve fixed both the upstream issue for SQLAlchemy 1.0, and also made a series of adjustments to oslo.db to accommodate SQLAlchemy 1.0’s system correctly as well as to work around the issue when SQLAlchemy 1.0 is present. That’s a three-series of patches that are unsurprisingly going to take some nursing to get through the gate, so I’ll be continuing with that next week. This series starts at https://review.openstack.org/139725 https://review.openstack.org/139733 https://review.openstack.org/139738 . 5. SQLA 1.0 stuff. - getting SQLAlchemy 1.0 close to release is becoming critical so I’ve been moving around issues and priorities to expedite this. There’s many stability enhancements oslo.db would benefit from as well as some major performance-related features that I’ve been planning all along to introduce to projects. 1.0 is very full of lots of changes that aren’t really being tested outside of my own CI, so getting something out the door on it is key, otherwise it will just be too different from 0.9 in order for people to have smooth upgrades. I do run SQLA 1.0 in CI against a subset of Neutron, Nova, Keystone and Oslo tests so we should be in OK shape, but there is still a lot to go. Work completed so far can be seen at http://docs.sqlalchemy.org/en/latest/changelog/migration_10.html. ___ OpenStack-dev mailing list
Re: [openstack-dev] Mike Bayer 20141205
this was sent to the wrong list! please ignore. (or if you find it interesting, then great!) On Dec 5, 2014, at 6:13 PM, Mike Bayer mba...@redhat.com wrote: 1. Alembic release - I worked through some regressions introduced by Alembic 0.7.0 and the subsequent 0.7.1 with the Neutron folks. This started on Monday with https://review.openstack.org/#/c/137989/, and by Wednesday I had identified enough small regressions in 0.7.0 that I had to put 0.7.1 out, so that review got expedited with https://review.openstack.org/#/c/138998/ following from Neutron devs to continue fixing. Version 0.7.1 includes the foreign key autogenerate support first proposed by Ann Kamyshnikova. Changelog at http://alembic.readthedocs.org/en/latest/changelog.html#change-0.7.1. 2. MySQL driver stuff. I have a SQLAlchemy user who is running some kind of heavy load with gevent and PyMySQL. While this user is not openstack-specific, the thing he is doing is a lot like what we might be doing if and when we move our MySQL drivers to MySQL-connector-Python, which is compatible with eventlet in that it is pure Python and can be monkeypatched.The issue observed by this user applies to both PyMySQL and MySQL-connector, and I can reproduce it *without* using SQLAlchemy, though it does use a very makeshift connection pool designed to approximate what SQLAlchemy’s does. The issue is scary because it illustrates Python code that should have been killed being invoked on a database connection that should have been dead, calling commit(), and then actually *succeeding* in committing only *part* of the data. This is not an issue that impacts Openstack right now but if the same thing applies to eventlet, then this would definitely be something we’d need to worry about if we start using MySQL-connector in a high load scenario (which has been the plan) so I’ve forwarded my findings onto openstack-dev to see if anyone can help me understand it. The intro + test case for this issue starts at http://lists.openstack.org/pipermail/openstack-dev/2014-December/052344.html. 3. enginefacade - The engine facade as I described in https://review.openstack.org/#/c/125181/, which we also talked about on the Nova compute call this week, is now built! I spent monday and tuesday on the buildout for this, and that can be seen and reviewed here: https://review.openstack.org/#/c/138215/ As of today I’m still nursing it through CI, as even with projects using the “legacy” APIs, they are still finding lots of little silly things that I keep having to fix (people calling the old EngineFacade with arguments I didn’t expect, people importing from oslo.db in an order I did not expect, etc). While these consuming projects could be fixed to not have these little issues, for now I am trying to push everything to work as identically as possible to how it was earlier, when the new API is not explicitly invoked. I’ll be continuing to get this to pass all tempest runs through next week. For enginefacade I’d like the folks from the call to take a look, and in particular if Matthew Booth wants to look into it, this is ready to start being used for prototyping Nova with it. 4. Connectivity stuff - today I worked a bunch with Viktor Sergeyev who has been trying to fix an issue with MySQL OperationalErrors that are raised when the database is shut off entirely; in oslo.db we have logic that wraps all exceptions unconditionally, including that it identifies disconnect exceptions. In the case where the DB throws a disconnect, and we loop around to “retry” this query in order to get it to reconnect, then that reconnect continues to fail, the second run doesn’t get wrapped. So today I’ve fixed both the upstream issue for SQLAlchemy 1.0, and also made a series of adjustments to oslo.db to accommodate SQLAlchemy 1.0’s system correctly as well as to work around the issue when SQLAlchemy 1.0 is present. That’s a three-series of patches that are unsurprisingly going to take some nursing to get through the gate, so I’ll be continuing with that next week. This series starts at https://review.openstack.org/139725 https://review.openstack.org/139733 https://review.openstack.org/139738 . 5. SQLA 1.0 stuff. - getting SQLAlchemy 1.0 close to release is becoming critical so I’ve been moving around issues and priorities to expedite this. There’s many stability enhancements oslo.db would benefit from as well as some major performance-related features that I’ve been planning all along to introduce to projects. 1.0 is very full of lots of changes that aren’t really being tested outside of my own CI, so getting something out the door on it is key, otherwise it will just be too different from 0.9 in order for people to have smooth upgrades. I do run SQLA 1.0 in CI against a subset of Neutron, Nova, Keystone and Oslo tests so we should be
