[openstack-dev] [sqlalchemy][db][oslo.db][mistral] Is there a recommended MySQL driver for OpenStack projects?
Hi, We’ve tried to address the bug [1] which is essentially caused by the fact that we saw that MySQLDb driver wasn’t compatible with eventlet’s green threads. In a nutshell, when we used the “eventlet” RPC executor (see [2]), the system would get stuck once in a while when dispatching green between green threads when it tried to hit Mysql, but since the driver wasn’t eventlet friendly it didn’t work. For that reason we had to use the “blocking” RPC executor so far for Mistral Engine that deals with DB transactions. Now, I am back to experiment with all this and see if we can actually switch to “eventlet” like most other project do. So far, it seems like the problem is gone in case if I’m using Pymysql driver (didn’t yet try other drivers like mysqlclient and the official mysql connector from Oracle). Previously, at least in production we always used MySQLDb on Python 2.7. So, I’m trying to understand if we have a “community recommended” (or may be even mandatory) Mysql driver to use and what consequences of the driver choice are. I’d appreciate any help with clarifying this (may be links to some previous discussions etc.) Thanks [1] https://bugs.launchpad.net/mistral/+bug/1696469 [2] https://docs.openstack.org/oslo.messaging/ocata/executors.html#eventlet Renat Akhmerov @Nokia __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sqlalchemy][db][oslo.db][mistral] Is there a recommended MySQL driver for OpenStack projects?
On 2018-07-03 18:33:44 +0700 (+0700), Renat Akhmerov wrote: [...] > So, I’m trying to understand if we have a “community recommended” > (or may be even mandatory) Mysql driver to use and what > consequences of the driver choice are. I’d appreciate any help > with clarifying this (may be links to some previous discussions > etc.) [...] There was a concerted effort around the first half of 2015 (prior to TC cycle goals or it probably would have been one) in which most projects switched to pymysql by default because mysql-python lacked Py3k support. Another significant up-side to pymysql is that it was implemented in pure Python rather than being a wrapper around libmysql, so simpler for dependency management. There was initially some concern that it would underperform, but subsequent benchmarking showed it not to be an issue in reality. https://wiki.openstack.org/wiki/PyMySQL_evaluation -- Jeremy Stanley signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sqlalchemy][db][oslo.db][mistral] Is there a recommended MySQL driver for OpenStack projects?
Excerpts from Renat Akhmerov's message of 2018-07-03 18:33:44 +0700: > Hi, > > We’ve tried to address the bug [1] which is essentially caused > by the fact that we saw that MySQLDb driver wasn’t compatible with > eventlet’s green threads. In a nutshell, when we used the “eventlet” > RPC executor (see [2]), the system would get stuck once in a while > when dispatching green between green threads when it tried to hit > Mysql, but since the driver wasn’t eventlet friendly it didn’t work. > For that reason we had to use the “blocking” RPC executor so far > for Mistral Engine that deals with DB transactions. If you have a scaling issue that may be solved by eventlet, that's one thing, but please don't adopt eventlet just because a lot of other projects have. We've tried several times to minimize our reliance on eventlet because new releases tend to introduce bugs. Have you tried the 'threading' executor? Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sqlalchemy][db][oslo.db][mistral] Is there a recommended MySQL driver for OpenStack projects?
On 07/03/2018 08:47 AM, Doug Hellmann wrote: If you have a scaling issue that may be solved by eventlet, that's one thing, but please don't adopt eventlet just because a lot of other projects have. We've tried several times to minimize our reliance on eventlet because new releases tend to introduce bugs. Have you tried the 'threading' executor? +1 -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sqlalchemy][db][oslo.db][mistral] Is there a recommended MySQL driver for OpenStack projects?
> > If you have a scaling issue that may be solved by eventlet, that's > one thing, but please don't adopt eventlet just because a lot of > other projects have. We've tried several times to minimize our > reliance on eventlet because new releases tend to introduce bugs. > > Have you tried the 'threading' executor? Yes, we’re trying to solve a scaling issue. Well, I tried “threading” executor also but there’s no visible performance boost. Renat __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
