On 7/11/14, 7:26 PM, Carl Baldwin wrote: > > > On Jul 11, 2014 5:32 PM, "Vishvananda Ishaya" <vishvana...@gmail.com <mailto:vishvana...@gmail.com>> wrote: > > > > I have tried using pymysql in place of mysqldb and in real world concurrency > > tests against cinder and nova it performs slower. I was inspired by the mention > > of mysql-connector so I just tried that option instead. Mysql-connector seems > > to be slightly slower as well, which leads me to believe that the blocking inside of > > Do you have some numbers? "Seems to be slightly slower" doesn't really stand up as an argument against the numbers that have been posted in this thread. > > > sqlalchemy is not the main bottleneck across projects. > > > > Vish > > > > P.S. The performanace in all cases was abysmal, so performance work definitely > > needs to be done, but just the guess that replacing our mysql library is going to > > solve all of our performance problems appears to be incorrect at first blush. > > The motivation is still mostly deadlock relief but more performance work should be done. I agree with you there. I'm still hopeful for some improvement from this.
To identify performance that's alleviated by async you have to establish up front that IO blocking is the issue, which would entail having code that's blazing fast until you start running it against concurrent connections, at which point you can identify via profiling that IO operations are being serialized. This is a very specific issue. In contrast, to identify why some arbitrary openstack app is slow, my bet is that async is often not the big issue. Every day I look at openstack code and talk to people working on things, I see many performance issues that have nothing to do with concurrency, and as I detailed in my wiki page at https://wiki.openstack.org/wiki/OpenStack_and_SQLAlchemy there is a long road to cleaning up all the excessive queries, hundreds of unnecessary rows and columns being pulled over the network, unindexed lookups, subquery joins, hammering of Python-intensive operations (often due to the nature of OS apps as lots and lots of tiny API calls) that can be cached. There's a clear path to tons better performance documented there and most of it is not about async - which means that successful async isn't going to solve all those issues.
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
