On Thu, Sep 21, 2017 at 3:46 PM, Ergo <erg...@gmail.com> wrote: > Thanks, we had errors with: > > "InternalError: (mysql.connector.errors.InternalError) Unread result found" > and "ResourceClosedError: This result object does not return rows. It has > been closed automatically." exceptions. > > I changed our driver back to old MySQL-Python and the problem is now > completly gone. > > That would hint a bug in oracle's driver correct? Maybe we can try some > other options (ideally with C extensions foe perfrormance).
This doesn't necessarily mean there's a bug in mysql-connector. MySQL-Python is written in C so in the case I illustrated with gevent, that doesn't happen because the socket internals can't be monkeypatched. In the case of adjusting for a connection that is shared among forks, there may be some other aspect of MySQL-Python preventing that from being an issue, but it is not uncommon that sharing the socket across forks causes problems regardless. For the Celery code you should ensure that any new fork has a clean engine within it, you should dispose() the engine at the beginning of the new process (see http://docs.sqlalchemy.org/en/latest/core/connections.html#engine-disposal). I would advise trying out the PyMySQL driver which will indicate more closely if the problem is specific to mysql-connector (if PyMySQL does not have the problem) or specific to pure-Python (if PyMySQL does have the problem). > > I think you mentioned to me in the past that pymysql is a decent option, > ideally the behavior should match MySQL-Python when it comes to type > handling. > > I should mention that we do not use gevent/eventlet with celery. > > > W dniu czwartek, 21 września 2017 16:12:08 UTC+2 użytkownik Mike Bayer > napisał: >> >> On Thu, Sep 21, 2017 at 7:27 AM, Ergo <erg...@gmail.com> wrote: >> > We are using: >> > >> > mysql-connector-python==2.1.6 >> > SQLAlchemy==1.1.13 >> > >> > I'm getting following exception in our celery tasks, it appears to >> > happen >> > when we do query.first(). >> >> what normally makes this happen is that you run a SELECT and then >> cursor.description is None. We use the presense of >> cursor.description to tell us if this result has rows to return or not >> and if not, we close the cursor (this autoclose otherwise occurs after >> you exhaust all the rows). >> >> We have seen issues with drivers in the past having this problem. >> Specifically we saw it using eventlet / gevent, I wrote a >> demonstration case at >> https://gist.github.com/zzzeek/b762218a5f855ec7d1c3a998fe996893. the >> idea there is that greenlets are being killed randomly. if a database >> connection is in process while this kill happens, and then gets >> returned to the connection pool, MySQL's protocol is casual enough >> that it lets you emit a new statement on the connection even though >> the conversation is totally broken, then you don't get the result back >> correctly. That specific issue however was resolved long ago in >> 1.1.0: >> https://docs.sqlalchemy.org/en/latest/changelog/changelog_11.html#change-4667b5bd5cde7049b907f70614059d00 >> . >> >> If Celery tasks are timing out in some way that the connection is >> interrupted and then re-used without a BaseException or other known >> MySQL disconnect exception of some kind being thrown, that might be >> the source of this issue. >> >> >> >> >> >> > >> > File points2shop/lib/tasks.py line 2362 in offer_federated_sync: >> > offer = Offer.get_offer(offer_id) >> > File points2shop/model/offer.py line 2365 in get_offer: >> > return offer.first() >> > File sqlalchemy/orm/query.py line 2755 in first: >> > ret = list(self[0:1]) >> > File sqlalchemy/orm/query.py line 2547 in __getitem__: >> > return list(res) >> > File sqlalchemy/orm/loading.py line 90 in instances: >> > util.raise_from_cause(err) >> > File sqlalchemy/util/compat.py line 203 in raise_from_cause: >> > reraise(type(exception), exception, tb=exc_tb, cause=cause) >> > File sqlalchemy/orm/loading.py line 57 in instances: >> > for query_entity in query._entities >> > File sqlalchemy/orm/query.py line 3700 in row_processor: >> > polymorphic_discriminator=self._polymorphic_discriminator >> > File sqlalchemy/orm/loading.py line 319 in _instance_processor: >> > getter = result._getter(col, False) >> > File sqlalchemy/engine/result.py line 659 in _getter: >> > return self._non_result(None) >> > File sqlalchemy/engine/result.py line 1077 in _non_result: >> > "This result object does not return rows. " >> > ResourceClosedError: This result object does not return rows. It has >> > been closed automatically. >> > >> > >> > Any common causes for this to happen? >> > >> > -- >> > SQLAlchemy - >> > The Python SQL Toolkit and Object Relational Mapper >> > >> > http://www.sqlalchemy.org/ >> > >> > To post example code, please provide an MCVE: Minimal, Complete, and >> > Verifiable Example. See http://stackoverflow.com/help/mcve for a full >> > description. >> > --- >> > You received this message because you are subscribed to the Google >> > Groups >> > "sqlalchemy" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> > an >> > email to sqlalchemy+...@googlegroups.com. >> > To post to this group, send email to sqlal...@googlegroups.com. >> > Visit this group at https://groups.google.com/group/sqlalchemy. >> > For more options, visit https://groups.google.com/d/optout. > > > -- > SQLAlchemy - > The Python SQL Toolkit and Object Relational Mapper > > http://www.sqlalchemy.org/ > > To post example code, please provide an MCVE: Minimal, Complete, and > Verifiable Example. See http://stackoverflow.com/help/mcve for a full > description. > --- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sqlalchemy+unsubscr...@googlegroups.com. > To post to this group, send email to sqlalchemy@googlegroups.com. > Visit this group at https://groups.google.com/group/sqlalchemy. > For more options, visit https://groups.google.com/d/optout. -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
