I've never used celery, but I believe it can fork: http://docs.celeryproject.org/en/latest/userguide/workers.html#concurrency
Is it possible that connections are being shared between workers, and if so could that potentially cause this problem? Simon On Thu, Sep 21, 2017 at 3:11 PM, Mike Bayer <mike...@zzzcomputing.com> wrote: > On Thu, Sep 21, 2017 at 7:27 AM, Ergo <erg...@gmail.com> wrote: >> We are using: >> >> mysql-connector-python==2.1.6 >> SQLAlchemy==1.1.13 >> >> I'm getting following exception in our celery tasks, it appears to happen >> when we do query.first(). > > what normally makes this happen is that you run a SELECT and then > cursor.description is None. We use the presense of > cursor.description to tell us if this result has rows to return or not > and if not, we close the cursor (this autoclose otherwise occurs after > you exhaust all the rows). > > We have seen issues with drivers in the past having this problem. > Specifically we saw it using eventlet / gevent, I wrote a > demonstration case at > https://gist.github.com/zzzeek/b762218a5f855ec7d1c3a998fe996893. the > idea there is that greenlets are being killed randomly. if a database > connection is in process while this kill happens, and then gets > returned to the connection pool, MySQL's protocol is casual enough > that it lets you emit a new statement on the connection even though > the conversation is totally broken, then you don't get the result back > correctly. That specific issue however was resolved long ago in > 1.1.0: > https://docs.sqlalchemy.org/en/latest/changelog/changelog_11.html#change-4667b5bd5cde7049b907f70614059d00 > . > > If Celery tasks are timing out in some way that the connection is > interrupted and then re-used without a BaseException or other known > MySQL disconnect exception of some kind being thrown, that might be > the source of this issue. > > > > > >> >> File points2shop/lib/tasks.py line 2362 in offer_federated_sync: >> offer = Offer.get_offer(offer_id) >> File points2shop/model/offer.py line 2365 in get_offer: >> return offer.first() >> File sqlalchemy/orm/query.py line 2755 in first: >> ret = list(self[0:1]) >> File sqlalchemy/orm/query.py line 2547 in __getitem__: >> return list(res) >> File sqlalchemy/orm/loading.py line 90 in instances: >> util.raise_from_cause(err) >> File sqlalchemy/util/compat.py line 203 in raise_from_cause: >> reraise(type(exception), exception, tb=exc_tb, cause=cause) >> File sqlalchemy/orm/loading.py line 57 in instances: >> for query_entity in query._entities >> File sqlalchemy/orm/query.py line 3700 in row_processor: >> polymorphic_discriminator=self._polymorphic_discriminator >> File sqlalchemy/orm/loading.py line 319 in _instance_processor: >> getter = result._getter(col, False) >> File sqlalchemy/engine/result.py line 659 in _getter: >> return self._non_result(None) >> File sqlalchemy/engine/result.py line 1077 in _non_result: >> "This result object does not return rows. " >> ResourceClosedError: This result object does not return rows. It has >> been closed automatically. >> >> >> Any common causes for this to happen? >> >> -- >> SQLAlchemy - >> The Python SQL Toolkit and Object Relational Mapper >> >> http://www.sqlalchemy.org/ >> >> To post example code, please provide an MCVE: Minimal, Complete, and >> Verifiable Example. See http://stackoverflow.com/help/mcve for a full >> description. >> --- >> You received this message because you are subscribed to the Google Groups >> "sqlalchemy" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to sqlalchemy+unsubscr...@googlegroups.com. >> To post to this group, send email to sqlalchemy@googlegroups.com. >> Visit this group at https://groups.google.com/group/sqlalchemy. >> For more options, visit https://groups.google.com/d/optout. > > -- > SQLAlchemy - > The Python SQL Toolkit and Object Relational Mapper > > http://www.sqlalchemy.org/ > > To post example code, please provide an MCVE: Minimal, Complete, and > Verifiable Example. See http://stackoverflow.com/help/mcve for a full > description. > --- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sqlalchemy+unsubscr...@googlegroups.com. > To post to this group, send email to sqlalchemy@googlegroups.com. > Visit this group at https://groups.google.com/group/sqlalchemy. > For more options, visit https://groups.google.com/d/optout. -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
