Thanks for those debug tips.
It doesn't seem to be a celery related problem.
Sessions are initiated after the fork and SQLAlchemy is not used as the
messaging backend (We use Redis).
Playing with the pool_recycle option in the Pyramid configuration I
clearly limited the number of exceptions.
But I still have some transaction related errors.
I'll continue my investigations and come back when I'll found the cause(s).
Le 17/04/2019 à 16:26, Jonathan Vanasco a écrit :
On Wednesday, April 17, 2019 at 4:36:30 AM UTC-4, tonthon wrote:
May the scoped_session factory, used in both services (web and
celery), may cause such a problem ?
Apologies for my earlier reply, that would only affect exceptions in
Celery. I do suggest checking your Celery code to make sure you are
not connecting to the database before the fork - Celery does run as
multi-processing by default.
Celery can potentially use SqlAlchemy in two places: the messaging
backend and the worker task runners. If you are using SqlAlchemy for
the messaging backend, that is potentially the cause. If you are
using another technology (Redis/Rabbit/etc) for messaging, Pyramid
should never be invoking the scoped_session for Celery. In either
situation, you can drop some debug lines around when the DB connection
is made to ensure you're connecting at the right time.
I've debugged issues in this area with PostgreSQL by enabling query
logging and including the transaction id, then looking backwards at
each error to find the query that broke the connection's state. If
your issue happens frequently enough, an hour or two of logging might
be enough to show you what happened.
--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper
http://www.sqlalchemy.org/
To post example code, please provide an MCVE: Minimal, Complete, and
Verifiable Example. See http://stackoverflow.com/help/mcve for a full
description.
---
You received this message because you are subscribed to the Google
Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to sqlalchemy+unsubscr...@googlegroups.com
<mailto:sqlalchemy+unsubscr...@googlegroups.com>.
To post to this group, send email to sqlalchemy@googlegroups.com
<mailto:sqlalchemy@googlegroups.com>.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper
http://www.sqlalchemy.org/
To post example code, please provide an MCVE: Minimal, Complete, and Verifiable
Example. See http://stackoverflow.com/help/mcve for a full description.
---
You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
