Re: [sqlalchemy] Multiple sessions same thread - How to?

2018-06-25 Thread Mike Bayer
On Mon, Jun 25, 2018 at 6:24 PM, HP3 wrote: > Thank you both > > With `before_cursor_execute` I am able to log each Connection, Cursor and > context instance. > > I see that every worker is doing its thing and the Connection instance is > always different for each SQL statement > > Also, each SQL

Re: [sqlalchemy] Multiple sessions same thread - How to?

2018-06-25 Thread HP3
Thank you both With `before_cursor_execute` I am able to log each Connection, Cursor and context instance. I see that every worker is doing its thing and the Connection instance is always different for each SQL statement Also, each SQL has its own distinct

Re: [sqlalchemy] Multiple sessions same thread - How to?

2018-06-25 Thread Mike Bayer
you might want to build out events to log exactly what you need and where: http://docs.sqlalchemy.org/en/latest/core/events.html#sql-execution-and-connection-events use your own logger, look at the connection / engine / current thread / etc. for whatever you need to keep track of. On Mon, Jun

Re: [sqlalchemy] Multiple sessions same thread - How to?

2018-06-25 Thread HP3
> > @HP3 just to test this, i would try adding a slightly different connection > string or argument to the celery connection. e.g. create a different user, > or toss in a config argument that doesn't affect your code. if the error > stops, that's most-likely the reason why. > I'll try that

Re: [sqlalchemy] Multiple sessions same thread - How to?

2018-06-25 Thread HP3
Thank you Mike! That's very useful. I can see which engine & worker is emitting each SQL statement (I had to bypass the usage of 'pyramid_celery` so all logging settings would "stick"). *Is there a way to identify every engine instance?* *Every celery worker is creating its own

Re: [sqlalchemy] Multiple sessions same thread - How to?

2018-06-25 Thread Jonathan Vanasco
On Monday, June 25, 2018 at 11:31:07 AM UTC-4, HP3 wrote: > > I'm confused about what you said about the underlined connection: I am > creating 2 different engines. Why would both share the same connection? > > That wasn't clear from the above, however.. looking at the code you've shared, it

Re: [sqlalchemy] Multiple sessions same thread - How to?

2018-06-25 Thread Mike Bayer
On Mon, Jun 25, 2018 at 11:31 AM, HP3 wrote: > I'm confused about what you said about the underlined connection: I am > creating 2 different engines. Why would both share the same connection? just as a note, if you do have two different engines, you can apply an individual logging name to each:

Re: [sqlalchemy] Multiple sessions same thread - How to?

2018-06-25 Thread HP3
> > > I verified that the failure checkpoint is not being persisted either when I raise an exception after 'do something' checkpoint. -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal,

Re: [sqlalchemy] Multiple sessions same thread - How to?

2018-06-25 Thread HP3
I'm confused about what you said about the underlined connection: I am creating 2 different engines. Why would both share the same connection? FYI - I just tried without zope.sqlalchemy and got a slightly different result: My concurrent transactions are no longer failing with

Re: [sqlalchemy] Multiple sessions same thread - How to?

2018-06-25 Thread HP3
Thank you Jonathan, This is how I create both engine/session pairs: from sqlalchemy import engine_from_config from sqlalchemy.orm import sessionmaker import zope.sqlalchemy class DBTask(app.task) _engine = None _domain_model_session = None def checkpoint(self, label): *#