Re: [sqlalchemy] Multiple sessions same thread - How to?

2018-06-22 Thread Jonathan Vanasco
Can you share/show how/where they engines and connections are created? This is odd. FWIW, with the forking bug- the issue isn't in the transaction/session but in the underlying database connections. The SqlAlchemy connection pool isn't threadsafe, so all the commits/rollbacks/etc in different

Re: [sqlalchemy] Multiple sessions same thread - How to?

2018-06-22 Thread HP3
No dice! I verified that engines and sessions are created after fork. By hijacking celery logging, verified each worker had its own transaction and session. The SQL logs I described above are indeed accurate and belong to the same worker. -- SQLAlchemy - The Python SQL Toolkit and Object

Re: [sqlalchemy] Multiple sessions same thread - How to?

2018-06-22 Thread Jonathan Vanasco
On Friday, June 22, 2018 at 1:32:15 PM UTC-4, HP3 wrote:... but I'll 2x check! > (I recall that task-inheritance in celery makes certain things happen > before and others after the fork - I am using prefork) > i don't use pyramid_celery, but my own pyramid and celery integration...

Re: [sqlalchemy] Re: Duplicate `self`, without affecting the original instance in memory…

2018-06-22 Thread Mike Bayer
There's no shortcut outside of merge () for copying attributes, copies are hard to generalize so you probably want to put copy constructors on your classes if you need to do that often. On Fri, Jun 22, 2018, 1:24 PM Jonathan Vanasco wrote: > > FWIW, i use these methods in my base class to copy

Re: [sqlalchemy] Multiple sessions same thread - How to?

2018-06-22 Thread HP3
> > Is there a chance there is a query/connection being made between the > initialization and worker process? > Hmmm ... I am using celery task inheritance (http://docs.celeryproject.org/en/latest/userguide/tasks.html#task-inheritance) with `@property`s for domain model engine and domain

[sqlalchemy] Re: Duplicate `self`, without affecting the original instance in memory…

2018-06-22 Thread Jonathan Vanasco
FWIW, i use these methods in my base class to copy the object into a dict. def columns_as_dict(self): """ Beware: this function will trigger a load of attributes if they have not been loaded yet. """ return dict((col.name, getattr(self, col.name))

Re: [sqlalchemy] Multiple sessions same thread - How to?

2018-06-22 Thread Jonathan Vanasco
Is there a chance there is a query/connection being made between the initialization and worker process? If so, that could screw up the connection pool. To address that, you can try adding an `engine.dispose()` before celery forks. I'll take a look at the code later. I was home sick this

Re: [sqlalchemy] test/dialect/test_sqlite.py::TypeReflectionTest::()::test_round_trip_direct_type_affinity fails

2018-06-22 Thread Mike Bayer
This PR resolves: https://github.com/zzzeek/sqlalchemy/pull/452 On Fri, Jun 22, 2018, 7:03 AM Hans-Peter Jansen wrote: > Hi %, > > packaging 1.2.8 (and before) fails for openSUSE Tumbleweed, while it > succeeds > for many former distributions due to a single failing test with Python > 2.7.15, >

Re: [sqlalchemy] Queries comparison

2018-06-22 Thread Mike Bayer
Traverse each statement using visitors.traverse() is one way, however your best approach is to get that string to be deterministic. Where you generate that where clause, make sure you use either ordered dictionaries / sets or perform a sorted() on the records you send into and_(), ordering by

Re: [sqlalchemy] How to use labels directly in compare expression

2018-06-22 Thread Mike Bayer
Just use literal_column for now , that is the clearest statement of intention , I'll try to find time to look at the code to see if there's some flag I'm forgetting On Fri, Jun 22, 2018, 3:08 AM wrote: > Yes, I tried that. It produces the following SQL: > > SELECT json_value(user.meta,

Re: [sqlalchemy] Multiple sessions same thread - How to?

2018-06-22 Thread HP3
I added logging to `session.transaction` and `session.transaction.parent` (til parent=None) and `session.transaction.nested` and they all seemed to be different, never nested, never subtransaction. No hint there. I modified my code slightly by adding a new

[sqlalchemy] Bulk insert using sql alchemy

2018-06-22 Thread siddhesh
Hi, I want to insert 4 records in single query using connection.execute() in sql alchemy. I am using sql alchemy + cx_oracle My sql staement is: INSERT INTO orders (ORDER_ID, ORDER_NO, CUSTOMER_ID, ORDER_TOTAL) (?,?,?,?) VALUES ((1, '232323', 1, 300),(2, '232323', 1, 300), (3, '232323', 1,

Re: [sqlalchemy] Multiple sessions same thread - How to?

2018-06-22 Thread HP3
Thank you Jonathan! I checked if my celery_state_session had been mistakenly connected to zope.sqlalchemy as I am using pyramid_celery. I didn't find anything. On the other hand, `domain_model_session` is wired to transaction.manager by zope.sqlalchemy (zope.sqlalchemy.registry()). I even

[sqlalchemy] test/dialect/test_sqlite.py::TypeReflectionTest::()::test_round_trip_direct_type_affinity fails

2018-06-22 Thread Hans-Peter Jansen
Hi %, packaging 1.2.8 (and before) fails for openSUSE Tumbleweed, while it succeeds for many former distributions due to a single failing test with Python 2.7.15, sqlite 3.24, pytest 3.6.0: FAIL test/dialect/test_sqlite.py::TypeReflectionTest:: ()::test_round_trip_direct_type_affinity Full

[sqlalchemy] Queries comparison

2018-06-22 Thread Антонио Антуан
Hi there! I have a very complicated queries generator and I want to test it. Currently it looks like that: def test(): query = generate_query() compiled = compile_query(query) assert compiled == 'SELECT * FROM table' Problems begin when I add some filters to query: def test():

[sqlalchemy] Duplicate `self`, without affecting the original instance in memory…

2018-06-22 Thread Dave von Umlaut
I am trying to make a duplicate (with new PK) of an instance *from within* the instance. All the solutions I have found so far, using `make_transient(self)` or `session.merge`, change the original instance in memory, rather than create a separate object, and I cannot figure a way to keep the

Re: [sqlalchemy] How to use labels directly in compare expression

2018-06-22 Thread yoch . melka
In standard SQL, I might use a subquery with filter: code = func.json_value(User.meta, '$.code').label('code') q = db.query(code).subquery() db.query(q).filter(q.c.code!=None).all() Le vendredi 22 juin 2018 10:08:04 UTC+3, yoch@gmail.com a écrit : > > Yes, I tried that. It produces the

Re: [sqlalchemy] How to use labels directly in compare expression

2018-06-22 Thread yoch . melka
Yes, I tried that. It produces the following SQL: SELECT json_value(user.meta, :json_value_1) AS code FROM user HAVING json_value(user.meta, :json_value_1) IS NOT NULL This make sense in general, because in MySQL you cannot refer to an alias in the WHERE part. But I want to refer directly to