On Thu, Feb 22, 2018 at 6:28 AM, Антонио Антуан <a.ch....@gmail.com> wrote: > Hello Mike. > Finally got to this. Works perfectly with synchronize_session equals False > or 'evaluate'. > > Got a problem when synchronize_session='fetch'. In that case used > BulkFetch._do_pre_synchronize (as you know :)). There is query instance with > set `_bind_id` attribute, but it don't passed through `execute`. > > Here is an example that can illustrate that problem: > https://gist.github.com/aCLr/a992ca92138aeee86bf9432693be6d6c > It contains Query and Session implementations, which used in my project: > MultiBoundQuery and MultiBoundSession. > Also, it contains `delete` and `update` execution and checking for update > and delete operation. Tracebacks included.
continuing this at https://bitbucket.org/zzzeek/sqlalchemy/issues/4196/support-sharded-query-for-bulk-update , the gerrit needs to have tests for these use cases. > > > > вт, 13 февр. 2018 г. в 20:37, Антонио Антуан <a.ch....@gmail.com>: >> >> I understand. Excuse me, drown in work. Going to test your code in a >> couple of days. >> >> >> вт, 13 февр. 2018 г., 20:32 Mike Bayer <mike...@zzzcomputing.com>: >>> >>> i don't plan to move on this until I get feedback. >>> >>> On Tue, Feb 6, 2018 at 7:32 PM, Mike Bayer <mike...@zzzcomputing.com> >>> wrote: >>> > code review at >>> > https://gerrit.sqlalchemy.org/#/c/zzzeek/sqlalchemy/+/656 >>> > and ideally would include basic ShardedQuery support >>> > >>> > On Tue, Feb 6, 2018 at 7:23 PM, Mike Bayer <mike...@zzzcomputing.com> >>> > wrote: >>> >> On Tue, Feb 6, 2018 at 5:09 PM, Антонио Антуан <a.ch....@gmail.com> >>> >> wrote: >>> >>> Hello Mike! >>> >>> First of all, thank you for your help with this problem. Me and my >>> >>> crew >>> >>> appreciate it. >>> >>> >>> >>> >>> >>> I have a question in case of `identity_token` for `update` and >>> >>> `delete` >>> >>> methods of `Query` instances. >>> >>> >>> >>> If we take a look on sqlalchemy.orm.persistence.BulkDelete._do_exec, >>> >>> we see, >>> >>> that there is no additonal kwargs passed to `execute` method. So I >>> >>> don't see >>> >>> any way to pass additional kwargs to Session.get_bind. Here is code >>> >>> of 1.0 >>> >>> version: >>> >>> >>> >>> def _do_exec(self): >>> >>> delete_stmt = sql.delete(self.primary_table, >>> >>> self.context.whereclause) >>> >>> >>> >>> self.result = self.query.session.execute( >>> >>> delete_stmt, >>> >>> params=self.query._params, >>> >>> mapper=self.mapper, >>> >>> # need to pass here additional kwargs >>> >>> ) >>> >>> self.rowcount = self.result.rowcount >>> >>> >>> >>> Code from master branch (almost the same): >>> >>> >>> >>> def _execute_stmt(self, stmt): >>> >>> self.result = self.query.session.execute( >>> >>> stmt, params=self.query._params, >>> >>> mapper=self.mapper) >>> >>> self.rowcount = self.result.rowcount >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> As you may remember, I have some kind of `CustomQuery` with >>> >>> `set_bind` >>> >>> method (like the same into >>> >>> sqlalchemy.ext.horizontal_shard.ShardedQuery). So >>> >>> I have `_bind_id` attribute within instances of that class. >>> >>> My question is: Is there any way to pass `self.query._bind_id` to >>> >>> `self.query.session.execute` (see above)? >>> >> >>> >> for bulk operations to work with sharding it would need to be able to >>> >> emit the UPDATE or DELETE statement across multiple shards and work in >>> >> a similar way to ShardedQuery._execute_and_instances, giving it the >>> >> chance to consult with query_chooser. >>> >> >>> >> can you please see if you can work with this? thanks >>> >> >>> >> diff --git a/lib/sqlalchemy/orm/persistence.py >>> >> b/lib/sqlalchemy/orm/persistence.py >>> >> index dc0ae1c38..6c55dee92 100644 >>> >> --- a/lib/sqlalchemy/orm/persistence.py >>> >> +++ b/lib/sqlalchemy/orm/persistence.py >>> >> @@ -1327,9 +1327,8 @@ class BulkUD(object): >>> >> self._do_post() >>> >> >>> >> def _execute_stmt(self, stmt): >>> >> - self.result = self.query.session.execute( >>> >> - stmt, params=self.query._params, >>> >> - mapper=self.mapper) >>> >> + self.result = self.query._execute_crud(stmt, self.mapper) >>> >> + >>> >> self.rowcount = self.result.rowcount >>> >> >>> >> @util.dependencies("sqlalchemy.orm.query") >>> >> diff --git a/lib/sqlalchemy/orm/query.py b/lib/sqlalchemy/orm/query.py >>> >> index 4f7b22cc9..91adfb79d 100644 >>> >> --- a/lib/sqlalchemy/orm/query.py >>> >> +++ b/lib/sqlalchemy/orm/query.py >>> >> @@ -2901,6 +2901,12 @@ class Query(object): >>> >> result = conn.execute(querycontext.statement, self._params) >>> >> return loading.instances(querycontext.query, result, >>> >> querycontext) >>> >> >>> >> + def _execute_crud(self, stmt, mapper): >>> >> + conn = self._connection_from_session( >>> >> + mapper=mapper, clause=stmt, close_with_result=True) >>> >> + >>> >> + return conn.execute(stmt, self._params) >>> >> + >>> >> def _get_bind_args(self, querycontext, fn, **kw): >>> >> return fn( >>> >> mapper=self._bind_mapper(), >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >>> >>> >>> -- >>> >>> SQLAlchemy - >>> >>> The Python SQL Toolkit and Object Relational Mapper >>> >>> >>> >>> http://www.sqlalchemy.org/ >>> >>> >>> >>> To post example code, please provide an MCVE: Minimal, Complete, and >>> >>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full >>> >>> description. >>> >>> --- >>> >>> You received this message because you are subscribed to the Google >>> >>> Groups >>> >>> "sqlalchemy" group. >>> >>> To unsubscribe from this group and stop receiving emails from it, >>> >>> send an >>> >>> email to sqlalchemy+unsubscr...@googlegroups.com. >>> >>> To post to this group, send email to sqlalchemy@googlegroups.com. >>> >>> Visit this group at https://groups.google.com/group/sqlalchemy. >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >>> -- >>> SQLAlchemy - >>> The Python SQL Toolkit and Object Relational Mapper >>> >>> http://www.sqlalchemy.org/ >>> >>> To post example code, please provide an MCVE: Minimal, Complete, and >>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full >>> description. >>> --- >>> You received this message because you are subscribed to the Google Groups >>> "sqlalchemy" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to sqlalchemy+unsubscr...@googlegroups.com. >>> To post to this group, send email to sqlalchemy@googlegroups.com. >>> Visit this group at https://groups.google.com/group/sqlalchemy. >>> For more options, visit https://groups.google.com/d/optout. >> >> >> >> -- >> >> Антон > > -- > > Антон > > -- > SQLAlchemy - > The Python SQL Toolkit and Object Relational Mapper > > http://www.sqlalchemy.org/ > > To post example code, please provide an MCVE: Minimal, Complete, and > Verifiable Example. See http://stackoverflow.com/help/mcve for a full > description. > --- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sqlalchemy+unsubscr...@googlegroups.com. > To post to this group, send email to sqlalchemy@googlegroups.com. > Visit this group at https://groups.google.com/group/sqlalchemy. > For more options, visit https://groups.google.com/d/optout. -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
