OK, I have to admit that I haven't fully understood the details of what you are doing. However, it sounds like you are implementing horizontal sharding, which SQLAlchemy has a bit of support for:
http://docs.sqlalchemy.org/en/latest/orm/extensions/horizontal_shard.html http://docs.sqlalchemy.org/en/latest/_modules/examples/sharding/attribute_shard.html In the example, the query_chooser function is inspecting the query to see which backend database the query should be issued against, which I guess is what your "_analyze_base_query" method does. Would this work for you? Simon On Tue, Jul 12, 2016 at 3:03 PM, Антонио Антуан <a.ch....@gmail.com> wrote: > Ok, I define my situation. > At first, I'm appologizing for my accent :) > > I have two databases, postgresql and vertica. > I have some amount of large tables, besides other tables. > > Postgres stores only last two hours (or last week, depends on table) of > large tables, vertica stores all other data. > > I implemented SplittedDataQuery, subclass of Query, which used with my > Session like query_cls. > The subclass overrides __iter__(), all(), count(), first(), one() methods. > It has attribute _base_query, which refers on self, _pg_query and > _vertica_query. > > Before invoking them, SplittedDataQuery checks necessity (by whereclause > inside query and `splitting_time` - moment when the data splitted between > databases) of getting data only from Pg, or only from vertica, or from both > with data union. > > Data from vertica returns by _vertica_query, data from postgres - by > _pg_query > _vertica_query binds with SessionVertica. > all other queries (_pg_query and _base_query) binds with Session > > Function, creates SessionVertica, has nothing unusual. > > Session for pg creates like this: > def _create_session(self, conn_string, splitted_query_cls=True): > engine = create_engine(conn_string) > > from core.splitted_data import SplittedDataQuery > session = ScopedSession(sessionmaker(bind=engine, > query_cls=SplittedDataQuery if splitted_query_cls else Query)) # query_cls > choosing only for SplittedDataQuery._pg_query > > return session > > > There is overrided all(): > def all(self): > return self._get_result() > > Implementation of _get_result() looks like this: > def _get_result(self): > self._analyze_base_query() > self._split_query() > self._initialize_db_queries() > result = self._vertica_query.all() > if self._needs_to_get_from_pg: > result.extend(self._pg_query.all()) > > _db_queries initialization inside SplittedDataQuery class: > def __init_pg_query(self): > original_session = > _create_session(config['sqlalchemy']['conn_string'], > splitted_query_cls=False) > self._pg_query = self._query.filter() \ > .with_session(original_session) > > > def __init_vertica_query(self): > from hasoffers.core.model.meta import SessionVertica > self._vertica_query = self._query.filter() \ > .with_session(SessionVertica) > > If I don't create _pg_query using splitted_query_cls=False (with std Query > class), then _pg_query invokation of methods (all(), one() and other) will > always apply overrided methods (because it uses SplittedDataQuery). > But, if I create _pg_query by defined way, it creates new session and, so, > new connection to Postgres. > > As you see, I don't want nor the first, neither the second option :) > вторник, 12 июля 2016 г., 13:35:48 UTC+3 пользователь Simon King написал: >> >> Could you describe what you are trying to achieve? There's nothing >> about Mike's suggestion that means you need to create a new session - >> you can reuse any existing session. >> >> What does your CustomQueryCls do? Perhaps there's another way of doing >> what you want? >> >> Simon >> >> On Tue, Jul 12, 2016 at 11:09 AM, Антонио Антуан <a.ch...@gmail.com> >> wrote: >> > But it means that I should create one more session with one more >> > connection >> > to DB. This is not good for me :( >> > >> > понедельник, 11 июля 2016 г., 22:23:11 UTC+3 пользователь Антонио Антуан >> > написал: >> >> >> >> Can I specify query_cls only for a one query? I try to change >> >> query.session._query_cls and, of course, it doesn't work... >> >> >> >> I want to perform something like this: >> >> >> >> query = Session.query(MyModel)... >> >> default_querycls_result = query.all() >> >> custom_querycls_result = query.change_query_cls(CustomQueryCls).all() >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups >> > "sqlalchemy" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> > an >> > email to sqlalchemy+...@googlegroups.com. >> > To post to this group, send email to sqlal...@googlegroups.com. >> > Visit this group at https://groups.google.com/group/sqlalchemy. >> > For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sqlalchemy+unsubscr...@googlegroups.com. > To post to this group, send email to sqlalchemy@googlegroups.com. > Visit this group at https://groups.google.com/group/sqlalchemy. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
