Thank you for your help!
I have another question this time:
Currently all query executed against every selected shard
sequentially. I have eight shards, and rather slow mysql servers. And
I want to implement parallel execution of this requests and combine
their answers together (аs in ShardedQuery). Is it possible?
On Feb 28, 9:53 pm, Michael Bayer <[EMAIL PROTECTED]> wrote:
> On Feb 28, 2008, at 12:54 PM, Andrew Stromnov wrote:
>
>
>
>
>
> > Thanks!
>
> > Here another issue (Session fully configured):
>
> > def shard_chooser(mapper, instance, clause=None):
> > log.warning("shard_chooser(mapper=%s, instance=%s, clause=%s)" %
> > (mapper, instance, clause))
> > return 'master_db'
> > ....
>
> > query = select([FGM.c.person_id,
> > func
> > .count
> > (FGM
> > .c
> > .person_id
> > ).label
> > ('friends')]).group_by(FGM.c.person_id).having(column('friends')>100)
> > for m in Session.execute(query):
> > print m
> > .....
>
> > 17:37:58,843 DEBUG [widgets.model] query_chooser(query='<SELECT
> > "FGM".person_id AS "FGM_person_id", "FGM".fgroup_id AS
> > "FGM_fgroup_id", "FGM".uid AS "FGM_uid"
> > FROM "FGM"
> > WHERE ("FGM".person_id = :FGM_person_id_1 OR "FGM".uid = :FGM_uid_1)
> > AND "FGM".fgroup_id IN (:FGM_fgroup_id_1, :FGM_fgroup_id_2) ORDER BY
> > "FGM".person_id>')
>
> > 17:37:58,854 DEBUG [widgets.model] shard_chooser(mapper=None,
> > instance=None, clause=SELECT "FGM".person_id, count("FGM".person_id)
> > AS friends
> > FROM "FGM" GROUP BY "FGM".person_id
> > HAVING friends > :friends_1)
>
> > 1) Why shard_chooser() executed?
>
> the ShardedSession can't execute any query without knowing which shard
> it should be querying against. If you send the shard_id=<someshard>
> argument to Session.execute(), shard_chooser() wont be called.
>
> > 2) Why shard_chooser() executed with first two None arguments?
>
> the first two arguments are a mapper and an instance. the mapper
> argument is available when you say session.query(SomeObject).....,
> because we know the mapper is against "SomeObject". The
> session.execute() method has an optional argument "mapper" which you
> can use to send hints to shard_chooser(). The "instance" argument is
> sent typically at flush time, when the mapper is iterating through
> instances to be saved; each instance is sent to this function so that
> its known which shard each instance should be persisted towards. So
> neither argument applies here, since you are executing just a plain
> SQL statement with no mapper.
>
> > I want to execute this query against several shards (yes, with per-
> > shard grouping)
>
> you should call execute() separately for each shard. You can send the
> shard_id=<someshard> argument to Session.execute() or
> Session.connection() to control the shard chosen. the aggregating of
> multiple results only occurs within
> Query._execute_and_instances...while we will have a future release
> that allows column-based queries to happen via Query, its still not
> doing anything like GROUP BY or ORDER BY, it just lumps each result
> set into a single list...so in any case you'd have to grab common
> person_id's from each result set and group them together yourself (and
> probably do the HAVING thing in memory too).
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~----------~----~----~----~------~----~------~--~---
