You really did great job. Regarding sqlalchemy is so great a piece of work,
sharding deserves more love. I'm myself only touch the surface of it in a
real project, maybe I can feedback more or contribute some code if I can
write the right one:)
On Thu, Apr 21, 2011 at 9:23 AM, Michael Bayer <mike...@zzzcomputing.com>wrote:
> ah well, glad to hear that ! It certainly can be more of a core element
> if it had some more dedicated maintainers. I haven't actually used it in a
> real project so i can't vouch strongly for it. It also might be a nice
> third-party project. Sharding is very tough and there's lots of complex
> cases that come up pretty fast.
>
>
> On Apr 20, 2011, at 8:05 PM, can xiang wrote:
>
> Thanks anyway.
>
> It's sad horizontal shard extension is only considered as a example. I
> hardly believe it, because it works so great in some of my simple use case.
> I really hope there would be more work on this extension or more docs on how
> to do it.
>
> Best regards!
> can
>
> On Thu, Apr 21, 2011 at 3:19 AM, Michael Bayer
> <mike...@zzzcomputing.com>wrote:
>
>>
>> On Apr 20, 2011, at 4:37 AM, can xiang wrote:
>>
>> > Hi,
>> >
>> > I have a problem of bulk deleting from a sharded session, I tried to
>> > search the previous posts without any finding.
>> >
>> > I have the following table:
>> >
>> > usersession_table = Table('kg_usersession', meta,
>> > Column('session_id', String(32),
>> > primary_key=True),
>> > Column('user_id', BigInteger, index=True,
>> > nullable=False),
>> > Column('create_time', DateTime, index=True),
>> > Column('expire_time', DateTime, index=True),
>> > Column('site', String(10)),
>> > mysql_engine='MyISAM'
>> > )
>> >
>> > I use horizontal sharding by "session_id", with the following shard
>> > chooser:
>> >
>> > def shard_chooser(mapper, instance, clause=None):
>> > if instance:
>> > return shard_value(instance.session_id)
>> >
>> > Then, I want to delete all record earlier than a given expire_time,
>> > with the following code:
>> >
>> >
>> session.query(UserSession).filter(UserSession.expire_time<=expire_time).delete();
>>
>> range deletions are not supported by the ShardedSession extension right
>> now. You'd need to implement your own delete() onto ShardedQuery.
>>
>> Note that the horizontal shard extension really should have been an
>> example, not a full extension. It's really just a proof of concept and
>> real-world horizontal sharding scenarios will usually need to tweak it for
>> specific use cases.
>>
>>
>>
>>
>>
>> >
>> > It raises an error:
>> >
>> > Traceback (most recent call last):
>> > File "delete_expire_session.py", line 20, in <module>
>> > delete_expire_session(expire_time)
>> > File "delete_expire_session.py", line 13, in delete_expire_session
>> >
>> >
>> session.query(UserSession).filter(UserSession.expire_time<=expire_time).delete();
>> > File "/usr/local/lib/python2.6/dist-packages/SQLAlchemy-0.6.5-
>> > py2.6.egg/sqlalchemy/orm/query.py", line 2142, in delete
>> > result = session.execute(delete_stmt, params=self._params)
>> > File "/usr/local/lib/python2.6/dist-packages/SQLAlchemy-0.6.5-
>> > py2.6.egg/sqlalchemy/orm/session.py", line 726, in execute
>> > engine = self.get_bind(mapper, clause=clause, **kw)
>> > File "/usr/local/lib/python2.6/dist-packages/SQLAlchemy-0.6.5-
>> > py2.6.egg/sqlalchemy/ext/horizontal_shard.py", line 73, in get_bind
>> > return self.__binds[shard_id]
>> > KeyError: None
>> >
>> > I guess shard_chooser return None because of "instance" is None at
>> > runtime. I read from the docs: shard_chooser maybe in "some round-
>> > robin scheme". But I don't have any idea what does it exactly mean in
>> > my case.
>> >
>> > I appreciate any advice.
>> >
>> > Best regards!
>> > can
>> >
>> >
>> > PS: you can access partial source code in the gist:
>> https://gist.github.com/930708
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> Groups "sqlalchemy" group.
>> > To post to this group, send email to sqlalchemy@googlegroups.com.
>> > To unsubscribe from this group, send email to
>> sqlalchemy+unsubscr...@googlegroups.com.
>> > For more options, visit this group at
>> http://groups.google.com/group/sqlalchemy?hl=en.
>> >
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "sqlalchemy" group.
>> To post to this group, send email to sqlalchemy@googlegroups.com.
>> To unsubscribe from this group, send email to
>> sqlalchemy+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/sqlalchemy?hl=en.
>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To post to this group, send email to sqlalchemy@googlegroups.com.
> To unsubscribe from this group, send email to
> sqlalchemy+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/sqlalchemy?hl=en.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To post to this group, send email to sqlalchemy@googlegroups.com.
> To unsubscribe from this group, send email to
> sqlalchemy+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/sqlalchemy?hl=en.
>
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
