How common is it to need this kind of logic? What kind of behavior
did you start to see and what was the qps load when it happened? In
all the talk of this issue, nobody ever described the failure mode...
datastore timeouts?
I hesitate to bake exotic, data-mangling behavior like this into
Objectify because once it gets added, it's pretty much impossible to
remove. Even if Google "fixes" the underlying issue.
Jeff
On Mon, May 7, 2012 at 10:06 PM, Michael Hermus
<michael.her...@gmail.com> wrote:
> Jeff: I will poke around there, thanks. I love Objectify by the way - great
> work.
>
> I think one workaround might be to create a 'Sharded Index'. Let's say we
> want to index a timestamp; we could do the following:
>
>
> public void setTimestamp(Long timestamp) {
> int shardNum= <well distributed numeric value in a defined range, either
> randomly generated or from existing variable>
> this.shardedTimestamp= String.valueOf(shardNum) +
> String.valueOf(timestamp);
>
> }
>
> public Long getTimestamp() {
> return Long.parseLong(timestamp.substring(1));
> }
>
>
>
> When it comes time to query, we create a utility wrapper that creates
> sharded queries. Something like this:
>
> public class ShardedQuery<T>
> {
> int numberOfShards;
> public List<Query<T>> queries;
>
> public ShardedQuery(Query<T> q, int numberOfShards){
> this.numberOfShards= numberOfShards;
> this.queries= new ArrayList<Query<T>>(numberOfShards);
> for(int i=0; i < numberOfShards; i++){
> this.queries.add(q.clone());
> }
> }
>
> public void addShardedFilter(String condition, String value){
> for(int i=0; i < numberOfShards; i++){
> String shardedVal= String.valueOf(i) + value;
> this.queries.get(i).filter(condition, shardedVal);
> }
> }
>
> public List<T> execute(){
> List<QueryResultIterable<T>> iterables= new
> ArrayList<QueryResultIterable<T>>(numberOfShards);
> for(int i=0; i < numberOfShards; i++){
> iterables.add(this.queries.get(i).fetch());
> }
>
> List<T> out= new LinkedList<T>();
> for(QueryResultIterable<T> result : iterables){
> for (T item : result){
> out.add(item);
> }
> }
>
> return out;
> }
>
> }
>
> Actually, you could probably add something into Objectify to handle all this
> for you.. perhaps create a '@ShardedIndex' annotation that would
> automatically prepend a shardNum and split queries for you.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/SQT-fD0inH8J.
>
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
