Re: [appengine-java] Re: How to query objects with criterias defined in child entities

2010-03-15 Thread John Patterson
Nice one. If you have any questions please use the Twig discussion group: http://groups.google.com/group/twig-persist Im about like 24/7 at the moment! On 16 Mar 2010, at 13:45, Max wrote: Hi John, Many thanks. I will prepare large set of testing data and try to use Twig to model/ query

Re: [appengine-java] Re: How to query objects with criterias defined in child entities

2010-03-15 Thread John Patterson
On 16 Mar 2010, at 13:05, Max wrote: 3, use a batch get to obtain all User entities Until merged AND is finished you would need to do this with the low- level service directly datastore.getService().get(allMatchingKeys); // bulk get you can then convert the resulting Entities into typesa

[appengine-java] Re: How to query objects with criterias defined in child entities

2010-03-15 Thread Max
Hi John, Many thanks. I will prepare large set of testing data and try to use Twig to model/ query them. Will get back to you soon. Best regards, Max On Mar 16, 2:14 pm, John Patterson wrote: > Thats right unfortunately.  Like Jeff said, that is what a RDBMS is   > probably doing anyway but it

Re: [appengine-java] Re: How to query objects with criterias defined in child entities

2010-03-15 Thread John Patterson
Thats right unfortunately. Like Jeff said, that is what a RDBMS is probably doing anyway but it does locally and returns only results over the wire back to you. But with Twigs parallel commands no matter how many queries you need to run, they will not take extra time. You could query on 1

[appengine-java] Re: How to query objects with criterias defined in child entities

2010-03-15 Thread Max
to query with n filters on one relation index entity *should be* to query with n filters on User with many UserSkill entity children On Mar 16, 2:05 pm, Max wrote: > Hi John, > > Thanks for your reply. To be honest I am quite disappointed that GAE > can not perform such query directly. > > Accord

[appengine-java] Re: How to query objects with criterias defined in child entities

2010-03-15 Thread Max
Hi John, Thanks for your reply. To be honest I am quite disappointed that GAE can not perform such query directly. According do your suggestion, to query with n filters on one relation index entity, we need to: 1, perform n separate queries to obtain n cursors of UserSkill entities by some sortin

[appengine-java] Re: How to query objects with criterias defined in child entities

2010-03-15 Thread Max
We have already done some testings on RDBMS and the performance is not acceptable to us. (for the second query, that means self join a table with 10 million records for n times). That's why we try GAE now. Thank you. On Mar 16, 1:54 am, Max wrote: > Hi John, > > I am designing a quite similar da

[appengine-java] Re: How to query objects with criterias defined in child entities

2010-03-15 Thread Max
Hi John, I am designing a quite similar data model of cited *User-Skill* problem, but not exactly the same. People may not be familiar with our domain. Basically it will be a track record system. User can perform different tasks and same task can be done by many users. According to historical dat

[appengine-java] Re: How to query objects with criterias defined in child entities

2010-03-15 Thread jd
On Mar 15, 4:40 pm, Jeff Schnitzer wrote: >  * Get all the parent userids of the results, putting them in a hashset. As long as the results are sorted on the same fields there is no need to load all ids into a hashset. You can just use the "zig-zag" method to filter results that are in both i

Re: [appengine-java] Re: How to query objects with criterias defined in child entities

2010-03-15 Thread John Patterson
I was meaning just put the UserSkills of the two people into the set. Each person only has a small number of skills yeah? Perhaps I mis understood your last requirement "similarity between user A and User B" On 15 Mar 2010, at 14:23, Max wrote: Hi John, Thanks for your reply. I need som

Re: [appengine-java] Re: How to query objects with criterias defined in child entities

2010-03-15 Thread Jeff Schnitzer
On Sun, Mar 14, 2010 at 9:16 PM, Max wrote: > > If I would like to find all users that have java level > 5 and c++ > level > 2, then how to write a query in your suggested data model? Every time this kind of question comes up I usually think to myself "how would an RDBMS do this?" and then roughl

[appengine-java] Re: How to query objects with criterias defined in child entities

2010-03-15 Thread Max
Hi John, Thanks for your reply. I need some time to study and test your codes. For the last point, Sets.intersection() means we need to load all keys into memory and perform an in memory Sets.intersection(). Is that possible to do this by a query directly. In other words, is that possible to use

Re: [appengine-java] Re: How to query objects with criterias defined in child entities

2010-03-14 Thread John Patterson
Hi Max, Regarding your original question, a more efficient solution would be to embed the UesrSkill in the User instance which would allow you to find all results in a single query. Th problem is that embedded instances can only be queried on a single value. There would be no way to que

[appengine-java] Re: How to query objects with criterias defined in child entities

2010-03-14 Thread Max
Thanks John, Bret Slatkins' talk is impressive. Let's say we have m skills with n levels. (i.e., m x n SkillLevel entities). Each SkillLevel entity consists of at least one SkillLevelIndex. We define similarity between user A and User B as number of skills with same level. i.e., number of SkillL

[appengine-java] Re: How to query objects with criterias defined in child entities

2010-03-14 Thread Max
Thanks Jeff, If I would like to find all users that have java level > 5 and c++ level > 2, then how to write a query in your suggested data model? Best regards, Max On Mar 10, 12:18 pm, Jeff Schnitzer wrote: > Create a UserSkill entity with a parent key of User.  Do a keysOnly > query for UserS

Re: [appengine-java] Re: How to query objects with criterias defined in child entities

2010-01-02 Thread Rusty Wright
For the > 1000 results, there's a way to paginate results, but I haven't tried it yet. Sorry, I don't know the answer to your first question. reini77 wrote: > Sounds like a good idea. This would mean that I'm able to query users > by a skill pretty fast. > But is there a chance to query users w

[appengine-java] Re: How to query objects with criterias defined in child entities

2010-01-01 Thread reini77
Sounds like a good idea. This would mean that I'm able to query users by a skill pretty fast. But is there a chance to query users which have a set of skills efficient in app-engine? Or would i need to query for each skill seperatly and filter the list of users (which have all the required skills)