Interesting results. Thanks for the info.
Robert
On Sat, Nov 13, 2010 at 09:11, Remigius <remigius.stal...@gmail.com> wrote:
> I have performed some experiments. I generated test data on which an
> IN query was performed. The test data consists of the following:
>
> o an optional parent for all further test entities (i.e. the tests
> were performed with and without common parent)
> o a set of <n> referred entities, n taking the values 30, 50, 100, 500
> o a set of <n> referrer entities, each having a field that contains
> the key of one of the referred entities
>
> For each experiment, there were two (no parent) resp. three (with
> parent) queries, as follows (I hope the code can still be read after
> reformatting in the post):
>
> public static final String parentEntityKind = "TestInQueryParent";
> public static final String referredEntityKind =
> "TestInQueryReferred";
> public static final String referrerEntityKind =
> "TestInQueryReferrer";
> public static final String referrerField = "ref";
>
> private void queryData(StringBuilder out) {
> DatastoreServiceConfig config =
> DatastoreServiceConfig.Builder.withDefaults();
> DatastoreService datastoreService =
> DatastoreServiceFactory.getDatastoreService(config);
>
> if(withParent) {
> Query parentQuery = new Query(parentEntityKind);
> Entity parent =
> datastoreService.prepare(parentQuery).asSingleEntity();
> out.append("fetched parent");
> }
>
> Query referredQuery = withParent ? new Query(referredEntityKind,
> parent.getKey()) : new Query(referredEntityKind);
> referredQuery.setKeysOnly();
> long startTime = new Date().getTime();
> Iterable<Entity> referreds =
> datastoreService.prepare(referredQuery).asIterable();
> ArrayList<Entity> referredList = new ArrayList<Entity>();
> for(Entity referred : referreds) {
> referredList.add(referred);
> }
> long endTime = new Date().getTime();
> out.append("<br>fetched ");
> out.append(referredList.size());
> out.append(" referred entities in ");
> out.append(Log.timeDiff(startTime, endTime));
>
> List<Key> keyList = new ArrayList<Key>();
> for (Entity entity : referredList) {
> keyList.add(entity.getKey());
> }
> Query inQuery = new Query(referrerEntityKind);
> inQuery.addFilter(referrerField, Query.FilterOperator.IN,
> keyList);
> startTime = new Date().getTime();
> List<Entity> result =
> datastoreService.prepare(inQuery).asList(FetchOptions.Builder.withDefaults());
> endTime = new Date().getTime();
> out.append("<br>fetched ");
> out.append(result.size());
> out.append(" referrer entities in ");
> out.append(Log.timeDiff(startTime, endTime));
> }
>
> Log.timeDiff(startTime, endTime) computes the difference between start
> time and end time and converts it to a String.
>
> The results can be found here:
>
> <https://spreadsheets.google.com/ccc?
> key=0AvB6ADVFJKfSdHRWYjhNUWVwLWJDX2lkWEdSVnpNREE&hl=en>
>
> I have enabled appstats, but it is not feasible to add a screen shot
> of the experiment with 500 entity pairs here. However, the picture is
> very similar to the Python one (except for the lack of a limit of 30,
> of course). It looks again as if the IN query is split into 500
> individual queries that are executed sequentially, each of which takes
> about 2-6ms elapse time, resulting in an RPC total of 2683ms (11305ms
> api) and a Grand Total of 3192ms (11305ms cpu+api), which means that
> it's not only possible to execute such queries, but moreover they
> execute in a reasonnable amount of time. Of course, in this particular
> case it would be easier to query the entities directly using a
> get(List<Key>) call, but if you have multiple referrers to query (like
> an 1 to n relationship), you can't avoid this kind of query.
>
> Cheers, Remigius.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
