Well, I've read a lot of posts about google datastore and the problems
with batch operations, relational approach to arrange data in bigtable
etc. but I always thought the problem wasn't in datastore itself but
in the way people use it. Now I can see with my experience that it
acts just in an unpredictable way. I deployed a test java app that
tries to clear 500 (guaranteed amount!) entries per request. All
entries are in the same entity group and delete is executed in batch
in single transaction. All operations are executed with low level API
so no possible overhead is involved. Here is a sample code and logs:
Code (cut):
=============
Query q = new Query(World.class.getSimpleName()); // create query
Iterator<Entity> i = datastoreService.prepare(q).asIterator();
idx = 0;
while (i.hasNext() && idx<500) {
keys.add(i.next().getKey());
idx++;
}
// delete keys in batch
Transaction t = datastoreService.beginTransaction();
datastoreService.delete(keys);
t.commit();
==============
1st request (all goes well, 500 entries removed)
-------------------------------------------------
1.
I 04-30 07:52AM 02.091 org.itvn.controller.TvnController
clearDbBySize: Reading 500 entity keys...
See details
2.
I 04-30 07:52AM 03.832 org.itvn.controller.TvnController
clearDbBySize: Removing keys by groups, total groups: 1
3.
I 04-30 07:52AM 03.832 org.itvn.controller.TvnController
clearDbBySize: Trying to remove 500 entities...
4.
I 04-30 07:52AM 07.873 org.itvn.controller.TvnController
clearDbBySize: Removed 500 entities.
2nd request - timeout exception, on READ operation (i.hasNext())
-------------------------------------------------
1.
I 04-30 07:52AM 22.719 org.itvn.controller.TvnController
clearDbBySize: Reading 500 entity keys...
See details
2.
W 04-30 07:52AM 26.551 Nested in
org.springframework.web.util.NestedServletException: Request
processing failed; nested exception is
com.google.appengine.api.datastore.Datas
3.
W 04-30 07:52AM 26.552 /clear_db/500
com.google.appengine.api.datastore.DatastoreTimeoutException:
datastore timeout: operation took too long. at
com.google.appengine.api.d
4.
C 04-30 07:52AM 26.555 Uncaught exception from servlet
com.google.appengine.api.datastore.DatastoreTimeoutException:
datastore timeout: operation took too long. at com.goog
Here we go, first request executes well, and the next (only a few
seconds later) fails! Note that this is only a test application, with
no load at all. Am I doing something wrong? What's the RELIABLE way to
read/remove 500 entities? Is it a problem with quantity (500)? If so
how much entities could be read without timeout? Can someone give the
reasonable answer to this? If you need more details about app, I can
share this test case in public.
Oleg
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~----------~----~----~----~------~----~------~--~---
