Have you tried surrounding the select and delete queries in once transaction? See: http://code.google.com/appengine/docs/java/datastore/transactions.html
There is no reason it shouldn't work otherwise, but it may be worth a shot. Also, with it being the same 213 objects, maybe those particular objects are corrupt. Have you tried to manually delete them from the admin console? On Dec 8, 6:22 pm, Michael <[email protected]> wrote: > I have an hourly cron job that deletes data from the datastore older than > one month (this data is archived elsewhere for long term storage). The > first run of that cron job on Tuesday after the datastore came back up > behaved quite unusually, ended with "java.lang.OutOfMemoryError: Java heap > space", and hasn't completed once since then. While it is possible that > this is pure coincidence, I'm wondering if something done during the > maintenance resulted in this behavior. I have been unable to get this cron > job to run correctly since then. > > The job is quite simple, and has been running happily for about a year; I > will present the idea here for brevity and attach the source for those > interested. In essence the job does the following: > - retrieve at most 500 entities older than 1 month by their keys only > - send all resulting keys as a list to datastore.delete() > - repeat until no results are returned > > The first run after maintenance produced the attached log-excerpt.txt. The > brief version is the following: > - deleted 500 objects > - deleted 465 objects > - deleted 213 objects (repeated 395 times) > - out of memory > > It seems that, after actually deleting the first 752 objects of the query, > the datastore got stuck on the next 213. The same 213 objects were sent > repeatedly to datastore.delete(). No exceptions were generated, but the > data was obviously not deleted. > > The next attempt (the job was retried since it crashed) produced almost > identical output. This time, it actually deleted 174 objects, then tried > to delete the same 213 objects over and over until it, too, crashed with an > OutOfMemoryError. The run after that actually deleted 8 objects before it > crashed in the same manner. This continued until the error ran my > application out of quota for the day, at which point I got a notification > email and went to go pause the queue that these jobs run under. > > Note, I am not on the high replication datastore. I do not know why this > is happening, but it is currently an insurmountable obstacle. I tried > unpausing the queue temporarily and running the problematic job, and this > time I did not even get the previously frustrating but informative output; > instead, I merely got the "A serious problem was encountered . . ." message > on both runs. > > Any help in getting this fixed or understanding the problem would be > greatly appreciated. > > log-excerpt.txt > 60KViewDownload > > DeleteOldDataServer.java > 4KViewDownload -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
