[appengine-java] Re: Datastore Limits
What do you mean by times out? Do you mean you hit the 30 second deadline? On Sep 14, 3:59 am, hg hgo...@gmail.com wrote: Hi, I am writing a script that is supposed to run quite a few inserts to the datastore - up to a couple of hundred. My script keeps timing out at 92. After I ran it a few times this evening, it wiped out my entire datastore! I did some research and came across the following error documentation: for the DatastoreTimeoutException DatastoreTimeoutException is thrown when a datastore operation times out. This can happen when you attempt to put, get, or delete too many entities or an entity with too many properties, or if the datastore is overloaded or having trouble. I was not getting this error, but it does seem to have some bearing on this case. What are the limit on entity 'putting' for a script? Any suggestions would be appreciated. Thanks! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Datastore Limits
read this thread ... you might want to batch the writes you should get around 200-300 objects per write (under the 5 sec timeout) and probably around 2k per request before hitting the 30sec request timeout (all depends on the complexity of your objects) On Sep 14, 5:54 am, Iain iain.robe...@gmail.com wrote: What do you mean by times out? Do you mean you hit the 30 second deadline? On Sep 14, 3:59 am, hg hgo...@gmail.com wrote: Hi, I am writing a script that is supposed to run quite a few inserts to the datastore - up to a couple of hundred. My script keeps timing out at 92. After I ran it a few times this evening, it wiped out my entire datastore! I did some research and came across the following error documentation: for the DatastoreTimeoutException DatastoreTimeoutException is thrown when a datastore operation times out. This can happen when you attempt to put, get, or delete too many entities or an entity with too many properties, or if the datastore is overloaded or having trouble. I was not getting this error, but it does seem to have some bearing on this case. What are the limit on entity 'putting' for a script? Any suggestions would be appreciated. Thanks!- Hide quoted text - - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Datastore Limits
No, I don't mean the 30 second timeout. First I thought that that was the issue, but then I counted and it wasn't near 30 seconds. I also realized that the script always stopped after 92 writes. On Sep 14, 3:54 pm, Iain iain.robe...@gmail.com wrote: What do you mean by times out? Do you mean you hit the 30 second deadline? On Sep 14, 3:59 am, hg hgo...@gmail.com wrote: Hi, I am writing a script that is supposed to run quite a few inserts to the datastore - up to a couple of hundred. My script keeps timing out at 92. After I ran it a few times this evening, it wiped out my entire datastore! I did some research and came across the following error documentation: for the DatastoreTimeoutException DatastoreTimeoutException is thrown when a datastore operation times out. This can happen when you attempt to put, get, or delete too many entities or an entity with too many properties, or if the datastore is overloaded or having trouble. I was not getting this error, but it does seem to have some bearing on this case. What are the limit on entity 'putting' for a script? Any suggestions would be appreciated. Thanks! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Datastore Limits
Thanks for replying. A couple of questions: Which thread should I read? Batching the writes seems like a great idea, I'll try it. On Sep 14, 7:01 pm, Larry Cable larry.ca...@gmail.com wrote: read this thread ... you might want to batch the writes you should get around 200-300 objects per write (under the 5 sec timeout) and probably around 2k per request before hitting the 30sec request timeout (all depends on the complexity of your objects) On Sep 14, 5:54 am, Iain iain.robe...@gmail.com wrote: What do you mean by times out? Do you mean you hit the 30 second deadline? On Sep 14, 3:59 am, hg hgo...@gmail.com wrote: Hi, I am writing a script that is supposed to run quite a few inserts to the datastore - up to a couple of hundred. My script keeps timing out at 92. After I ran it a few times this evening, it wiped out my entire datastore! I did some research and came across the following error documentation: for the DatastoreTimeoutException DatastoreTimeoutException is thrown when a datastore operation times out. This can happen when you attempt to put, get, or delete too many entities or an entity with too many properties, or if the datastore is overloaded or having trouble. I was not getting this error, but it does seem to have some bearing on this case. What are the limit on entity 'putting' for a script? Any suggestions would be appreciated. Thanks!- Hide quoted text - - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Datastore Limits
Thank you for replying. I tried batching the writes, and it didn't solve the problem. However, it did show me that my script is dying after I create 92 (exactly) entities, whether or not they have all been persisted. Could this be connected to the 2k per request? What exactly is the 2k per request? Thanks! On Sep 14, 7:01 pm, Larry Cable larry.ca...@gmail.com wrote: read this thread ... you might want to batch the writes you should get around 200-300 objects per write (under the 5 sec timeout) and probably around 2k per request before hitting the 30sec request timeout (all depends on the complexity of your objects) On Sep 14, 5:54 am, Iain iain.robe...@gmail.com wrote: What do you mean by times out? Do you mean you hit the 30 second deadline? On Sep 14, 3:59 am, hg hgo...@gmail.com wrote: Hi, I am writing a script that is supposed to run quite a few inserts to the datastore - up to a couple of hundred. My script keeps timing out at 92. After I ran it a few times this evening, it wiped out my entire datastore! I did some research and came across the following error documentation: for the DatastoreTimeoutException DatastoreTimeoutException is thrown when a datastore operation times out. This can happen when you attempt to put, get, or delete too many entities or an entity with too many properties, or if the datastore is overloaded or having trouble. I was not getting this error, but it does seem to have some bearing on this case. What are the limit on entity 'putting' for a script? Any suggestions would be appreciated. Thanks!- Hide quoted text - - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
