Re: [appengine-java] Re: Recommended maximum number of entities in an entity group

2010-10-25 Thread Ikai Lan (Google)
I was unclear in my last point: real world performance should be above 1 transaction per second, but you should engineer your entity group writes with an expectation of 1 transaction/second for best results. -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine

[appengine-java] Re: Recommended maximum number of entities in an entity group

2010-10-23 Thread nicanor.babula
Acually, I forgot about the fact that while a transaction is going on, all entities involved are locked. Therefore, I considered changing my database structure in order to get more, but smaller entity groups (every parentEntity will have at most 20-30 childs). That way, will be easier and faster to

Re: [appengine-java] Re: Recommended maximum number of entities in an entity group

2010-10-22 Thread Ikai Lan (Google)
Cyrille is right: have an expectation of 1 transaction/second, though real world performance should be below this. App Engine favors a model of eventual consistency. You don't get to scale by locking a ton of entities at once. -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: htt

Re: [appengine-java] Re: Recommended maximum number of entities in an entity group

2010-10-22 Thread Cyrille Vincey
>From my experience : I do NOT expect a better write performance than 1 transaction/second when creating entities inside one given entity group (with 1 entity created in each transaction). In your case : if dataset creation is an offline process, you can rely on entity groups and parent/child data

[appengine-java] Re: Recommended maximum number of entities in an entity group

2010-10-22 Thread Didier Durand
Hi Cristian, The limitation of transaction per entity group is not a limitation but a feature: it keeps transaction mechanisms much simple (remote unit of work instead of distributed unit of work -> more reliable) and so allows for much better throughput and performances Playing with tasks in tra

Re: [appengine-java] Re: Recommended maximum number of entities in an entity group

2010-10-22 Thread Nicanor Babula
Thanks guys for your answers. I really appreciate that. Anyway, in order to avoid other problems and performance issues, I changed the structure of my database so that I will be able to do the transactions I want. Unfortunately the fact that a transaction is limited to entities of the same entity g

[appengine-java] Re: Recommended maximum number of entities in an entity group

2010-10-21 Thread Ian Marshall
I guess that the limit is enormous. (That doesn't help you since you want to actually know!) Of course, you can only add entities to your entity group at a certain rate of transactions per unit time. This is a constraint on how the number of entities in any given entity group increases over time.

[appengine-java] Re: Recommended maximum number of entities in an entity group

2010-10-21 Thread Didier Durand
Hi, 1 info that is important to reach the value for this max number (if it exists) is the size of the entities: the bigger they are, the faster you will occupy the bandwith between the BigTable server and the processing server. So, if you want to go fast, you have to spread entities among many s

[appengine-java] Re: Recommended maximum number of entities in an entity group

2010-10-21 Thread nicanor.babula
Thank you all for your responses. @alesj No, I am not confusing entity "group" with the actual entities stored in the datastore. @Ian Marshall Actually no, because I already did that kind of analysis. I have to use transactions in order to maintain data consistency and therefore I have to define

[appengine-java] Re: Recommended maximum number of entities in an entity group

2010-10-21 Thread Ian Marshall
How about my comments below? http://www.google.com/url?url=http://groups.google.com/g/f907f736/t/f81ea99ea7c010af/d/9fac9b668ea24068%3Fq%3D%239fac9b668ea24068&ei=oDjATIKTGY_D_QaQ8cDyCA&sa=t&ct=res&cd=7&source=groups&usg=AFQjCNHfdZa1EiQX24dQ-dp94o0b7kBZ_w Do they help you? On Oct 20, 6:39 pm, "n

[appengine-java] Re: Recommended maximum number of entities in an entity group

2010-10-21 Thread alesj
I think you're confusing entity *group* with actual entities stored in datastore, as I doubt you'll have thousands of related entities. --> http://code.google.com/appengine/docs/java/datastore/transactions.html#Using_Transactions see "Entity groups" On Oct 20, 7:39 pm, "nicanor.babula" wrote: >