Re: [appengine-java] Re: Number of writes per second limitation

2011-11-07 Thread Max Ross (Google)
5 put() RPCs plus one more for the commit, although you could use a batch put to turn this into 1 put() RPC plus one more for the commit. These are implementation details though. We may at some point just hold on to the entire mutation until commit time (easy if you're not asking us to generate

Re: [appengine-java] Re: Number of writes per second limitation

2011-11-06 Thread de Witte
It is pseudo code. To be detailed, you have to store it as byte array inside a Blob. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-

Re: [appengine-java] Re: Number of writes per second limitation

2011-11-06 Thread Jeff Schnitzer
On Sun, Nov 6, 2011 at 12:56 PM, de Witte wrote: > > HashMap map = new HashMap(4000); > entity.setUnindexedProperty("dto", map) ; Have you actually tried this? If it works, this behavior is totally new to me, and I'd like to hear an official description of how this is represented in the raw data

[appengine-java] Re: Number of writes per second limitation

2011-11-06 Thread de Witte
There are three main tiers. 1) The client. (gwt/browser). 2) The frontend (your server code), 3) The backend (google's magic), At the backend, the entity is serialized as whole, but between the frontend and backend, you can decide to request only a sub set of the enities properties. - The sol

[appengine-java] Re: Number of writes per second limitation

2011-11-05 Thread J.Ganesan
Thanks, Jeff. I understand that an entity is a blob from serialization point of view. I thought that there is a notion of sub-blob corresponding to each property and that when a property is updated, only a sub-blob is written out to disk. You clarified that this is not the case. Thanks. But, this

Re: [appengine-java] Re: Number of writes per second limitation

2011-11-05 Thread Jeff Schnitzer
You're still thinking of an RDBMS. Here is a more accurate mental model of the datastore: It's a big persistent HashMap. The key is a Key (tuple of ancestor keys, kind, and id), the value is a serialized (protobuf) blob of your properties. There is no update except loading an entity *whole* and

[appengine-java] Re: Number of writes per second limitation

2011-11-05 Thread J.Ganesan
Thank you, Ikai. Your comments were useful. I was fixated on "one object one entity". Following your advice, I collapsed the entities. Now, they are only handful. One last doubt - how does writing to the disk work in the following scenario ? // transaction 1 // set two objects as properties // no

Re: [appengine-java] Re: Number of writes per second limitation

2011-11-04 Thread Jeff Schnitzer
On Fri, Nov 4, 2011 at 3:56 PM, Ikai Lan (Google) wrote: > If all 4000 entites are in a single entity group, in theory you can do this > because it counts as a single transactional write. There's a maximum RPC > size of 11mb (implementation detail) so if you trip this, you're in some > trouble - t

Re: [appengine-java] Re: Number of writes per second limitation

2011-11-04 Thread Ikai Lan (Google)
If all 4000 entites are in a single entity group, in theory you can do this because it counts as a single transactional write. There's a maximum RPC size of 11mb (implementation detail) so if you trip this, you're in some trouble - the RPC size include not only the size of the entity but also the s

[appengine-java] Re: Number of writes per second limitation

2011-11-04 Thread J.Ganesan
Thank you, Gerald. I will look for alternative implementations if I can not call put() many times within a transaction. I am waiting for Ikai's comments. J.Ganesan On Nov 4, 11:02 am, Gerald Tan wrote: > If there is no need to reference the objects from outside the group, you > would probably fi

[appengine-java] Re: Number of writes per second limitation

2011-11-03 Thread Gerald Tan
If there is no need to reference the objects from outside the group, you would probably find it a lot more efficient to store the while array serialized as a byte array. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To view thi

[appengine-java] Re: Number of writes per second limitation

2011-11-03 Thread J.Ganesan
On Nov 4, 6:00 am, "Ikai Lan (Google)" wrote: > I'm confused ... an Object[] array? This is bad for a ton of reasons. > Calling put() 4000 times in a single request is just bad for a lot of > reasons. > > I'm not sure I understand what problem you are trying to solve. Can you > explain so we can