[google-appengine] Re: Massive datastore batch put, how to?

2009-08-21 Thread Stakka
ve.  130K items for me consumes nearly 6.5 hours of CPU.  Not > very efficient > > On Aug 16, 5:36 pm, Stakka wrote: > > > I implemented a rough version of my solution, and it seems to work up > > to ~15k entities. Above that I hit the undocumented transaction write >

[google-appengine] Re: Massive datastore batch put, how to?

2009-08-16 Thread Stakka
al price I will be paying to run the app. > > On Aug 14, 9:24 pm, Stakka wrote: > > > Thanks for the tip, but why write a web app when Java Applets are > > required, that whouldn't be a good solution. Also, the uploaded file > > needs to be parsed in it's entir

[google-appengine] Re: Transactionally updating multiple entities over 1MB

2009-08-16 Thread Stakka
Wierd, I just hit a limit on the size of a transaction when commiting: "java.lang.IllegalArgumentException: datastore transaction or write too big." All (23) entities in the transaction where in the same entity group, not using batch put and ~990k in size. On 23 Juli, 12:30, "Nick Johnson (Goo

[google-appengine] Re: Massive datastore batch put, how to?

2009-08-14 Thread Stakka
and let us know, so that we can use > it too. > > On Aug 12, 11:36 pm, Stakka wrote: > > > I'am working on an browser based accounting app which has a feature to > > import ledger transactions through file uploads. I'am currently only > > running on the loca

[google-appengine] Datastore put performance?

2009-08-14 Thread Stakka
Has anyone benchmarked the live datastore, especially the low-level batch put operation? How many entities per second can I expect? I know it all depends on number of properties, ancestors, indices, etc,. but any estimate would be appreciated. --~--~-~--~~~---~--~~

[google-appengine] Re: How to handle a file upload containing thousands of entities?

2009-08-13 Thread Stakka
Sorry for the dup posting, 1st post screening took ages. http://groups.google.com/group/google-appengine/browse_thread/thread/b11a07e4a71566b6# --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" grou

[google-appengine] How to handle a file upload containing thousands of entities?

2009-08-13 Thread Stakka
I'am working on an browser based accounting app which has a feature to import ledger transactions through file uploads. Currently it's only running on the local dev server, but from what I've read datastore puts -- even batch -- is very slow and CPU (quota) intensive when deployed live. So, how d

[google-appengine] Massive datastore batch put, how to?

2009-08-13 Thread Stakka
I'am working on an browser based accounting app which has a feature to import ledger transactions through file uploads. I'am currently only running on the local dev server, but from what I've read datastore puts -- even batch -- is very slow and CPU (quota) intensive when deployed live. How do I