[appengine-java] Best performance for deleting and inserting entities
I want to upload big number of objects (from an xls file) into datastore. What is the best way (least time consuming) for inserting and deleting a collection of entities ? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Best performance for deleting and inserting entities
You can use the RemoteDatastore to do bulk puts directly from your desktop to the live datastore. It is extremely fast as it simply forwards on the binary protocol buffer data from one environment to another. http://code.google.com/p/remote-datastore/ On 10 Jul 2010, at 20:06, Robert Lancer wrote: What I would do is create a servlet that accepts a row or rows of the spreadsheet as params, and then create a local java app to read from the spreadsheet and gradually feed the spreadsheet to the servlet, app engine accepts up to 500 entities in a put. On Jul 10, 7:56 am, Ice13ill andrei.fifi...@gmail.com wrote: I want to upload big number of objects (from an xls file) into datastore. What is the best way (least time consuming) for inserting and deleting a collection of entities ? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com . To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en . -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Best performance for deleting and inserting entities
It is about the same speed. I've just uploaded 500,000 entities in about 4 hours (over a very slow connection) but my batch size was only 50 - perhaps larger batches would be faster. The new Mapping API sounds like the ideal solution but it is python only at the moment. You could use the Blobstore to roll your own using the Deferred class to process a byte range at a time. On 11 Jul 2010, at 01:35, Peter Ondruska wrote: John, is Remote Datastore faster than Python bulk insert? I tried to load millions of entities using Python but had to give up as it took about an hour to load just 100K. Peter On Jul 10, 4:22 pm, John Patterson jdpatter...@gmail.com wrote: You can use the RemoteDatastore to do bulk puts directly from your desktop to the live datastore. It is extremely fast as it simply forwards on the binary protocol buffer data from one environment to another. http://code.google.com/p/remote-datastore/ On 10 Jul 2010, at 20:06, Robert Lancer wrote: What I would do is create a servlet that accepts a row or rows of the spreadsheet as params, and then create a local java app to read from the spreadsheet and gradually feed the spreadsheet to the servlet, app engine accepts up to 500 entities in a put. On Jul 10, 7:56 am, Ice13ill andrei.fifi...@gmail.com wrote: I want to upload big number of objects (from an xls file) into datastore. What is the best way (least time consuming) for inserting and deleting a collection of entities ? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com . To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com . For more options, visit this group athttp://groups.google.com/group/google-appengine-java?hl=en . -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com . To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en . -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: memcache best practice or framework
Hi John, There is also a CompositeCache class that allows you to layer the caches so that it first checks in-memory, then memcache , then the datastore. Puts go to all levels and cache hits refresh the higher levels. e.g. if an item is not in-memory and has been flushed from memcache but is still present in the datastore then the other two will be updated. http://code.google.com/p/stick-cache/ What's the usage for in-memory caching via stick-cache? The docs show this: MemoryCacheString, Hotel mc = new MemoryCacheString, Hotel(50); but on Jun 22, 2010; that class was deleted revisionded84586e4Delete /src/main/java/com/vercer/cache/MemoryCache.java Is it no longer supported? Shawn -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: memcache best practice or framework
On 11 Jul 2010, at 09:12, Shawn Brown wrote: MemoryCacheString, Hotel mc = new MemoryCacheString, Hotel(50); but on Jun 22, 2010; that class was deleted revision ded84586e4Delete /src/main/java/com/vercer/cache/ MemoryCache.java Is it no longer supported? Try an update now - it is back. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: image service is changing some white backgrounds to black
you are welcome. Rgds. R 2010/7/6 JakeP jake.pier...@gmail.com Thanks Ronmell! -jake On Jul 6, 11:03 am, Ronmell Fuentes ringe...@gmail.com wrote: Hi JakeP as far as I know, when you have an PNG image this allows you to write metadata in order to make the background transparent, but in JPEG/JPG files this cannot be done. So all I can see is that the app is working properly but when an image in JPG/JPEG is showed, the viewer doesn't know how to interpret the background-metadata so, the easiest thing to do is show it in black colour. Rgds. Ronmell 2010/7/6 JakeP jake.pier...@gmail.com I have an app that does two image transforms - it reduces the size of images from rss feeds and spits out jpeg format images that are stored in the datastore. It also stores the original size images that it stores in jpeg format. image1: original image = resize = I'm Feeling Lucky = jpeg image2: original image = I'm Feeling Lucky = jpeg When I look at the reduced size image in a browser, sometimes, the image's white background is changed to black. Known issue or Bug? orig: http://salonnewsapp.appspot.com/server?imagekey=da866db2b26c03e8e7866. .. reduced size: http://salonnewsapp.appspot.com/server?imagekey=da866db2b26c03e8e7866. .. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.comgoogle-appengine-java%2B unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- ausencia de evidencia ≠ evidencia de ausenciahttp:// culturainteractiva.blogspot.com/ -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- ausencia de evidencia ≠ evidencia de ausencia http://culturainteractiva.blogspot.com/ -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
