I'm currently developing a website that will require users to upload a few photos of their houses. There is a pretty good chance that those photos will be taken with a digital camera, and thus bigger than 1Mb. In order to save space, I would like to resize them to more web- friendly dimensions, like 600*480 px max. Uploading an image that is bigger than 1Mb is not a problem anymore, however processing it through Google's image API is not possible as API calls are still limited to 1Mb.
I've opened a new issue against Google App Engine asking to up the limit on the image.resize() call, so that we would be able to process those images: http://code.google.com/p/googleappengine/issues/detail?id=1422 I could just encourage you to flag this issue, so that it might eventually be addressed by Google. However, I don't want to be too negative on this issue, as it is stated on the App Engine Product Roadmap ( http://code.google.com/appengine/docs/roadmap.html ) Future Releases: January 2009 - June 2009 * Service for storing and serving large files I don't know if this will enable us to resize the images, or just store them. I hope we'll be able to resize them, as serving multiple 3+Mb images would really be a usability issue for a website! Otherwise, I'll be looking for some kind of Flash/Java app that would enable to resize the images before uploading them (which would decrease the time a user needs to upload one of its digital images and thus improve it's user experience). I'm still not that happy to force the user to use this kind of technology, but I guess there is not much else available for now. Would any of you know of an open-sourced resizer/uploader that could easily be integrated? I didn't plan on having to develop this feature! On 21 avr, 23:48, Anuraag Agrawal <anura...@gmail.com> wrote: > > It has been said that the image API is actually executed on an image > > processing cluster, not the application servers. > >http://code.google.com/appengine/docs/python/images/overview.html > > mentions an "Images service" in several locations. > > I guess it would make sense to outsource to an image processing > cluster for cases with lots of queued up image transformations. > Though does using the word "service" automatically make it an external > resource? I'd be pretty surprised if URL Fetch was also sent to > separate machines. > > > I imigine it is not infinately scalable. So limiting image sizes - > > thus limiting the potential intensive > > ness of image operations which would block a single process which is > > an app engine instance > > for a very long time is a good thing. (asynchonous processing from > > queues which wouldn't block > > interactive services is a different ball game and I would imagine > > different limits will be placed > > once work queues become available) > > I can see the logic here but one issue I have with this is that there > is already a time limit on App Engine requests (30 seconds after the > February update). This limit is undoubtedly to address the very issue > of an app engine instance blocking interactive services for a long > time, and it seems redundant to add a size limit on top to enforce > what is essentially a time limit anyways. That being said, indeed > implementing work queues might be the better solution to just raising > the size limit. > > > Not sure what you mean by problems with ClientLogin and captchas - > > can't say I have encountered those > > apart from signup or incorrect passwords repeatedly entered. > > So it sounds like users upload images to their own Google Accounts > using Picasaweb. I'm trying to avoid requiring a Google account for > login (too many login systems on the web now adays to limit to just > that) and trying to use Picasaweb in that scenario would, if I > interpreted the docs correctly, require logging in using ClientLogin > with an account for the web application itself, which is thwarted by > captchas. There are other image APIs around though, so hopefully > there'll be a good interim solution there. > > On Apr 20, 4:55 pm, Tim Hoffman <zutes...@gmail.com> wrote: > > > Hi > > > I suppose you need to remember app engine is a horizontally scalable > > application framework > > and not a cpu intensive cluster. Whilst google's infrastructure might > > seem massive > > I imigine it is not infinately scalable. So limiting image sizes - > > thus limiting the potential intensive > > ness of image operations which would block a single process which is > > an app engine instance > > for a very long time is a good thing. (asynchonous processing from > > queues which wouldn't block > > interactive services is a different ball game and I would imagine > > different limits will be placed > > once work queues become available) > > > In the meantime, > > > I am using picassaweb by getting users to upload photos via > > picassaweb, organising via folders and > > tagging, and then grabbing the gallery urls (mostly public) using them > > within app engine, and in > > fact not really manipluaingimagesfrom app engine at all. Mainly > > providing a layet of meta data > > and organisation beyond what picasaweb provides and presenting > > thoseimagesin a non image management > > context. > > > Not sure what you mean by problems with ClientLogin and captchas - > > can't say I have encountered those > > apart from signup or incorrect passwords repeatedly entered. > > > Rgds > > > T > > > On Apr 19, 1:12 pm, Anuraag Agrawal <anura...@gmail.com> wrote: > > > > Indeed the entire API is limited to 1MB, but the point of discussion > > > I'd like to make is that for data-related API calls like the datastore > > > and memcache, it's easy to come up with hardware/implementation > > > constraints that would warrant such a limit, and there are usually > > > relatively simple ways to work around them. For a processing API call > > > like an image API call, there should be no dependence on BigTable or > > > any sort of clustering, so the limit seems a little arbitrary, > > > especially in light of the request limit increase to 10MB which does > > > not seem to have any value for image requests. I'm honestly hoping > > > it's just an oversight that will be fixed in the short term. > > > > In the interim, using an external image API seems to be the best > > > solution indeed. Right now, I'm looking into the Imageshack API which > > > seems to offer enough functionality, but can you go into how you use > > > the Picasa API? It seems to be very well suited to importing a user's > > > photos, but to upload and manage photos as a website would require > > > using ClientLogin, and since ClientLogin uses a captcha, it doesn't > > > work well on app engine. Or at least, that's what's written in the > > > App Engine Data API docs. > > > > Thanks. > > > > On Apr 18, 10:55 pm, Tim Hoffman <zutes...@gmail.com> wrote: > > > > > Why don't you stick theimagesin Picasa and just manage them through > > > > app engine ? > > > > > Thats what I am doing > > > > > T > > > > > On Apr 18, 4:28 pm, Anuraag Agrawal <anura...@gmail.com> wrote: > > > > > > When App Engine raised its request size limit from 1MB to 10MB, it > > > > > seemed like we would finally be able to use it for an image sharing > > > > > website as while reasonably sized digital cameraimagesover 1MB are > > > > > very likely, it'd take an extremely professional camera image to break > > > > > the 10MB limit, which seemed like an acceptable limit to place on a > > > > > user's file uploads since those users are probably knowledgeable > > > > > enough to resize theimagesthemselves anyways. API calls were still > > > > > limited to 1MB, and as the examples listed on the blog post were > > > > > memcache and datastore, it seemed to make sense since App Engine is > > > > > probably still designed to place a 1MB limit on its datastore > > > > > entries. This seemed like it'd be ok since it should be possible to > > > > > use theimagesAPI to resize any inputimagesto less than 1MB before > > > > > storing them in the datastore, completely acceptable for our task. > > > > > However, after trying this and looking into some server errors, it > > > > > seems theimagesAPI is also limited to 1MB input files (which fits > > > > > with the 1MB limit on API calls, the fact just didn't register at > > > > > first). At least, that's how I'm interpreting the > > > > > RequestTooLargeError > > > > > (The request to API callimages.Transform() was too large) I get when > > > > > submitting a 1.5MB file. > > > > > > Is the limit on theimagesAPI by design/constraint? I imagine image > > > > > API calls aren't split across computers in a cluster or anything and > > > > > are run in place, with possibly some temp memory that's cleared > > > > > immediately, which makes having a limit smaller than the request size > > > > > seem a little strange to me. A 1MB limit on image files makes it hard > > > > > to support user submitted image uploads in a practical setting. I > > > > > know it's possible to split the image over datastore entries just to > > > > > store them, but we also need to be able to resize them to generate > > > > > thumbnails, etc. > > > > > > And if anyone's come up with a workaround splitting the input file > > > > > into parts to resize in parts, it'd be nice to hear. While PNG uses > > > > > DEFLATE and might not work, JPEG as far as I know cosine transforms > > > > > blocks independently so it seems like it could be possible. Though > > > > > it'd probably increase the load on the servers more than just having a > > > > > > >1MB API call limit. > > > > > > Thanks. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
