> It has been said that the image API is actually executed on an image > processing cluster, not the application servers. > http://code.google.com/appengine/docs/python/images/overview.html > mentions an "Images service" in several locations.
I guess it would make sense to outsource to an image processing cluster for cases with lots of queued up image transformations. Though does using the word "service" automatically make it an external resource? I'd be pretty surprised if URL Fetch was also sent to separate machines. > I imigine it is not infinately scalable. So limiting image sizes - > thus limiting the potential intensive > ness of image operations which would block a single process which is > an app engine instance > for a very long time is a good thing. (asynchonous processing from > queues which wouldn't block > interactive services is a different ball game and I would imagine > different limits will be placed > once work queues become available) I can see the logic here but one issue I have with this is that there is already a time limit on App Engine requests (30 seconds after the February update). This limit is undoubtedly to address the very issue of an app engine instance blocking interactive services for a long time, and it seems redundant to add a size limit on top to enforce what is essentially a time limit anyways. That being said, indeed implementing work queues might be the better solution to just raising the size limit. > Not sure what you mean by problems with ClientLogin and captchas - > can't say I have encountered those > apart from signup or incorrect passwords repeatedly entered. So it sounds like users upload images to their own Google Accounts using Picasaweb. I'm trying to avoid requiring a Google account for login (too many login systems on the web now adays to limit to just that) and trying to use Picasaweb in that scenario would, if I interpreted the docs correctly, require logging in using ClientLogin with an account for the web application itself, which is thwarted by captchas. There are other image APIs around though, so hopefully there'll be a good interim solution there. On Apr 20, 4:55 pm, Tim Hoffman <zutes...@gmail.com> wrote: > Hi > > I suppose you need to remember app engine is a horizontally scalable > application framework > and not a cpu intensive cluster. Whilst google's infrastructure might > seem massive > I imigine it is not infinately scalable. So limiting image sizes - > thus limiting the potential intensive > ness of image operations which would block a single process which is > an app engine instance > for a very long time is a good thing. (asynchonous processing from > queues which wouldn't block > interactive services is a different ball game and I would imagine > different limits will be placed > once work queues become available) > > In the meantime, > > I am using picassaweb by getting users to upload photos via > picassaweb, organising via folders and > tagging, and then grabbing the gallery urls (mostly public) using them > within app engine, and in > fact not really manipluaingimagesfrom app engine at all. Mainly > providing a layet of meta data > and organisation beyond what picasaweb provides and presenting thoseimagesin > a non image management > context. > > Not sure what you mean by problems with ClientLogin and captchas - > can't say I have encountered those > apart from signup or incorrect passwords repeatedly entered. > > Rgds > > T > > On Apr 19, 1:12 pm, Anuraag Agrawal <anura...@gmail.com> wrote: > > > Indeed the entire API is limited to 1MB, but the point of discussion > > I'd like to make is that for data-related API calls like the datastore > > and memcache, it's easy to come up with hardware/implementation > > constraints that would warrant such a limit, and there are usually > > relatively simple ways to work around them. For a processing API call > > like an image API call, there should be no dependence on BigTable or > > any sort of clustering, so the limit seems a little arbitrary, > > especially in light of the request limit increase to 10MB which does > > not seem to have any value for image requests. I'm honestly hoping > > it's just an oversight that will be fixed in the short term. > > > In the interim, using an external image API seems to be the best > > solution indeed. Right now, I'm looking into the Imageshack API which > > seems to offer enough functionality, but can you go into how you use > > the Picasa API? It seems to be very well suited to importing a user's > > photos, but to upload and manage photos as a website would require > > using ClientLogin, and since ClientLogin uses a captcha, it doesn't > > work well on app engine. Or at least, that's what's written in the > > App Engine Data API docs. > > > Thanks. > > > On Apr 18, 10:55 pm, Tim Hoffman <zutes...@gmail.com> wrote: > > > > Why don't you stick theimagesin Picasa and just manage them through > > > app engine ? > > > > Thats what I am doing > > > > T > > > > On Apr 18, 4:28 pm, Anuraag Agrawal <anura...@gmail.com> wrote: > > > > > When App Engine raised its request size limit from 1MB to 10MB, it > > > > seemed like we would finally be able to use it for an image sharing > > > > website as while reasonably sized digital cameraimagesover 1MB are > > > > very likely, it'd take an extremely professional camera image to break > > > > the 10MB limit, which seemed like an acceptable limit to place on a > > > > user's file uploads since those users are probably knowledgeable > > > > enough to resize theimagesthemselves anyways. API calls were still > > > > limited to 1MB, and as the examples listed on the blog post were > > > > memcache and datastore, it seemed to make sense since App Engine is > > > > probably still designed to place a 1MB limit on its datastore > > > > entries. This seemed like it'd be ok since it should be possible to > > > > use theimagesAPI to resize any inputimagesto less than 1MB before > > > > storing them in the datastore, completely acceptable for our task. > > > > However, after trying this and looking into some server errors, it > > > > seems theimagesAPI is also limited to 1MB input files (which fits > > > > with the 1MB limit on API calls, the fact just didn't register at > > > > first). At least, that's how I'm interpreting the RequestTooLargeError > > > > (The request to API callimages.Transform() was too large) I get when > > > > submitting a 1.5MB file. > > > > > Is the limit on theimagesAPI by design/constraint? I imagine image > > > > API calls aren't split across computers in a cluster or anything and > > > > are run in place, with possibly some temp memory that's cleared > > > > immediately, which makes having a limit smaller than the request size > > > > seem a little strange to me. A 1MB limit on image files makes it hard > > > > to support user submitted image uploads in a practical setting. I > > > > know it's possible to split the image over datastore entries just to > > > > store them, but we also need to be able to resize them to generate > > > > thumbnails, etc. > > > > > And if anyone's come up with a workaround splitting the input file > > > > into parts to resize in parts, it'd be nice to hear. While PNG uses > > > > DEFLATE and might not work, JPEG as far as I know cosine transforms > > > > blocks independently so it seems like it could be possible. Though > > > > it'd probably increase the load on the servers more than just having a > > > > > >1MB API call limit. > > > > > Thanks. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
