[google-appengine] Re: how to delete a table/entity?

[ajaxer]

but i only need a way to delete an entity by one command or action.
not by a program(and it doesn't always works),
it is certainly not a matter of how it is implemented,
but a feature of the platform.

also 1000 index limit makes it not possible to fetcher older data on
paging.

for if we need an indexed page more than 1 items,
it would cost us a lot of cpu time to calculate the base for GQL
to fetch the data with index less than 1000.

how can the platform be scalable,
for currently the platform aches with only 1 items.

On Dec 9, 1:25 pm, Tim Hoffman  wrote:
> imho I think you do need to understand any new platform to a certain
> degree if you really want to take advantage of it.
> It is completely different from SQL/RDBMS which means if you don't
> change your thinking and adapt to the platform
> it "can" only be a toy for you.  If you can't get your head around the
> fact that its not a rdbms then possibly it is the wrong
> tool for whatever you are trying to do.  (That doesn't mean app engine
> couldn't use a whole range of improvements/features)
>
> For the rest of us its is a very compelling delivery platform for a
> range of applications.
>
> Rgds
>
> T
>
> On Dec 9, 10:55 am, ajaxer  wrote:
>
>
>
> > thanks for the explanation.
> > but I have no interest to learn such things as big table or something.
> > the only reason that i keep an eye on this project is it may bring me
> > convenience in my web development
> > not that it will bring me some knowledge of science or technology.
>
> > before the data manipulation tool becomes good enough, I will only
> > keep it as a toy.
>
> > On Dec 7, 11:00 am, OvermindDL1  wrote:
>
> > > On Sun, Dec 6, 2009 at 7:36 PM, ajaxer  wrote:
> > > > yes, i have tried.
> > > > but i alway get timeouts.
> > > > and the amount is more than millions maybe
> > > > I uploading this data by bulk uploading for more than a day.
>
> > > > I think it is very important to be able delete a table in a single act
> > > > on the panel
>
> > > You are thinking about this wrong.
> > > GQL does not have a relational database design, it is an indexed blob
> > > database, there is no concept of "table" as there is in SQL and such.
> > > Instead you create 'kinds', which are more analogous to a message in
> > > Google Protocol Buffers.  The things you specify as 'indexes' are
> > > actually just those things pulled from the 'message'.  Each index you
> > > specify creates a whole new 'table' with some duplicated data between
> > > all the 'tables'.  So for any given kind you will have multiple
> > > 'tables' (this is not a SQL like table, this is more like a map/dict,
> > > with ordering and such...).  Each index type can also have multiple
> > > kinds in it (such as when you change the format of a kind, you are
> > > creating a new one, but all still indexed together, meaning if you try
> > > to access an old one from your new one, you get a nice exception
> > > thrown, hence why you should always change your kind name when you
> > > change your kind too).
>
> > > So, a kind can have multiple indexes(tables), and each index(table)
> > > can have multiple kinds.
>
> > > GQL is not like SQL, you need to relearn things, starting with getting
> > > the concept of tables out of your mind.  :)
>
> > > P.S.  The above description is not perfectly accurate as to how it
> > > works, but close enough to get the idea across.  :)

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Introducing App Engine SDK 1.3.0

[ajaxer]

my suggestions on appengine:
enable entities deleted by one click.
cancel the limit on offset index.
remain the limit on data fetched.

On Dec 15, 12:00 pm, "Jason (Google)"  wrote:
> Hi Everyone. We just released version 1.3.0 of the App Engine SDK for
> both Python and Java. The most notable change is the new experimental
> Blobstore API which allows billed apps to store files up to 50 MB. The
> release also includes some performance tweaks to the Java runtime.
>
> Blog 
> post:http://googleappengine.blogspot.com/2009/12/app-engine-sdk-130-releas...
>
> Release notes:
> Python:http://code.google.com/p/googleappengine/wiki/SdkReleaseNotes
> Java:http://code.google.com/p/googleappengine/wiki/SdkForJavaReleaseNotes
>
> Cheers!
> - Jason

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Is it possible to deploy Wave server on GAE ( will it be wihin free limits)

[yadoo]

Guys, where can I get the source code for the wave???

On Dec 8, 11:04 pm, mdipierro  wrote:
> I am not sure if this is what you are asking but here is a google wave
> service running on google app engine:
>
> http://wavedirectory.appspot.com
>
> Here is some howto:
>
> http://wavedirectory.appspot.com/init/default/wave?w=googlewave.com%2...
>
> Massimo
>
> On Dec 6, 1:00 am, saurabh  wrote:
>
>
>
> > Hi ,
>
> > I am trying o make an application based on wave server functionality
> > ( I dont want to use the Wave GUI ) just the wave server functionality
> > of realtime collobration.
>
> > But as fo now the no of wave users is very less and i cant ask my
> > users to join google wave so I am thinking of deploying the wave
> > server myself and iuse it as a backend. I want to know If I can deploy
> > the wave server on GAE server , Has anybody tried it
>
> > Thanks
> > Saurabh

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Is it possible to deploy Wave server on GAE ( will it be wihin free limits)

[yadoo]

Guys, where can I get the source code for the wave google project ??

On Dec 6, 11:00 am, saurabh  wrote:
> Hi ,
>
> I am trying o make an application based on wave server functionality
> ( I dont want to use the Wave GUI ) just the wave server functionality
> of realtime collobration.
>
> But as fo now the no of wave users is very less and i cant ask my
> users to join google wave so I am thinking of deploying the wave
> server myself and iuse it as a backend. I want to know If I can deploy
> the wave server on GAE server , Has anybody tried it
>
> Thanks
> Saurabh

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Cron jobs fail with error "Request was aborted after waiting too long to attempt to service your request."

[fhtino]

I have the same problem.  In my case 1 req/min.
Normally the execution time is 100 ms or less. Sometimes time is more
than 1 ms and I get http error 500 and
"Request was aborted after waiting too long to attempt to service your
request."

I don't understand the reason.


   fabrizio


On Dec 14, 1:11 pm, Abhi  wrote:
> Sometimes my cron jobs fail with a HTTP 500 error and message:-
>
> Request was aborted after waiting too long to attempt to service your
 ..

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Blob Store PUT

[oli]

It's nice that it's now possible to upload files larger than 1MB.

Do you have any plans to support PUT uploads for Blob Store? e.g. for
implementing a S3-like interface on appengine. Would the appengine
infrastructure allow this or is it impossible?

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Billing Setup Started for about 2 weeks

[Peter Ondruska]

I have setup billing for my application at Nov 30 and since then it is
still not activated and no other change in status since then. Can
please anybody at Google have a look, appid kaibo-www? Thank you

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Blob Store PUT

[G]

Perhaps this ancient PUT bug is part of the problem?

http://code.google.com/p/googleappengine/issues/detail?id=170

If so, try the workaround.

-if self.method != 'POST':
+if self.method not in ['POST','PUT']:

--
G


On Dec 15, 2:08 am, oli  wrote:
> It's nice that it's now possible to upload files larger than 1MB.
>
> Do you have any plans to support PUT uploads for Blob Store? e.g. for
> implementing a S3-like interface on appengine. Would the appengine
> infrastructure allow this or is it impossible?

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Introducing App Engine SDK 1.3.0

[Tim Hoffman]

Hi

There does not appear to be any provision for creating blobs via
remote_api.
Any thoughts on this ?

T


On Dec 15, 12:00 pm, "Jason (Google)"  wrote:
> Hi Everyone. We just released version 1.3.0 of the App Engine SDK for
> both Python and Java. The most notable change is the new experimental
> Blobstore API which allows billed apps to store files up to 50 MB. The
> release also includes some performance tweaks to the Java runtime.
>
> Blog 
> post:http://googleappengine.blogspot.com/2009/12/app-engine-sdk-130-releas...
>
> Release notes:
> Python:http://code.google.com/p/googleappengine/wiki/SdkReleaseNotes
> Java:http://code.google.com/p/googleappengine/wiki/SdkForJavaReleaseNotes
>
> Cheers!
> - Jason

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Introducing App Engine SDK 1.3.0

[Stephen Mayer]

Hi Jason,

Looks like an exciting new feature!  My concern after looking thru the
docs is how we tell what file the user actually uploaded ... was it an
image, a video, etc ... for validation and sanity checks?  It seems
like you could use the fetch api to get the actual file and work on it
in some way, but that seems a bit ackward.  I do think its a great
feature tho for admin uploaded content that is trusted. Or am I
missing something here?

Stephen

On Dec 14, 11:00 pm, "Jason (Google)"  wrote:
> Hi Everyone. We just released version 1.3.0 of the App Engine SDK for
> both Python and Java. The most notable change is the new experimental
> Blobstore API which allows billed apps to store files up to 50 MB. The
> release also includes some performance tweaks to the Java runtime.
>
> Blog 
> post:http://googleappengine.blogspot.com/2009/12/app-engine-sdk-130-releas...
>
> Release notes:
> Python:http://code.google.com/p/googleappengine/wiki/SdkReleaseNotes
> Java:http://code.google.com/p/googleappengine/wiki/SdkForJavaReleaseNotes
>
> Cheers!
> - Jason

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Manually entering ReferenceProperty Keys into the DataStore Viewer?

[kor]

Hi Scott,

I would like to recommend to perform a quick look on AppWrench(http://
appwrench.onpositive.com). AppWrench has full support for editing
relations as well as lists and other data types supported by Google
App Engine.

Regards,
Pavel

On Nov 9, 11:03 pm, DarkMarmot  wrote:
> Hi,
>
>    I'm trying to set up some initial object relationships in my
> application, and I can set them up through code without a problem, but
> when I try to copy and paste the same keys into the edit fields of 
> thedatastoreviewer, it never accepts the update.
>
>   Is data entry for ReferenceProperty fields broken in theDataStoreviewer? Or 
> am I doing something stupid? Any help would be
> appreciated!
>
> Thanks,
> Scott Southworth

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Blob Store PUT

[oli]

Unfortunately it's not that simple. For a Blob Store upload you have
to generate a temporary URL like
http://localhost:8080/_ah/upload/agN3b3hyGwsSFV9fQmxvYlVwbG9hZFNlc3Npb25fXxgUDA

With a raw PUT request / S3-like API you don't even use multipart/form-
data, just send the blob in the body of the request. What's more
important: some random generated URL doesn't work. You send the blob
to a specific URL the client know in advance, like
http://localhost:8080/bucket/steve_mcqueen.jpg

This is not supported by the Blob Store service. And I wonder if it's
even possible to support it in appengine or if this is another
limitation that will not go away anytime soon.

On Dec 15, 1:59 pm, G  wrote:
> Perhaps this ancient PUT bug is part of the problem?
>
> http://code.google.com/p/googleappengine/issues/detail?id=170
>
> If so, try the workaround.
>
> -        if self.method != 'POST':
> +        if self.method not in ['POST','PUT']:
>
> --
> G

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Is it possible to deploy Wave server on GAE ( will it be wihin free limits)

[Stephen]

On Dec 15, 6:46 am, yadoo  wrote:
> Guys, where can I get the source code for the wave google project ??


http://code.google.com/p/wave-protocol/

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: how to delete a table/entity?

[Stephen]

On Dec 15, 8:04 am, ajaxer  wrote:
>
> also 1000 index limit makes it not possible to fetcher older data on
> paging.
>
> for if we need an indexed page more than 1 items,
> it would cost us a lot of cpu time to calculate the base for GQL
> to fetch the data with index less than 1000.


http://code.google.com/appengine/articles/paging.html

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Index updating time as number of entities grows

[Nick Johnson (Google)]

Hi Nickolas,

On Tue, Dec 15, 2009 at 6:39 AM, Nickolas Daskalou wrote:

> I've read that the time for a Datastore query is independent of the
> number of entities in an app, which is great.
>
> However, does index updating take longer as the number of entities
> increases?
>

No, the time required for a datastore put is also independent of the size of
the datastore.


>
> Eg. If I'm making a high score tracking system (a Player kind with one
> of the properties being high_score) and want to order by the highest
> score for each player, as I update a player's high score will the
> index be updated in the same amount of time if there were 100 Player
> entities as it would if there were 100,000,000 Player entities?
>
> I already know that the put() time will not change because it returns
> before the index has been updated, I'm just concerned at how long the
> high scores table (Player.all().order("-high_score")) will take to
> reflect the changes.
>

Indexes are updated synchronously - eg, before the put operation completes.

-Nick Johnson


>
> --
>
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
>


-- 
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Unable to Delete Blobs in Admin Console

[jorge]

I have a few blobs that I have uploaded using the new blob feature,
but I am unable to delete them via the Blob Viewer in the admin
console, I am getting 500 errors.

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Introducing App Engine SDK 1.3.0

[Barry Hunter]

Stephen, have you seen?
http://code.google.com/appengine/docs/python/blobstore/blobinfoclass.html
looks like you should be able to query that to get the mime-type
(admittedly it is the client provided one, not a server-side verified
one. )

2009/12/15 Stephen Mayer :
> Hi Jason,
>
> Looks like an exciting new feature!  My concern after looking thru the
> docs is how we tell what file the user actually uploaded ... was it an
> image, a video, etc ... for validation and sanity checks?  It seems
> like you could use the fetch api to get the actual file and work on it
> in some way, but that seems a bit ackward.  I do think its a great
> feature tho for admin uploaded content that is trusted. Or am I
> missing something here?
>
> Stephen
>
> On Dec 14, 11:00 pm, "Jason (Google)"  wrote:
>> Hi Everyone. We just released version 1.3.0 of the App Engine SDK for
>> both Python and Java. The most notable change is the new experimental
>> Blobstore API which allows billed apps to store files up to 50 MB. The
>> release also includes some performance tweaks to the Java runtime.
>>
>> Blog 
>> post:http://googleappengine.blogspot.com/2009/12/app-engine-sdk-130-releas...
>>
>> Release notes:
>> Python:http://code.google.com/p/googleappengine/wiki/SdkReleaseNotes
>> Java:http://code.google.com/p/googleappengine/wiki/SdkForJavaReleaseNotes
>>
>> Cheers!
>> - Jason
>
> --
>
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to 
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/google-appengine?hl=en.
>
>

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: "Request was aborted after waiting too long" followed by random DeadlineExceededError on import.

[Jason C]

Ikai,

We see daily DeadlineExceededErrors on app id 'steprep' from 6.30am to
7.30am (log time).

Can you look into that as well?

Thanks,
j

On Dec 14, 3:32 pm, "Ikai L (Google)"  wrote:
> Do you see that it's consistent at the same times? What's your application
> ID? I'll look into it.
>
>
>
>
>
> On Mon, Dec 14, 2009 at 11:28 AM, Dave Peck  wrote:
> > Hello,
>
> > I have an app (citygoround.org) that, especially in the morning, often
> > has 10-15 minutes of outright downtime due to server errors.
>
> > Looking into it, I see that right before the downtime starts, a few
> > requests log the following warning message:
>
> >    > Request was aborted after waiting too long to attempt to service
> > your request.
> >    > Most likely, this indicates that you have reached your
> > simultaneous dynamic request limit.
>
> > I'm certainly not over my limit, but I can believe that the request in
> > question could take a while. (I'll get to the details of that request
> > in a moment.)
>
> > Immediately after these warnings, my app has a large amount of time
> > (10+ minutes) where *all requests* -- no matter how unthreatening --
> > raise a DeadlineExceededError. Usually this is raised during the
> > import of an innocuous module like "re" or "time" or perhaps a Django
> > 1.1 module. (We use use_library.)
>
> > My best theory at the moment is that:
>
> > 1. It's a cold start, so nothing is cached.
> > 2. App Engine encounters the high latency request and bails.
> > 3. We probably inadvertently catch the DeadlineExceededError, so the
> > runtime doesn't clean up properly.
> > 4. Future requests are left in a busted state.
>
> > Does this sound at all reasonable? I see a few related issues (2396,
> > 2266, and 1409) but no firm/completely clear discussion of what's
> > happening in any of them.
>
> > Thanks,
> > Dave
>
> > PS:
>
> > The specifics about our high latency request are *not* strictly
> > relevant to the larger problem I'm having, but I will include them
> > because I have a second "side" question to ask about it.
>
> > The "high latency" request is serving an image. Our app lets users
> > upload images and we store them in the data store. When serving an
> > image, our handler:
>
> > 1. Checks to see if the bytes for the image are in memcache, and if so
> > returns them immediately.
> > 2. Otherwise grabs the image from the datastore, and if it is smaller
> > than 64K, adds the bytes to the memcache
> > 3. Returns the result
>
> > I'm wondering if using memcache in this way is a smart idea -- it may
> > very well be the cause of our latency issues. It's hard to tell.
>
> > Alternatively, the issue could be: we have a page that shows a large
> > number (~100) of such images. If someone requests this page, we may
> > have a lot of simultaneous image-producing requests happening at the
> > same time. Perhaps _this_ is the root cause of the original "Request
> > was aborted" issue?
>
> > Just not sure here...
>
> > --
>
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-appeng...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine+unsubscr...@googlegroups.com > e...@googlegroups.com>
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine?hl=en.
>
> --
> Ikai Lan
> Developer Programs Engineer, Google App Engine

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Is it possible to deploy Wave server on GAE ( will it be wihin free limits)

[yadoo]

Wait, I think this is just the Protocol, am I right? I need GUI(to be
more specific GWT) codes also...

On Dec 15, 7:32 pm, Stephen  wrote:
> On Dec 15, 6:46 am, yadoo  wrote:
>
> > Guys, where can I get the source code for the wave google project ??
>
> http://code.google.com/p/wave-protocol/

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Java or Python; Which should I recommend to others who are very sensitive to costs?

[Takashi Matsuo]

Hello,

Today I noticed that App Engine Java environment became much faster
then before. The spin up cost is about 700cpu_ms with the simplest
servlet. Additionally, when it comes to serving with a hot instance,
the cost reduces to 0-2cpu_ms, while python environment takes about
5-7cpu_ms even with the simplest handler.

To make it simple here, lets say Java takes 1cpu_ms while Python takes
6cpu_ms for serving very simple page.
How many requests can they serve with 1 cpu hour?

Java: 360 requests/1 cpu hour
Python: 60 requests/1 cpu hour

This is a big deference; 6 times! If your app exceeds free quota, this
deference can impact total amount of costs significantly. I'm a big
Python fan and I have believed that appengine Python runtime is
superior to Java runtime, so I've been trying to persuade others to
use Python rather than Java for now.

Having said that, today it turns out for me that Java runtime is much
more cost effective than Python runtime in some cases, so should I
recommend others to use Apppengine Java if they are very sensitive to
cpu costs?

I'd appreciate if anyone could share one's thoughts/experiences on this.

TIA

-- 
Takashi Matsuo
Kay's daddy

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Blob Store PUT

[G]

Ah, my bad.  Without reading the docs

"The Blobstore API allows your app to serve data objects, called
blobs, that are much larger than the size allowed for objects in the
Datastore service."

"Apps never access blob data directly; instead, blobs are created
indirectly, by a submitted web form or other HTTP POST request, and
served indirectly through the Blobstore API."

I didn't know the new blobs were something other than a larger size
limit for BlobProperty, and I didn't know direct access wasn't
implemented.

Seeing the mention of PUT, which I know is broken (application/x-www-
form-urlencoded and any other format, for small or large sized data),
I thought there might be an easy fix.  Obviously I was wrong, the
context is more general.  Without reading the source, I don't know if
Blobstore is a workaround for Datastore limitations, but would guess
that pulling 50M through a POST (or patched PUT) would run into
Datastore size issues (presumably that is why Blobstore is (currently)
what it is).

--
G


On Dec 15, 7:17 am, oli  wrote:
> Unfortunately it's not that simple. For a Blob Store upload you have
> to generate a temporary URL 
> likehttp://localhost:8080/_ah/upload/agN3b3hyGwsSFV9fQmxvYlVwbG9hZFNlc3Np...
>
> With a raw PUT request / S3-like API you don't even use multipart/form-
> data, just send the blob in the body of the request. What's more
> important: some random generated URL doesn't work. You send the blob
> to a specific URL the client know in advance, 
> likehttp://localhost:8080/bucket/steve_mcqueen.jpg
>
> This is not supported by the Blob Store service. And I wonder if it's
> even possible to support it in appengine or if this is another
> limitation that will not go away anytime soon.
>
> On Dec 15, 1:59 pm, G  wrote:
>
> > Perhaps this ancient PUT bug is part of the problem?
>
> >http://code.google.com/p/googleappengine/issues/detail?id=170
>
> > If so, try the workaround.
>
> > -        if self.method != 'POST':
> > +        if self.method not in ['POST','PUT']:
>
> > --
> > G

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Java or Python; Which should I recommend to others who are very sensitive to costs?

[trung]

How about jruby with rails? :)

On Dec 15, 10:02 am, Takashi Matsuo  wrote:
> Hello,
>
> Today I noticed that App Engine Java environment became much faster
> then before. The spin up cost is about 700cpu_ms with the simplest
> servlet. Additionally, when it comes to serving with a hot instance,
> the cost reduces to 0-2cpu_ms, while python environment takes about
> 5-7cpu_ms even with the simplest handler.
>
> To make it simple here, lets say Java takes     1cpu_ms while Python takes
> 6cpu_ms for serving very simple page.
> How many requests can they serve with 1 cpu hour?
>
> Java: 360 requests/1 cpu hour
> Python: 60 requests/1 cpu hour
>
> This is a big deference; 6 times! If your app exceeds free quota, this
> deference can impact total amount of costs significantly. I'm a big
> Python fan and I have believed that appengine Python runtime is
> superior to Java runtime, so I've been trying to persuade others to
> use Python rather than Java for now.
>
> Having said that, today it turns out for me that Java runtime is much
> more cost effective than Python runtime in some cases, so should I
> recommend others to use Apppengine Java if they are very sensitive to
> cpu costs?
>
> I'd appreciate if anyone could share one's thoughts/experiences on this.
>
> TIA
>
> --
> Takashi Matsuo
> Kay's daddy

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Introducing App Engine SDK 1.3.0

[trung]

This is awesome.

But the image API limit is still capped at 1MB!!!

I still rather be able to resize a 50MB uploaded image down to 1MB or
less to cut down the time and bandwidth.

I assume that increasing the Image API limit is the next logical
step. :)


On Dec 14, 8:00 pm, "Jason (Google)"  wrote:
> Hi Everyone. We just released version 1.3.0 of the App Engine SDK for
> both Python and Java. The most notable change is the new experimental
> Blobstore API which allows billed apps to store files up to 50 MB. The
> release also includes some performance tweaks to the Java runtime.
>
> Blog 
> post:http://googleappengine.blogspot.com/2009/12/app-engine-sdk-130-releas...
>
> Release notes:
> Python:http://code.google.com/p/googleappengine/wiki/SdkReleaseNotes
> Java:http://code.google.com/p/googleappengine/wiki/SdkForJavaReleaseNotes
>
> Cheers!
> - Jason

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Trouble creating an application

[Steve S.]

1. I begin at the page  and click
"Create an application" and am taken to .
2. I check the availability of a domain. It is available (I've tried
random URLS, so domain availability is not the issue.)
3. I am taken back to . My domain
cannot be re-registered AND I can't see where to go from here.

I don't think I have done anything wrong. Is there someone I can
contact about this? I hope I can retrieve the original URL that I
requested, too!

Also it may be worth noting that I am using Google Chrome 4.0.266.0 as
my browser.

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Java or Python; Which should I recommend to others who are very sensitive to costs?

[Michael Langford]

>If your app exceeds free quota, this deference can impact total amount of 
>costs significantly.

As can the difference in price/productivity between the two languages
as far as total cost of development goes.

The incremental costs on appengine between the two technologies are
dwarfed by the productivity differences between them for most
project's I'd imagine. I'd say if you're charging clients the same
amount to develop an app in Java as you are in Python, you're vastly
overcharging for the python or undercharging for the java (baring
language specific libraries available for one platform and not on the
other).

For a huge number of programs, the difference in development time
between a higher level language and a lower level language is never
recouped by increased runtime in the faster, lower level language.

For many apps, even java developers are faster writing python than
java (Although they all seem to go for Ruby instead).


 --Michael


On Tue, Dec 15, 2009 at 1:02 PM, Takashi Matsuo
 wrote:
> Hello,
>
> Today I noticed that App Engine Java environment became much faster
> then before. The spin up cost is about 700cpu_ms with the simplest
> servlet. Additionally, when it comes to serving with a hot instance,
> the cost reduces to 0-2cpu_ms, while python environment takes about
> 5-7cpu_ms even with the simplest handler.
>
> To make it simple here, lets say Java takes     1cpu_ms while Python takes
> 6cpu_ms for serving very simple page.
> How many requests can they serve with 1 cpu hour?
>
> Java: 360 requests/1 cpu hour
> Python: 60 requests/1 cpu hour
>
> This is a big deference; 6 times! If your app exceeds free quota, this
> deference can impact total amount of costs significantly. I'm a big
> Python fan and I have believed that appengine Python runtime is
> superior to Java runtime, so I've been trying to persuade others to
> use Python rather than Java for now.
>
> Having said that, today it turns out for me that Java runtime is much
> more cost effective than Python runtime in some cases, so should I
> recommend others to use Apppengine Java if they are very sensitive to
> cpu costs?
>
> I'd appreciate if anyone could share one's thoughts/experiences on this.
>
> TIA
>
> --
> Takashi Matsuo
> Kay's daddy
>
> --
>
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to 
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/google-appengine?hl=en.
>
>
>



-- 
Michael Langford
Phone: 404-386-0495
Consulting: http://www.RowdyLabs.com

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Index updating time as number of entities grows

[Nickolas Daskalou]

Hi Nick,

Thanks for your incredible continued support in these groups!

Re: put() returning after indexes are updated - I confused myself with
the milestone A/B scenario from the transaction isolation article
(http://code.google.com/appengine/articles/
transaction_isolation.html), thanks for clearing that up.

Just to make sure I'm understanding correctly, if I do something like:

  # Get the top 2 Players
  players = Player.all().order("-high_score").fetch(2)
  # Move 2nd place above 1st place
  players[1].high_score = players[0].high_score + 1
  players[1].put()
  # Get the #1 Player (which should be the player that was 2nd)
  player = Players.all().order("-high_score").fetch(1)[0]

will the time taken to run this script be basically the same with 100
Player entities as it would with 100,000,000? If this is the case
then, much like low carb beer, the Datastore is truly a modern day
miracle.

Nick


On Dec 16, 3:46 am, "Nick Johnson (Google)" 
wrote:
> Hi Nickolas,
>
> On Tue, Dec 15, 2009 at 6:39 AM, Nickolas Daskalou wrote:
>
> > I've read that the time for a Datastore query is independent of the
> > number of entities in an app, which is great.
>
> > However, does index updating take longer as the number of entities
> > increases?
>
> No, the time required for a datastore put is also independent of the size of
> the datastore.
>
>
>
> > Eg. If I'm making a high score tracking system (a Player kind with one
> > of the properties being high_score) and want to order by the highest
> > score for each player, as I update a player's high score will the
> > index be updated in the same amount of time if there were 100 Player
> > entities as it would if there were 100,000,000 Player entities?
>
> > I already know that the put() time will not change because it returns
> > before the index has been updated, I'm just concerned at how long the
> > high scores table (Player.all().order("-high_score")) will take to
> > reflect the changes.
>
> Indexes are updated synchronously - eg, before the put operation completes.
>
> -Nick Johnson
>
>
>
> > --
>
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-appeng...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine+unsubscr...@googlegroups.com
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine?hl=en.
>
> --
> Nick Johnson, Developer Programs Engineer, App Engine
> Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
> 368047

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Trouble creating an application

[Steve S.]

Problem resolved: The apps were created in my Google Apps account. I
wish the sign-up page would have told me this, though!

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: "Request was aborted after waiting too long" followed by random DeadlineExceededError on import.

[Dave Peck]

Hi Ikai,

Any further details on your end? I get the feeling we're not the only
ones, and we've experienced very serious downtime in the last ~48
hours.

This is a critical issue for us to resolve, but at the same time we
lack key pieces of data that would help us solve it on our own...

Thanks,
Dave

On Dec 15, 9:14 am, Jason C  wrote:
> Ikai,
>
> We see daily DeadlineExceededErrors on app id 'steprep' from 6.30am to
> 7.30am (log time).
>
> Can you look into that as well?
>
> Thanks,
> j
>
> On Dec 14, 3:32 pm, "Ikai L (Google)"  wrote:
>
>
>
> > Do you see that it's consistent at the same times? What's your application
> > ID? I'll look into it.
>
> > On Mon, Dec 14, 2009 at 11:28 AM, Dave Peck  wrote:
> > > Hello,
>
> > > I have an app (citygoround.org) that, especially in the morning, often
> > > has 10-15 minutes of outright downtime due to server errors.
>
> > > Looking into it, I see that right before the downtime starts, a few
> > > requests log the following warning message:
>
> > >    > Request was aborted after waiting too long to attempt to service
> > > your request.
> > >    > Most likely, this indicates that you have reached your
> > > simultaneous dynamic request limit.
>
> > > I'm certainly not over my limit, but I can believe that the request in
> > > question could take a while. (I'll get to the details of that request
> > > in a moment.)
>
> > > Immediately after these warnings, my app has a large amount of time
> > > (10+ minutes) where *all requests* -- no matter how unthreatening --
> > > raise a DeadlineExceededError. Usually this is raised during the
> > > import of an innocuous module like "re" or "time" or perhaps a Django
> > > 1.1 module. (We use use_library.)
>
> > > My best theory at the moment is that:
>
> > > 1. It's a cold start, so nothing is cached.
> > > 2. App Engine encounters the high latency request and bails.
> > > 3. We probably inadvertently catch the DeadlineExceededError, so the
> > > runtime doesn't clean up properly.
> > > 4. Future requests are left in a busted state.
>
> > > Does this sound at all reasonable? I see a few related issues (2396,
> > > 2266, and 1409) but no firm/completely clear discussion of what's
> > > happening in any of them.
>
> > > Thanks,
> > > Dave
>
> > > PS:
>
> > > The specifics about our high latency request are *not* strictly
> > > relevant to the larger problem I'm having, but I will include them
> > > because I have a second "side" question to ask about it.
>
> > > The "high latency" request is serving an image. Our app lets users
> > > upload images and we store them in the data store. When serving an
> > > image, our handler:
>
> > > 1. Checks to see if the bytes for the image are in memcache, and if so
> > > returns them immediately.
> > > 2. Otherwise grabs the image from the datastore, and if it is smaller
> > > than 64K, adds the bytes to the memcache
> > > 3. Returns the result
>
> > > I'm wondering if using memcache in this way is a smart idea -- it may
> > > very well be the cause of our latency issues. It's hard to tell.
>
> > > Alternatively, the issue could be: we have a page that shows a large
> > > number (~100) of such images. If someone requests this page, we may
> > > have a lot of simultaneous image-producing requests happening at the
> > > same time. Perhaps _this_ is the root cause of the original "Request
> > > was aborted" issue?
>
> > > Just not sure here...
>
> > > --
>
> > > You received this message because you are subscribed to the Google Groups
> > > "Google App Engine" group.
> > > To post to this group, send email to google-appeng...@googlegroups.com.
> > > To unsubscribe from this group, send email to
> > > google-appengine+unsubscr...@googlegroups.com > >  e...@googlegroups.com>
> > > .
> > > For more options, visit this group at
> > >http://groups.google.com/group/google-appengine?hl=en.
>
> > --
> > Ikai Lan
> > Developer Programs Engineer, Google App Engine

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Java or Python; Which should I recommend to others who are very sensitive to costs?

[Jesaja Everling]

Maybe the people asking if it's possible to run Jython on App Engine
were on to something, after all! ;)
I also think choosing the toolkit that works best for the task at hand
is more important. Furthermore I'm confident the App Engine team will
speed up the Python runtime further; maybe that's part of what Unladen
Swallow is meant for.
A very interesting observation, though!

Best Regards,

Jesaja Everling


On Tue, Dec 15, 2009 at 7:34 PM, Michael Langford
 wrote:
>>If your app exceeds free quota, this deference can impact total amount of 
>>costs significantly.
>
> As can the difference in price/productivity between the two languages
> as far as total cost of development goes.
>
> The incremental costs on appengine between the two technologies are
> dwarfed by the productivity differences between them for most
> project's I'd imagine. I'd say if you're charging clients the same
> amount to develop an app in Java as you are in Python, you're vastly
> overcharging for the python or undercharging for the java (baring
> language specific libraries available for one platform and not on the
> other).
>
> For a huge number of programs, the difference in development time
> between a higher level language and a lower level language is never
> recouped by increased runtime in the faster, lower level language.
>
> For many apps, even java developers are faster writing python than
> java (Although they all seem to go for Ruby instead).
>
>
>     --Michael
>
>
> On Tue, Dec 15, 2009 at 1:02 PM, Takashi Matsuo
>  wrote:
>> Hello,
>>
>> Today I noticed that App Engine Java environment became much faster
>> then before. The spin up cost is about 700cpu_ms with the simplest
>> servlet. Additionally, when it comes to serving with a hot instance,
>> the cost reduces to 0-2cpu_ms, while python environment takes about
>> 5-7cpu_ms even with the simplest handler.
>>
>> To make it simple here, lets say Java takes     1cpu_ms while Python takes
>> 6cpu_ms for serving very simple page.
>> How many requests can they serve with 1 cpu hour?
>>
>> Java: 360 requests/1 cpu hour
>> Python: 60 requests/1 cpu hour
>>
>> This is a big deference; 6 times! If your app exceeds free quota, this
>> deference can impact total amount of costs significantly. I'm a big
>> Python fan and I have believed that appengine Python runtime is
>> superior to Java runtime, so I've been trying to persuade others to
>> use Python rather than Java for now.
>>
>> Having said that, today it turns out for me that Java runtime is much
>> more cost effective than Python runtime in some cases, so should I
>> recommend others to use Apppengine Java if they are very sensitive to
>> cpu costs?
>>
>> I'd appreciate if anyone could share one's thoughts/experiences on this.
>>
>> TIA
>>
>> --
>> Takashi Matsuo
>> Kay's daddy
>>
>> --
>>
>> You received this message because you are subscribed to the Google Groups 
>> "Google App Engine" group.
>> To post to this group, send email to google-appeng...@googlegroups.com.
>> To unsubscribe from this group, send email to 
>> google-appengine+unsubscr...@googlegroups.com.
>> For more options, visit this group at 
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>>
>>
>
>
>
> --
> Michael Langford
> Phone: 404-386-0495
> Consulting: http://www.RowdyLabs.com
>
> --
>
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to 
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/google-appengine?hl=en.
>
>
>

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Introducing App Engine SDK 1.3.0

[Matthew Blain]

While the limit for passing data directly to the Images (or other)
APIs has not changed, you can pass a Blob key to the Images API to do
exactly what you want: convert a 50MB uploaded image to a smaller
image.

More information here:
 
http://code.google.com/appengine/docs/python/images/overview.html#Transforming_Images_from_the_Blobstore
 
http://code.google.com/appengine/docs/java/images/overview.html#Transforming_Images_from_the_Blobstore

--Matthew

On Dec 15, 10:14 am, trung  wrote:
> This is awesome.
>
> But the image API limit is still capped at 1MB!!!
>
> I still rather be able to resize a 50MB uploaded image down to 1MB or
> less to cut down the time and bandwidth.
>
> I assume that increasing the Image API limit is the next logical
> step. :)
>
> On Dec 14, 8:00 pm, "Jason (Google)"  wrote:
>
>
>
> > Hi Everyone. We just released version 1.3.0 of the App Engine SDK for
> > both Python and Java. The most notable change is the new experimental
> > Blobstore API which allows billed apps to store files up to 50 MB. The
> > release also includes some performance tweaks to the Java runtime.
>
> > Blog 
> > post:http://googleappengine.blogspot.com/2009/12/app-engine-sdk-130-releas...
>
> > Release notes:
> > Python:http://code.google.com/p/googleappengine/wiki/SdkReleaseNotes
> > Java:http://code.google.com/p/googleappengine/wiki/SdkForJavaReleaseNotes
>
> > Cheers!
> > - Jason

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: "Request was aborted after waiting too long" followed by random DeadlineExceededError on import.

[Ikai L (Google)]

Dave,

You're correct that this is likely affecting other applications, but it's
not a global issue. There are hotspots in the cloud that we notice are being
especially impacted during certain times of the day. We're actively working
on addressing these issues, but in the meantime, there are manual steps we
can try to prevent your applications from becoming resource starved. We do
these on a one-off basis and reserve them only for applications that seem to
exhibit the behavior of seeing DeadlineExceeded on simple actions (not
initial JVM startup), and at fairly predictable intervals during the day.
I've taken these steps to try to remedy your application. Can you let us
know if these seem to help? If not, they may indicate that something is
going on with your application code, though that does not seem like the case
here.


On Tue, Dec 15, 2009 at 10:54 AM, Dave Peck  wrote:

> Hi Ikai,
>
> Any further details on your end? I get the feeling we're not the only
> ones, and we've experienced very serious downtime in the last ~48
> hours.
>
> This is a critical issue for us to resolve, but at the same time we
> lack key pieces of data that would help us solve it on our own...
>
> Thanks,
> Dave
>
> On Dec 15, 9:14 am, Jason C  wrote:
> > Ikai,
> >
> > We see daily DeadlineExceededErrors on app id 'steprep' from 6.30am to
> > 7.30am (log time).
> >
> > Can you look into that as well?
> >
> > Thanks,
> > j
> >
> > On Dec 14, 3:32 pm, "Ikai L (Google)"  wrote:
> >
> >
> >
> > > Do you see that it's consistent at the same times? What's your
> application
> > > ID? I'll look into it.
> >
> > > On Mon, Dec 14, 2009 at 11:28 AM, Dave Peck 
> wrote:
> > > > Hello,
> >
> > > > I have an app (citygoround.org) that, especially in the morning,
> often
> > > > has 10-15 minutes of outright downtime due to server errors.
> >
> > > > Looking into it, I see that right before the downtime starts, a few
> > > > requests log the following warning message:
> >
> > > >> Request was aborted after waiting too long to attempt to service
> > > > your request.
> > > >> Most likely, this indicates that you have reached your
> > > > simultaneous dynamic request limit.
> >
> > > > I'm certainly not over my limit, but I can believe that the request
> in
> > > > question could take a while. (I'll get to the details of that request
> > > > in a moment.)
> >
> > > > Immediately after these warnings, my app has a large amount of time
> > > > (10+ minutes) where *all requests* -- no matter how unthreatening --
> > > > raise a DeadlineExceededError. Usually this is raised during the
> > > > import of an innocuous module like "re" or "time" or perhaps a Django
> > > > 1.1 module. (We use use_library.)
> >
> > > > My best theory at the moment is that:
> >
> > > > 1. It's a cold start, so nothing is cached.
> > > > 2. App Engine encounters the high latency request and bails.
> > > > 3. We probably inadvertently catch the DeadlineExceededError, so the
> > > > runtime doesn't clean up properly.
> > > > 4. Future requests are left in a busted state.
> >
> > > > Does this sound at all reasonable? I see a few related issues (2396,
> > > > 2266, and 1409) but no firm/completely clear discussion of what's
> > > > happening in any of them.
> >
> > > > Thanks,
> > > > Dave
> >
> > > > PS:
> >
> > > > The specifics about our high latency request are *not* strictly
> > > > relevant to the larger problem I'm having, but I will include them
> > > > because I have a second "side" question to ask about it.
> >
> > > > The "high latency" request is serving an image. Our app lets users
> > > > upload images and we store them in the data store. When serving an
> > > > image, our handler:
> >
> > > > 1. Checks to see if the bytes for the image are in memcache, and if
> so
> > > > returns them immediately.
> > > > 2. Otherwise grabs the image from the datastore, and if it is smaller
> > > > than 64K, adds the bytes to the memcache
> > > > 3. Returns the result
> >
> > > > I'm wondering if using memcache in this way is a smart idea -- it may
> > > > very well be the cause of our latency issues. It's hard to tell.
> >
> > > > Alternatively, the issue could be: we have a page that shows a large
> > > > number (~100) of such images. If someone requests this page, we may
> > > > have a lot of simultaneous image-producing requests happening at the
> > > > same time. Perhaps _this_ is the root cause of the original "Request
> > > > was aborted" issue?
> >
> > > > Just not sure here...
> >
> > > > --
> >
> > > > You received this message because you are subscribed to the Google
> Groups
> > > > "Google App Engine" group.
> > > > To post to this group, send email to
> google-appeng...@googlegroups.com.
> > > > To unsubscribe from this group, send email to
> > > > google-appengine+unsubscr...@googlegroups.com e...@googlegroups.com>
> > > > .
> > > > For more options, visit this group at
> > > >http://groups.google.com/group/google-appengine?hl=en.
> >
> > > --
> > > Ika

[google-appengine] Re: "Request was aborted after waiting too long" followed by random DeadlineExceededError on import.

[Dave Peck]

Ikai,

We'll keep an eye on our app for the next ~24 hours and report back.

At what time did you make the changes to our instance? We had
substantial downtime earlier today, alas.

Can you provide any details about what sort of change was made?

Thanks,
Dave

On Dec 15, 11:26 am, "Ikai L (Google)"  wrote:
> Dave,
>
> You're correct that this is likely affecting other applications, but it's
> not a global issue. There are hotspots in the cloud that we notice are being
> especially impacted during certain times of the day. We're actively working
> on addressing these issues, but in the meantime, there are manual steps we
> can try to prevent your applications from becoming resource starved. We do
> these on a one-off basis and reserve them only for applications that seem to
> exhibit the behavior of seeing DeadlineExceeded on simple actions (not
> initial JVM startup), and at fairly predictable intervals during the day.
> I've taken these steps to try to remedy your application. Can you let us
> know if these seem to help? If not, they may indicate that something is
> going on with your application code, though that does not seem like the case
> here.
>
>
>
>
>
> On Tue, Dec 15, 2009 at 10:54 AM, Dave Peck  wrote:
> > Hi Ikai,
>
> > Any further details on your end? I get the feeling we're not the only
> > ones, and we've experienced very serious downtime in the last ~48
> > hours.
>
> > This is a critical issue for us to resolve, but at the same time we
> > lack key pieces of data that would help us solve it on our own...
>
> > Thanks,
> > Dave
>
> > On Dec 15, 9:14 am, Jason C  wrote:
> > > Ikai,
>
> > > We see daily DeadlineExceededErrors on app id 'steprep' from 6.30am to
> > > 7.30am (log time).
>
> > > Can you look into that as well?
>
> > > Thanks,
> > > j
>
> > > On Dec 14, 3:32 pm, "Ikai L (Google)"  wrote:
>
> > > > Do you see that it's consistent at the same times? What's your
> > application
> > > > ID? I'll look into it.
>
> > > > On Mon, Dec 14, 2009 at 11:28 AM, Dave Peck 
> > wrote:
> > > > > Hello,
>
> > > > > I have an app (citygoround.org) that, especially in the morning,
> > often
> > > > > has 10-15 minutes of outright downtime due to server errors.
>
> > > > > Looking into it, I see that right before the downtime starts, a few
> > > > > requests log the following warning message:
>
> > > > >    > Request was aborted after waiting too long to attempt to service
> > > > > your request.
> > > > >    > Most likely, this indicates that you have reached your
> > > > > simultaneous dynamic request limit.
>
> > > > > I'm certainly not over my limit, but I can believe that the request
> > in
> > > > > question could take a while. (I'll get to the details of that request
> > > > > in a moment.)
>
> > > > > Immediately after these warnings, my app has a large amount of time
> > > > > (10+ minutes) where *all requests* -- no matter how unthreatening --
> > > > > raise a DeadlineExceededError. Usually this is raised during the
> > > > > import of an innocuous module like "re" or "time" or perhaps a Django
> > > > > 1.1 module. (We use use_library.)
>
> > > > > My best theory at the moment is that:
>
> > > > > 1. It's a cold start, so nothing is cached.
> > > > > 2. App Engine encounters the high latency request and bails.
> > > > > 3. We probably inadvertently catch the DeadlineExceededError, so the
> > > > > runtime doesn't clean up properly.
> > > > > 4. Future requests are left in a busted state.
>
> > > > > Does this sound at all reasonable? I see a few related issues (2396,
> > > > > 2266, and 1409) but no firm/completely clear discussion of what's
> > > > > happening in any of them.
>
> > > > > Thanks,
> > > > > Dave
>
> > > > > PS:
>
> > > > > The specifics about our high latency request are *not* strictly
> > > > > relevant to the larger problem I'm having, but I will include them
> > > > > because I have a second "side" question to ask about it.
>
> > > > > The "high latency" request is serving an image. Our app lets users
> > > > > upload images and we store them in the data store. When serving an
> > > > > image, our handler:
>
> > > > > 1. Checks to see if the bytes for the image are in memcache, and if
> > so
> > > > > returns them immediately.
> > > > > 2. Otherwise grabs the image from the datastore, and if it is smaller
> > > > > than 64K, adds the bytes to the memcache
> > > > > 3. Returns the result
>
> > > > > I'm wondering if using memcache in this way is a smart idea -- it may
> > > > > very well be the cause of our latency issues. It's hard to tell.
>
> > > > > Alternatively, the issue could be: we have a page that shows a large
> > > > > number (~100) of such images. If someone requests this page, we may
> > > > > have a lot of simultaneous image-producing requests happening at the
> > > > > same time. Perhaps _this_ is the root cause of the original "Request
> > > > > was aborted" issue?
>
> > > > > Just not sure here...
>
> > > > > --
>
> > > > > You received this message be

[google-appengine] Re: Google App Engine for a huge website

[Rafe]

  Felix,

  The timing of your question is excellent.  Last night we just
released App Engine 1.3.0 which adds the BlobStore API.  This new API
supports file upload and download up to 50MB.

  
http://googleappengine.blogspot.com/2009/12/app-engine-sdk-130-released-including.html

  At least that's an answer for the file size limitation part of your
question.

  - Rafe Kaplan

On Dec 14, 11:57 am, Felix  wrote:
> Hi, I am thinking about using Google App Engine.It is going to be a
> huge website. In that case, what is your piece of advice about using
> Google App Engine. I heard GAE has restrictions like we cannot store
> images or files more than 1MB limit(they are going to change this from
> what I read in the GAE roadmap),query is limited to 1000 results, and
> I am also going to se web2py with GAE. So I would like to know your
> comments.
>
> Thanks

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Introducing App Engine SDK 1.3.0

[Dave Peck]

Well, that gets us partway there. Looking at the docs, it looks like
the output image must still be less than 1MB -- certainly fine for
thumbnailing, but possibly for not all types of tasks.

Also: right now (unless I've missed an API somewhere) to validate
images you must pass them to the Image API with a "no-op" transform
and see if execute_transforms() succeeds. So if I want to validate a
>1MB image, I still have the issue with the output side of the image
API.

It would be great if we could execute_transforms() directly back to a
blob and get a BlobInfo back?

Cheers,
Dave


On Dec 15, 11:18 am, Matthew Blain  wrote:
> While the limit for passing data directly to the Images (or other)
> APIs has not changed, you can pass a Blob key to the Images API to do
> exactly what you want: convert a 50MB uploaded image to a smaller
> image.
>
> More information here:
>  http://code.google.com/appengine/docs/python/images/overview.html#Tra...
>  http://code.google.com/appengine/docs/java/images/overview.html#Trans...
>
> --Matthew
>
> On Dec 15, 10:14 am, trung  wrote:
>
>
>
> > This is awesome.
>
> > But the image API limit is still capped at 1MB!!!
>
> > I still rather be able to resize a 50MB uploaded image down to 1MB or
> > less to cut down the time and bandwidth.
>
> > I assume that increasing the Image API limit is the next logical
> > step. :)
>
> > On Dec 14, 8:00 pm, "Jason (Google)"  wrote:
>
> > > Hi Everyone. We just released version 1.3.0 of the App Engine SDK for
> > > both Python and Java. The most notable change is the new experimental
> > > Blobstore API which allows billed apps to store files up to 50 MB. The
> > > release also includes some performance tweaks to the Java runtime.
>
> > > Blog 
> > > post:http://googleappengine.blogspot.com/2009/12/app-engine-sdk-130-releas...
>
> > > Release notes:
> > > Python:http://code.google.com/p/googleappengine/wiki/SdkReleaseNotes
> > > Java:http://code.google.com/p/googleappengine/wiki/SdkForJavaReleaseNotes
>
> > > Cheers!
> > > - Jason

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Introducing App Engine SDK 1.3.0

[Dave Peck]

> and see if execute_transforms() succeeds. So if I want to validate a 1++MB 
> image,
> I still have the issue with the output side of the image.

(I realize that, when validating, you can always resize the image so
that it's likely to be less than 1MB when finished. I just wish there
were a straightforward "is_valid_image" API, too...)

Cheers,
Dave


On Dec 15, 11:50 am, Dave Peck  wrote:
> Well, that gets us partway there. Looking at the docs, it looks like
> the output image must still be less than 1MB -- certainly fine for
> thumbnailing, but possibly for not all types of tasks.
>
> Also: right now (unless I've missed an API somewhere) to validate
> images you must pass them to the Image API with a "no-op" transform
> and see if execute_transforms() succeeds. So if I want to validate a>1MB 
> image, I still have the issue with the output side of the image
>
> API.
>
> It would be great if we could execute_transforms() directly back to a
> blob and get a BlobInfo back?
>
> Cheers,
> Dave
>
> On Dec 15, 11:18 am, Matthew Blain  wrote:
>
>
>
> > While the limit for passing data directly to the Images (or other)
> > APIs has not changed, you can pass a Blob key to the Images API to do
> > exactly what you want: convert a 50MB uploaded image to a smaller
> > image.
>
> > More information here:
> >  http://code.google.com/appengine/docs/python/images/overview.html#Tra...
> >  http://code.google.com/appengine/docs/java/images/overview.html#Trans...
>
> > --Matthew
>
> > On Dec 15, 10:14 am, trung  wrote:
>
> > > This is awesome.
>
> > > But the image API limit is still capped at 1MB!!!
>
> > > I still rather be able to resize a 50MB uploaded image down to 1MB or
> > > less to cut down the time and bandwidth.
>
> > > I assume that increasing the Image API limit is the next logical
> > > step. :)
>
> > > On Dec 14, 8:00 pm, "Jason (Google)"  wrote:
>
> > > > Hi Everyone. We just released version 1.3.0 of the App Engine SDK for
> > > > both Python and Java. The most notable change is the new experimental
> > > > Blobstore API which allows billed apps to store files up to 50 MB. The
> > > > release also includes some performance tweaks to the Java runtime.
>
> > > > Blog 
> > > > post:http://googleappengine.blogspot.com/2009/12/app-engine-sdk-130-releas...
>
> > > > Release notes:
> > > > Python:http://code.google.com/p/googleappengine/wiki/SdkReleaseNotes
> > > > Java:http://code.google.com/p/googleappengine/wiki/SdkForJavaReleaseNotes
>
> > > > Cheers!
> > > > - Jason

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: "Request was aborted after waiting too long" followed by random DeadlineExceededError on import.

[Ikai L (Google)]

I made the change right before I sent the email. Let me know how it works
for you.

Jason, I also made the change to your application. Please report back after
tomorrow if you continue to experience issues.

On Tue, Dec 15, 2009 at 11:39 AM, Dave Peck  wrote:

> Ikai,
>
> We'll keep an eye on our app for the next ~24 hours and report back.
>
> At what time did you make the changes to our instance? We had
> substantial downtime earlier today, alas.
>
> Can you provide any details about what sort of change was made?
>
> Thanks,
> Dave
>
> On Dec 15, 11:26 am, "Ikai L (Google)"  wrote:
> > Dave,
> >
> > You're correct that this is likely affecting other applications, but it's
> > not a global issue. There are hotspots in the cloud that we notice are
> being
> > especially impacted during certain times of the day. We're actively
> working
> > on addressing these issues, but in the meantime, there are manual steps
> we
> > can try to prevent your applications from becoming resource starved. We
> do
> > these on a one-off basis and reserve them only for applications that seem
> to
> > exhibit the behavior of seeing DeadlineExceeded on simple actions (not
> > initial JVM startup), and at fairly predictable intervals during the day.
> > I've taken these steps to try to remedy your application. Can you let us
> > know if these seem to help? If not, they may indicate that something is
> > going on with your application code, though that does not seem like the
> case
> > here.
> >
> >
> >
> >
> >
> > On Tue, Dec 15, 2009 at 10:54 AM, Dave Peck  wrote:
> > > Hi Ikai,
> >
> > > Any further details on your end? I get the feeling we're not the only
> > > ones, and we've experienced very serious downtime in the last ~48
> > > hours.
> >
> > > This is a critical issue for us to resolve, but at the same time we
> > > lack key pieces of data that would help us solve it on our own...
> >
> > > Thanks,
> > > Dave
> >
> > > On Dec 15, 9:14 am, Jason C  wrote:
> > > > Ikai,
> >
> > > > We see daily DeadlineExceededErrors on app id 'steprep' from 6.30am
> to
> > > > 7.30am (log time).
> >
> > > > Can you look into that as well?
> >
> > > > Thanks,
> > > > j
> >
> > > > On Dec 14, 3:32 pm, "Ikai L (Google)"  wrote:
> >
> > > > > Do you see that it's consistent at the same times? What's your
> > > application
> > > > > ID? I'll look into it.
> >
> > > > > On Mon, Dec 14, 2009 at 11:28 AM, Dave Peck 
> > > wrote:
> > > > > > Hello,
> >
> > > > > > I have an app (citygoround.org) that, especially in the morning,
> > > often
> > > > > > has 10-15 minutes of outright downtime due to server errors.
> >
> > > > > > Looking into it, I see that right before the downtime starts, a
> few
> > > > > > requests log the following warning message:
> >
> > > > > >> Request was aborted after waiting too long to attempt to
> service
> > > > > > your request.
> > > > > >> Most likely, this indicates that you have reached your
> > > > > > simultaneous dynamic request limit.
> >
> > > > > > I'm certainly not over my limit, but I can believe that the
> request
> > > in
> > > > > > question could take a while. (I'll get to the details of that
> request
> > > > > > in a moment.)
> >
> > > > > > Immediately after these warnings, my app has a large amount of
> time
> > > > > > (10+ minutes) where *all requests* -- no matter how unthreatening
> --
> > > > > > raise a DeadlineExceededError. Usually this is raised during the
> > > > > > import of an innocuous module like "re" or "time" or perhaps a
> Django
> > > > > > 1.1 module. (We use use_library.)
> >
> > > > > > My best theory at the moment is that:
> >
> > > > > > 1. It's a cold start, so nothing is cached.
> > > > > > 2. App Engine encounters the high latency request and bails.
> > > > > > 3. We probably inadvertently catch the DeadlineExceededError, so
> the
> > > > > > runtime doesn't clean up properly.
> > > > > > 4. Future requests are left in a busted state.
> >
> > > > > > Does this sound at all reasonable? I see a few related issues
> (2396,
> > > > > > 2266, and 1409) but no firm/completely clear discussion of what's
> > > > > > happening in any of them.
> >
> > > > > > Thanks,
> > > > > > Dave
> >
> > > > > > PS:
> >
> > > > > > The specifics about our high latency request are *not* strictly
> > > > > > relevant to the larger problem I'm having, but I will include
> them
> > > > > > because I have a second "side" question to ask about it.
> >
> > > > > > The "high latency" request is serving an image. Our app lets
> users
> > > > > > upload images and we store them in the data store. When serving
> an
> > > > > > image, our handler:
> >
> > > > > > 1. Checks to see if the bytes for the image are in memcache, and
> if
> > > so
> > > > > > returns them immediately.
> > > > > > 2. Otherwise grabs the image from the datastore, and if it is
> smaller
> > > > > > than 64K, adds the bytes to the memcache
> > > > > > 3. Returns the result
> >
> > > > > > I'm wondering if using memcache in this

Re: [google-appengine] Billing Setup Started for about 2 weeks

[Ikai L (Google)]

What specifically is wrong with the application? It can be accessed at
kaibo-www.appspot.com and the admin console seems to be working.

On Tue, Dec 15, 2009 at 2:13 AM, Peter Ondruska wrote:

> I have setup billing for my application at Nov 30 and since then it is
> still not activated and no other change in status since then. Can
> please anybody at Google have a look, appid kaibo-www? Thank you
>
> --
>
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
>


-- 
Ikai Lan
Developer Programs Engineer, Google App Engine

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Introducing App Engine SDK 1.3.0

[yadoo]

Excellent job Google... Hopefully, google provides us Backup/Restore
tools in the future release 

On Dec 15, 11:50 pm, Dave Peck  wrote:
> Well, that gets us partway there. Looking at the docs, it looks like
> the output image must still be less than 1MB -- certainly fine for
> thumbnailing, but possibly for not all types of tasks.
>
> Also: right now (unless I've missed an API somewhere) to validate
> images you must pass them to the Image API with a "no-op" transform
> and see if execute_transforms() succeeds. So if I want to validate a>1MB 
> image, I still have the issue with the output side of the image
>
> API.
>
> It would be great if we could execute_transforms() directly back to a
> blob and get a BlobInfo back?
>
> Cheers,
> Dave
>
> On Dec 15, 11:18 am, Matthew Blain  wrote:
>
>
>
> > While the limit for passing data directly to the Images (or other)
> > APIs has not changed, you can pass a Blob key to the Images API to do
> > exactly what you want: convert a 50MB uploaded image to a smaller
> > image.
>
> > More information here:
> >  http://code.google.com/appengine/docs/python/images/overview.html#Tra...
> >  http://code.google.com/appengine/docs/java/images/overview.html#Trans...
>
> > --Matthew
>
> > On Dec 15, 10:14 am, trung  wrote:
>
> > > This is awesome.
>
> > > But the image API limit is still capped at 1MB!!!
>
> > > I still rather be able to resize a 50MB uploaded image down to 1MB or
> > > less to cut down the time and bandwidth.
>
> > > I assume that increasing the Image API limit is the next logical
> > > step. :)
>
> > > On Dec 14, 8:00 pm, "Jason (Google)"  wrote:
>
> > > > Hi Everyone. We just released version 1.3.0 of the App Engine SDK for
> > > > both Python and Java. The most notable change is the new experimental
> > > > Blobstore API which allows billed apps to store files up to 50 MB. The
> > > > release also includes some performance tweaks to the Java runtime.
>
> > > > Blog 
> > > > post:http://googleappengine.blogspot.com/2009/12/app-engine-sdk-130-releas...
>
> > > > Release notes:
> > > > Python:http://code.google.com/p/googleappengine/wiki/SdkReleaseNotes
> > > > Java:http://code.google.com/p/googleappengine/wiki/SdkForJavaReleaseNotes
>
> > > > Cheers!
> > > > - Jason

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Introducing App Engine SDK 1.3.0

[mably]

For Backup/Restore tools, please vote for issue
http://code.google.com/p/googleappengine/issues/detail?id=776

Already 174 people have starred it.

François

On 15 déc, 22:21, yadoo  wrote:
> Excellent job Google... Hopefully, google provides us Backup/Restore
> tools in the future release 
>
> On Dec 15, 11:50 pm, Dave Peck  wrote:
>
>
>
> > Well, that gets us partway there. Looking at the docs, it looks like
> > the output image must still be less than 1MB -- certainly fine for
> > thumbnailing, but possibly for not all types of tasks.
>
> > Also: right now (unless I've missed an API somewhere) to validate
> > images you must pass them to the Image API with a "no-op" transform
> > and see if execute_transforms() succeeds. So if I want to validate a>1MB 
> > image, I still have the issue with the output side of the image
>
> > API.
>
> > It would be great if we could execute_transforms() directly back to a
> > blob and get a BlobInfo back?
>
> > Cheers,
> > Dave
>
> > On Dec 15, 11:18 am, Matthew Blain  wrote:
>
> > > While the limit for passing data directly to the Images (or other)
> > > APIs has not changed, you can pass a Blob key to the Images API to do
> > > exactly what you want: convert a 50MB uploaded image to a smaller
> > > image.
>
> > > More information here:
> > >  http://code.google.com/appengine/docs/python/images/overview.html#Tra...
> > >  http://code.google.com/appengine/docs/java/images/overview.html#Trans...
>
> > > --Matthew
>
> > > On Dec 15, 10:14 am, trung  wrote:
>
> > > > This is awesome.
>
> > > > But the image API limit is still capped at 1MB!!!
>
> > > > I still rather be able to resize a 50MB uploaded image down to 1MB or
> > > > less to cut down the time and bandwidth.
>
> > > > I assume that increasing the Image API limit is the next logical
> > > > step. :)
>
> > > > On Dec 14, 8:00 pm, "Jason (Google)"  wrote:
>
> > > > > Hi Everyone. We just released version 1.3.0 of the App Engine SDK for
> > > > > both Python and Java. The most notable change is the new experimental
> > > > > Blobstore API which allows billed apps to store files up to 50 MB. The
> > > > > release also includes some performance tweaks to the Java runtime.
>
> > > > > Blog 
> > > > > post:http://googleappengine.blogspot.com/2009/12/app-engine-sdk-130-releas...
>
> > > > > Release notes:
> > > > > Python:http://code.google.com/p/googleappengine/wiki/SdkReleaseNotes
> > > > > Java:http://code.google.com/p/googleappengine/wiki/SdkForJavaReleaseNotes
>
> > > > > Cheers!
> > > > > - Jason

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] View this page "Google App Engine Open Source Projects"

[Jason (Google)]

Adding Lollysite and JobTracker

Click on 
http://groups.google.com/group/google-appengine/web/google-app-engine-open-source-projects
- or copy & paste it into your browser's address bar if that doesn't
work.

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Java or Python; Which should I recommend to others who are very sensitive to costs?

[Michael Langford]

Jython works just fine on App Engine. There is even a list of popular
API's that do.

I don't know about speed differentials there.

http://groups.google.com/group/google-appengine-java/web/will-it-play-in-app-engine?pli=1

--Michael

On Tue, Dec 15, 2009 at 2:01 PM, Jesaja Everling  wrote:
> Maybe the people asking if it's possible to run Jython on App Engine
> were on to something, after all! ;)
> I also think choosing the toolkit that works best for the task at hand
> is more important. Furthermore I'm confident the App Engine team will
> speed up the Python runtime further; maybe that's part of what Unladen
> Swallow is meant for.
> A very interesting observation, though!
>
> Best Regards,
>
> Jesaja Everling
>
>
> On Tue, Dec 15, 2009 at 7:34 PM, Michael Langford
>  wrote:
>>>If your app exceeds free quota, this deference can impact total amount of 
>>>costs significantly.
>>
>> As can the difference in price/productivity between the two languages
>> as far as total cost of development goes.
>>
>> The incremental costs on appengine between the two technologies are
>> dwarfed by the productivity differences between them for most
>> project's I'd imagine. I'd say if you're charging clients the same
>> amount to develop an app in Java as you are in Python, you're vastly
>> overcharging for the python or undercharging for the java (baring
>> language specific libraries available for one platform and not on the
>> other).
>>
>> For a huge number of programs, the difference in development time
>> between a higher level language and a lower level language is never
>> recouped by increased runtime in the faster, lower level language.
>>
>> For many apps, even java developers are faster writing python than
>> java (Although they all seem to go for Ruby instead).
>>
>>
>>     --Michael
>>
>>
>> On Tue, Dec 15, 2009 at 1:02 PM, Takashi Matsuo
>>  wrote:
>>> Hello,
>>>
>>> Today I noticed that App Engine Java environment became much faster
>>> then before. The spin up cost is about 700cpu_ms with the simplest
>>> servlet. Additionally, when it comes to serving with a hot instance,
>>> the cost reduces to 0-2cpu_ms, while python environment takes about
>>> 5-7cpu_ms even with the simplest handler.
>>>
>>> To make it simple here, lets say Java takes     1cpu_ms while Python takes
>>> 6cpu_ms for serving very simple page.
>>> How many requests can they serve with 1 cpu hour?
>>>
>>> Java: 360 requests/1 cpu hour
>>> Python: 60 requests/1 cpu hour
>>>
>>> This is a big deference; 6 times! If your app exceeds free quota, this
>>> deference can impact total amount of costs significantly. I'm a big
>>> Python fan and I have believed that appengine Python runtime is
>>> superior to Java runtime, so I've been trying to persuade others to
>>> use Python rather than Java for now.
>>>
>>> Having said that, today it turns out for me that Java runtime is much
>>> more cost effective than Python runtime in some cases, so should I
>>> recommend others to use Apppengine Java if they are very sensitive to
>>> cpu costs?
>>>
>>> I'd appreciate if anyone could share one's thoughts/experiences on this.
>>>
>>> TIA
>>>
>>> --
>>> Takashi Matsuo
>>> Kay's daddy
>>>
>>> --
>>>
>>> You received this message because you are subscribed to the Google Groups 
>>> "Google App Engine" group.
>>> To post to this group, send email to google-appeng...@googlegroups.com.
>>> To unsubscribe from this group, send email to 
>>> google-appengine+unsubscr...@googlegroups.com.
>>> For more options, visit this group at 
>>> http://groups.google.com/group/google-appengine?hl=en.
>>>
>>>
>>>
>>
>>
>>
>> --
>> Michael Langford
>> Phone: 404-386-0495
>> Consulting: http://www.RowdyLabs.com
>>
>> --
>>
>> You received this message because you are subscribed to the Google Groups 
>> "Google App Engine" group.
>> To post to this group, send email to google-appeng...@googlegroups.com.
>> To unsubscribe from this group, send email to 
>> google-appengine+unsubscr...@googlegroups.com.
>> For more options, visit this group at 
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>>
>>
>
> --
>
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to 
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/google-appengine?hl=en.
>
>
>



-- 
Michael Langford
Phone: 404-386-0495
Consulting: http://www.RowdyLabs.com

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Blob Store Post/Redirect/Get & Django Forms

[Dave Peck]

Hi,

This morning I started to modify the code to CityGoRound to use the
blobstore for user-uploaded screenshots.

We use Django forms in our app. One of our forms (http://
citygoround.org/apps/add/) allows users to upload a new "transit app"
to our app gallery. They must include one screenshot; they can include
up to five.

Blob store handlers must issue a 30x-series redirect once they're done
with their work. Understandable.

Unfortunately, PGR makes handling blobs in the context of Django forms
fairly tricky. Especially for a complex form like ours, we want to
provide good feedback if the user does something wrong elsewhere in
the form. It appears that, in order to do this, we must now redirect
to a GET URL with form contents part of the URL string itself.

If you're familiar with Django, you'll see what this doesn't fit into
the typical form pattern. Does anyone have suggestions about how this
can be cleanly handled?

PS: I notice that if a user fails to attach a file, a zero-length blob
with mimetype 'text/plain' is created anyway. Is this really
desirable? I'm just going to turn around and delete that blob...

Thanks,
Dave

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: AppWrench 1.0 released: App Engine Profiler, Data Store Viewer and more

[Erem]

Hey, has anyone tried this out?

>From the screenshots and vidcaps It actually seems really cool.
Unfortunately I'm working on some more businessy aspects of my startup
right now and haven't had a chance to use it.

AppWrench guys: have you considered hiring a designer for your
webpage? It looks...terrible.

Sorry, but hopefully I can give some more constructive feedback come
January =)

Erem
On Dec 14, 5:38 am, kor  wrote:
> OnPositive Technologies is happy to announce the release of AppWrench
> 1.0, the first collection of powerful productivity tools created
> specifically for Google App Engine.
>
> AppWrench includes the following tools:
>
> * App Engine Java Profiler
>
> A unique tool which allows profiling Java applications deployed to
> local or production Google App Engine servers.
>
> AppEngine Profiler is capable of collecting information about system
> time spent executing application methods, per-method data store usage
> statistics and per-method mem cache hit/miss statistics.
>
> * Java Console
>
> A productivity tool which allows to execute fragments of Java code in
> the context of a running App Engine application deployed to Google App
> Engine servers or the local development server. Best for one-off tasks
> and quick experiments.
>
> * Data Store Viewer
>
> Allows viewing and modifying entities in the local and production data
> stores. The only tool that allows 100% editing of any entities,
> including key, list and blob properties. Has powerful detail
> formatting and filtering functions.
>
> * Log Viewer
>
> Allows to view Google App Engine logs and jump to error locations
> directly from your Eclipse environment.
>
> The suite integrates into Eclipse 3.4+ and runs with the local App
> Engine development server for free. 7-day trial licenses are available
> to test the suite with the production servers. Requires App Engine SDK
> 1.2.6+.
>
> Learn more about AppWrench onhttp://appwrench.onpositive.com
>
> We are highly devoted to develop this product in a way the community
> wants, so please share your experience, ideas and suggestions 
> onhttp://appwrench.zendesk.com
>
> Sincerely yours,
> Pavel Petrochenko,
> AppWrench development team.

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] bulkloader/index bug: random number of results returned

[bsb]

I'm downloading data from the bulkloader using the command:

appcfg.py download_data --config_file=ExporterFormat.py --
filename=prices.csv --kind Model_price .

The command finishes without errors, but I only get 39820 entities. A
quick look into the statistics on the website tells me there are at
least 114000 entities (they never get deleted, so it's rather more
than less).

I mentioned a similar problem over at this thread:

http://groups.google.com/group/google-appengine/browse_thread/thread/62ae96ef9dea3475

It seems that the remote api sometimes uses the wrong index, or the
index itself is broken. Either way, what happens is that the entity
keys are occasionally returned in descending rather than ascending
order, causing the loading to skip a whole range of entities and
return others in duplicate.

Please can someone look into this. I'm really struggling to get a full
datastore backup and cannot achieve it without a way to cycle through
all entities, which, given this issue, is not possible at the moment.

Thanks in advance,
Benjamin

PS: Sorry Nick for CCing you directly, but I'm afraid some issues
might get lost in the daily noise, and this one has been simmering for
weeks now, so I wanted to give it a bump

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Blob Store Post/Redirect/Get & Django Forms

[Dave Peck]

I suppose the clean way to do this is to add stuff to the user's
session in the blob handler, and then pick it up in the GET request
that we redirect to.

Not too hard, though from the perspective of the Django forms API not
a natural fit.

Cheers,
Dave

On Dec 15, 2:17 pm, Dave Peck  wrote:
> Hi,
>
> This morning I started to modify the code to CityGoRound to use the
> blobstore for user-uploaded screenshots.
>
> We use Django forms in our app. One of our forms (http://
> citygoround.org/apps/add/) allows users to upload a new "transit app"
> to our app gallery. They must include one screenshot; they can include
> up to five.
>
> Blob store handlers must issue a 30x-series redirect once they're done
> with their work. Understandable.
>
> Unfortunately, PGR makes handling blobs in the context of Django forms
> fairly tricky. Especially for a complex form like ours, we want to
> provide good feedback if the user does something wrong elsewhere in
> the form. It appears that, in order to do this, we must now redirect
> to a GET URL with form contents part of the URL string itself.
>
> If you're familiar with Django, you'll see what this doesn't fit into
> the typical form pattern. Does anyone have suggestions about how this
> can be cleanly handled?
>
> PS: I notice that if a user fails to attach a file, a zero-length blob
> with mimetype 'text/plain' is created anyway. Is this really
> desirable? I'm just going to turn around and delete that blob...
>
> Thanks,
> Dave

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Introducing App Engine SDK 1.3.0

[Jason (Google)]

Stephen, from the docs:

"If you don't specify a content type, the Blobstore will try to infer
content type from the file extension. If no content type can be determined,
the newly created blob is assigned content type application/octet-stream."

- Jason

On Tue, Dec 15, 2009 at 6:05 AM, Stephen Mayer wrote:

> Hi Jason,
>
> Looks like an exciting new feature!  My concern after looking thru the
> docs is how we tell what file the user actually uploaded ... was it an
> image, a video, etc ... for validation and sanity checks?  It seems
> like you could use the fetch api to get the actual file and work on it
> in some way, but that seems a bit ackward.  I do think its a great
> feature tho for admin uploaded content that is trusted. Or am I
> missing something here?
>
> Stephen
>
> On Dec 14, 11:00 pm, "Jason (Google)"  wrote:
> > Hi Everyone. We just released version 1.3.0 of the App Engine SDK for
> > both Python and Java. The most notable change is the new experimental
> > Blobstore API which allows billed apps to store files up to 50 MB. The
> > release also includes some performance tweaks to the Java runtime.
> >
> > Blog post:
> http://googleappengine.blogspot.com/2009/12/app-engine-sdk-130-releas...
> >
> > Release notes:
> > Python:http://code.google.com/p/googleappengine/wiki/SdkReleaseNotes
> > Java:
> http://code.google.com/p/googleappengine/wiki/SdkForJavaReleaseNotes
> >
> > Cheers!
> > - Jason
>
> --
>
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
>

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Keeping an updated dictionary list of over 1000 in App Engine

[killer barney]

I have a question that perhaps someone can give me some insight on.

I have a list of thousands of keywords in my website that I am going
to constantly need to query and iterate through.  So I thought rather
than querying for every keyword in the datastore everytime, I should
store it in the memcache.  But even this doesn't seem like a very good
solution as the memcache gets eliminated and I'm going to have to
somehow query the whole list and store it back into memcache everytime
it gets disposed of.

I thought about having the list stored as sharded arrays into
memcache, so when it does get disposed of, then I only have to update
the keywords that start with "N", for example, but this still doesn't
seem like the best solution.

Is there a better way to do this?

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Java or Python; Which should I recommend to others who are very sensitive to costs?

[Andy Freeman]

> Having said that, today it turns out for me that Java runtime is much
> more cost effective than Python runtime in some cases

The question is not whether the Java or Python runtime is more cost
effective in some cases, it's whether it which is more cost effective
in your cases.

Suppose that your application does one datastore operation for each
page and that datastore operation and other code takes the same amount
of time in both Python and Java.  Datastore operations are so much
slower than startup that this alone would make the startup difference
almost unnoticeable.

And, as someone else pointed out, development time is a cost too.

On Dec 15, 10:02 am, Takashi Matsuo  wrote:
> Hello,
>
> Today I noticed that App Engine Java environment became much faster
> then before. The spin up cost is about 700cpu_ms with the simplest
> servlet. Additionally, when it comes to serving with a hot instance,
> the cost reduces to 0-2cpu_ms, while python environment takes about
> 5-7cpu_ms even with the simplest handler.
>
> To make it simple here, lets say Java takes     1cpu_ms while Python takes
> 6cpu_ms for serving very simple page.
> How many requests can they serve with 1 cpu hour?
>
> Java: 360 requests/1 cpu hour
> Python: 60 requests/1 cpu hour
>
> This is a big deference; 6 times! If your app exceeds free quota, this
> deference can impact total amount of costs significantly. I'm a big
> Python fan and I have believed that appengine Python runtime is
> superior to Java runtime, so I've been trying to persuade others to
> use Python rather than Java for now.
>
> Having said that, today it turns out for me that Java runtime is much
> more cost effective than Python runtime in some cases, so should I
> recommend others to use Apppengine Java if they are very sensitive to
> cpu costs?
>
> I'd appreciate if anyone could share one's thoughts/experiences on this.
>
> TIA
>
> --
> Takashi Matsuo
> Kay's daddy

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Application ID

[Marco Antonio da Silva Castanheira]

Hi,

I'm new on App Engine. I developed my first application, called *artezel*,
but when I tried create the application on App Engine, an error has ocurred.
The application was not created, but the id, artezel, is now unavailable.
How I can turn this id available again?

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Bug with blobstore internal redirect in dev_appserver.py

[Dave Peck]

On the local server, when the blobstore code performs the internal
redirect to whatever URL you specified in create_upload_url(), the
POST contents are not properly encoded.

According to the RFCs, you must end lines with CRLF, but dev_appserver
(and, perhaps, the production environment?) ends lines only with LF.

This causes Django 1.1's multipart parser to fail (in
parse_boundary_stream), since it is hardcoded to look for \r\n\r\n at
the end of each part's header.

As a result, I'm blocked on django+blobstore integration work...

I've logged this as issue 2515.

Thanks,
Dave


--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Application ID

[Nickolas Daskalou]

Are you using a Google Apps account?

If so, visit the home page of this group:
http://groups.google.com/group/google-appengine

Read the bold paragraph that starts with "If you've created an
application".

Nick


On Dec 16, 11:20 am, Marco Antonio da Silva Castanheira
 wrote:
> Hi,
>
> I'm new on App Engine. I developed my first application, called *artezel*,
> but when I tried create the application on App Engine, an error has ocurred.
> The application was not created, but the id, artezel, is now unavailable.
> How I can turn this id available again?

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] How to determine top 10 scoring users from last 24 hours?

[Amir Michail]

Hello,

I asked this question before but perhaps this is more feasible now
with the latest version of the GAE?

http://groups.google.com/group/google-appengine/browse_thread/thread/e7f892a2efabed59/2ced4b6790ce1f1b

Amir

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.