[google-appengine] Re: Help resolve massive performance regression in 2.7 vs 2.5 runtime

[Pol]

Hi Brian,

  So on December 1st, the 50% discount for front-instances is gone. The
  idea is to compensate by switching to Python 2.7 with multithreading,
  but it looks like at this point it's a lose-lose situation: it runs
  more requests at the same time, but they take longer. We're mid-
  November already, do you guys think you'll have all of this working
  perfectly within 2 weeks?

 No, the issues with concurrent requests won't be fixed by the end of November.

 But note that concurrent requests will *not* improve the utilization
 of CPU-bound requests. Running multiple threads on the same CPU just
 proportionally slows each thread down.

That doesn't make sense: apps do a mix of CPU stuff and RPC stuff (and
possibly URL requests). What's the points of concurrent requests if it
slows down the CPU stuff while allowing to parallelize your RPC calls?
The end result will be the same number of instance as requests end up
taking longer. Isn't the scheduler supposed to watch all this and
ensure the CPU on each physical machine is not saturated?

Only apps that do long poll URL requests and barely use the CPU would
benefit of concurrent requests then.

We were told: don't worry so much about hours-based pricing, just wait
for 2.7 runtime, it'll have concurrent requests, it'll compensate
things. Clearly that doesn't work as promised if just turning
threadsafe ON makes a 2 seconds requests turn into a 30-60 seconds
one: the scheduler is not doing the right thing.

It seems what you need is a per WSGIApplication instance control of
the concurrent setting instead of global one, so you can turn it on
only where it makes sense.

Finally, no matter what, concurrent or not, there's still a problem as
2.7 runtime appears slower than 2.5 in this simple empirical test. I'm
starting to suspect you are using the 2.7 transition as a opportunity
to run more virtual instances per physical machine.

- Pierre

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Help resolve massive performance regression in 2.7 vs 2.5 runtime

[Pol]

Hi Brian,

It helps, it's now at ~5s instead, but still at least 2x slower than
on the 2.5 runtime.

So on December 1st, the 50% discount for front-instances is gone. The
idea is to compensate by switching to Python 2.7 with multithreading,
but it looks like at this point it's a lose-lose situation: it runs
more requests at the same time, but they take longer. We're mid-
November already, do you guys think you'll have all of this working
perfectly within 2 weeks? The 1.6 SDK which actually allows to run 2.7
locally was just released, so we're only starting to test now. Seems
to me deferring the payment change 1 more month to January 1st 2012
wouldn't hurt :)

On Nov 12, 10:09 pm, Brian Quinlan bquin...@google.com wrote:
 Hi Pol,

 Thanks for getting back to me.

 On Sun, Nov 13, 2011 at 4:50 PM, Pol p...@everpix.net wrote:
  Hi Brian,

  threadsafe is true

 There is a known issue where concurrent requests (enabled with
 threadsafe) can be much slower than non-concurrent requests,
 especially if the request is CPU-bound.

 You might want to set threadsafe to false and see if that fixes the problem.

 Cheers,
 Brian







  Here's an example password_hash for an original 8 characters password:

  $2a$04$cbM2uHwDphIG3jFFRpq1mui5aVjevnDUwhvQ77S/WG/qvJMpiXAL6

  On Nov 12, 6:58 pm, Brian Quinlan bquin...@google.com wrote:
  Hi Pol,

  On Sun, Nov 13, 2011 at 1:48 PM, Pol p...@everpix.net wrote:
   Hi,

   Since switching to 2.7 runtime, logging in tohttp://www.everpix.com
   went from about a second to anywhere from 15s to 60s. I tracked it
   down to this single password checking line:

   from bcrypt import bcrypt
   bcrypt.hashpw(password, self.password_hash) == self.password_hash

  What value are you using for threadsafe in your app.yaml?

  How large is self.password_hash?

  Cheers,
  Brian

   This comes from a native Python implementation of the py-bcrypt
   package fromhttp://www.mindrot.org/projects/py-bcrypt/; grabbed from
   here:https://github.com/erlichmen/py-bcrypt.

   So what's happening here and how can we fix this?

   Thanks,

   - Pol

   --
   You received this message because you are subscribed to the Google 
   Groups Google App Engine group.
   To post to this group, send email to google-appengine@googlegroups.com.
   To unsubscribe from this group, send email to 
   google-appengine+unsubscr...@googlegroups.com.
   For more options, visit this group 
   athttp://groups.google.com/group/google-appengine?hl=en.

  --
  You received this message because you are subscribed to the Google Groups 
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to 
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group 
  athttp://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Help resolve massive performance regression in 2.7 vs 2.5 runtime

[Pol]

Hi,

Since switching to 2.7 runtime, logging in to http://www.everpix.com
went from about a second to anywhere from 15s to 60s. I tracked it
down to this single password checking line:

from bcrypt import bcrypt
bcrypt.hashpw(password, self.password_hash) == self.password_hash

This comes from a native Python implementation of the py-bcrypt
package from http://www.mindrot.org/projects/py-bcrypt/; grabbed from
here: https://github.com/erlichmen/py-bcrypt.

So what's happening here and how can we fix this?

Thanks,

- Pol

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Help resolve massive performance regression in 2.7 vs 2.5 runtime

[Pol]

Hi Brian,

threadsafe is true

Here's an example password_hash for an original 8 characters password:

$2a$04$cbM2uHwDphIG3jFFRpq1mui5aVjevnDUwhvQ77S/WG/qvJMpiXAL6

On Nov 12, 6:58 pm, Brian Quinlan bquin...@google.com wrote:
 Hi Pol,

 On Sun, Nov 13, 2011 at 1:48 PM, Pol p...@everpix.net wrote:
  Hi,

  Since switching to 2.7 runtime, logging in tohttp://www.everpix.com
  went from about a second to anywhere from 15s to 60s. I tracked it
  down to this single password checking line:

  from bcrypt import bcrypt
  bcrypt.hashpw(password, self.password_hash) == self.password_hash

 What value are you using for threadsafe in your app.yaml?

 How large is self.password_hash?

 Cheers,
 Brian







  This comes from a native Python implementation of the py-bcrypt
  package fromhttp://www.mindrot.org/projects/py-bcrypt/; grabbed from
  here:https://github.com/erlichmen/py-bcrypt.

  So what's happening here and how can we fix this?

  Thanks,

  - Pol

  --
  You received this message because you are subscribed to the Google Groups 
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to 
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group 
  athttp://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Python 2.7 multithreaded: ereporter mapreduce compatibility

[Pol]

Hi Brian,

Although we can deploy to production using 2.7, accessing /mapreduce
fails:

Traceback (most recent call last):
  File /base/python27_runtime/python27_lib/versions/1/google/
appengine/runtime/wsgi.py, line 168, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
  File /base/python27_runtime/python27_lib/versions/1/google/
appengine/runtime/wsgi.py, line 217, in _LoadHandler
__import__(cumulative_path)
ImportError: No module named application

The app.yaml is set like this:

- url: /mapreduce(/.*)?
  script: google.appengine.ext.mapreduce.application
  login: admin

Thanks for your help!

On Nov 11, 1:26 pm, Brian Quinlan bquin...@google.com wrote:
 Hi Pol,

 On Sat, Nov 12, 2011 at 7:46 AM, Pol p...@everpix.net wrote:
  Indeed the problem is the parsing of app.yaml:
  - url: /mapreduce(/.*)?  script: $PYTHON_LIB/google/appengine/ext/
 mapreduce/main.py  login: admin- url: /ereporter.*  script:
  $PYTHON_LIB/google/appengine/ext/ereporter/report_generator.py  login:
  admin
  So what do I use instead for script paths?

 script: google.appengine.ext.mapreduce.application
 script: google.appengine.ext.ereporter.report_generator.main.APP

  Looking into the SDK
  source, these extensions are not2.7multithreaded ready.

 How so?

 Cheers,
 Brian







  On Nov 10, 6:10 pm, Brian Quinlan bquin...@google.com wrote:
  On Fri, Nov 11, 2011 at 2:44 AM, Pol p...@everpix.net wrote:
   Hi,

   What are the plans to get these 2 extensions compatible with Python
  2.7multithreaded?

   $PYTHON_LIB/google/appengine/ext/mapreduce/main.py
   $PYTHON_LIB/google/appengine/ext/ereporter/report_generator.py

  I assume that you are seeing an error during the parsing of your appcfg.py 
  file?

  The error message should say that you can't use a CGI with threadsafe.
  The solution is to use WSGI in your handlers instead i.e.

  google.appengine.ext.mapreduce.application
  google.appengine.ext.ereporter/report_generator.application

  Cheers,
  Brian

   Thanks,

   - Pol

   --
   You received this message because you are subscribed to the Google 
   Groups Google App Engine group.
   To post to this group, send email to google-appengine@googlegroups.com.
   To unsubscribe from this group, send email to 
   google-appengine+unsubscr...@googlegroups.com.
   For more options, visit this group 
   athttp://groups.google.com/group/google-appengine?hl=en.

  --
  You received this message because you are subscribed to the Google Groups 
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to 
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group 
  athttp://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Python 2.7 multithreaded: ereporter mapreduce compatibility

[Pol]

Indeed the problem is the parsing of app.yaml:
- url: /mapreduce(/.*)?  script: $PYTHON_LIB/google/appengine/ext/
mapreduce/main.py  login: admin- url: /ereporter.*  script:
$PYTHON_LIB/google/appengine/ext/ereporter/report_generator.py  login:
admin
So what do I use instead for script paths? Looking into the SDK
source, these extensions are not 2.7 multithreaded ready.

On Nov 10, 6:10 pm, Brian Quinlan bquin...@google.com wrote:
 On Fri, Nov 11, 2011 at 2:44 AM, Pol p...@everpix.net wrote:
  Hi,

  What are the plans to get these 2 extensions compatible with Python
  2.7 multithreaded?

  $PYTHON_LIB/google/appengine/ext/mapreduce/main.py
  $PYTHON_LIB/google/appengine/ext/ereporter/report_generator.py

 I assume that you are seeing an error during the parsing of your appcfg.py 
 file?

 The error message should say that you can't use a CGI with threadsafe.
 The solution is to use WSGI in your handlers instead i.e.

 google.appengine.ext.mapreduce.application
 google.appengine.ext.ereporter/report_generator.application

 Cheers,
 Brian







  Thanks,

  - Pol

  --
  You received this message because you are subscribed to the Google Groups 
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to 
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group 
  athttp://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Python 2.7 multithreaded: ereporter mapreduce compatibility

[Pol]

Ah OK it works, thanks. Didn't know had to replace path entirely...
Was wondering about compatibility since there were still some CGI
leftovers in these files.

On Nov 11, 1:26 pm, Brian Quinlan bquin...@google.com wrote:
 Hi Pol,

 On Sat, Nov 12, 2011 at 7:46 AM, Pol p...@everpix.net wrote:
  Indeed the problem is the parsing of app.yaml:
  - url: /mapreduce(/.*)?  script: $PYTHON_LIB/google/appengine/ext/
  mapreduce/main.py  login: admin- url: /ereporter.*  script:
  $PYTHON_LIB/google/appengine/ext/ereporter/report_generator.py  login:
  admin
  So what do I use instead for script paths?

 script: google.appengine.ext.mapreduce.application
 script: google.appengine.ext.ereporter.report_generator.main.APP

  Looking into the SDK
  source, these extensions are not 2.7 multithreaded ready.

 How so?

 Cheers,
 Brian







  On Nov 10, 6:10 pm, Brian Quinlan bquin...@google.com wrote:
  On Fri, Nov 11, 2011 at 2:44 AM, Pol p...@everpix.net wrote:
   Hi,

   What are the plans to get these 2 extensions compatible with Python
   2.7 multithreaded?

   $PYTHON_LIB/google/appengine/ext/mapreduce/main.py
   $PYTHON_LIB/google/appengine/ext/ereporter/report_generator.py

  I assume that you are seeing an error during the parsing of your appcfg.py 
  file?

  The error message should say that you can't use a CGI with threadsafe.
  The solution is to use WSGI in your handlers instead i.e.

  google.appengine.ext.mapreduce.application
  google.appengine.ext.ereporter/report_generator.application

  Cheers,
  Brian

   Thanks,

   - Pol

   --
   You received this message because you are subscribed to the Google 
   Groups Google App Engine group.
   To post to this group, send email to google-appengine@googlegroups.com.
   To unsubscribe from this group, send email to 
   google-appengine+unsubscr...@googlegroups.com.
   For more options, visit this group 
   athttp://groups.google.com/group/google-appengine?hl=en.

  --
  You received this message because you are subscribed to the Google Groups 
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to 
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group 
  athttp://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Python 2.7 multithreaded: ereporter mapreduce compatibility

[Pol]

Hi,

What are the plans to get these 2 extensions compatible with Python
2.7 multithreaded?

$PYTHON_LIB/google/appengine/ext/mapreduce/main.py
$PYTHON_LIB/google/appengine/ext/ereporter/report_generator.py

Thanks,

- Pol

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] How to set up task queues to prevent bursts?

[Pol]

Hi,

Through a task queue, we manage calls from GAE to an external system,
which is fully-scalable but takes minutes to do so. The task queue
rate is therefore set to 200/s.

So what happens is that sometimes we get bursts of activity, and the
task queue sends 200 requests at once: the vast majority fails in the
external system as it can't handle this sudden request in load. But
the external system doesn't start scaling up as there are no more
requests coming and it's scaling based on the CPU load. Then suddenly
you get another burst of requests from the task queue retrying, and
then again the majority fails and so on.

So how can we configure a task queue to have a very high rate *but*
prevent it to get to this rate too fast? I noticed the bucket size
parameter, but I'm not sure how to use it properly.

Thanks,

- Pol

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: How to set up task queues to prevent bursts?

[Pol]

Sorry, I realize I didn't explain everything: since the external
system can handle an arbitrary load (as long as this load doesn't grow
too fast) and requests get processed in 1 to 2 seconds, we already set
the processing rate to the maximum (200), as well as the concurrent
(200).

The problem is not to limit the number of requests being handled, but
preventing bursts e.g. scaling up smoothly.

On Oct 30, 9:22 pm, Nicholas Verne nve...@google.com wrote:
 If your external system is called synchronously by your tasks, you could
 try setting the queue's max_concurrent_requests parameter in queue.yaml/xml

 This is documented for Java, but the same general rules apply for Python or
 Go 
 applications.http://code.google.com/appengine/docs/java/config/queue.html#Defining...

 Nick Verne







 On Mon, Oct 31, 2011 at 3:13 PM, Pol p...@everpix.net wrote:
  Hi,

  Through a task queue, we manage calls from GAE to an external system,
  which is fully-scalable but takes minutes to do so. The task queue
  rate is therefore set to 200/s.

  So what happens is that sometimes we get bursts of activity, and the
  task queue sends 200 requests at once: the vast majority fails in the
  external system as it can't handle this sudden request in load. But
  the external system doesn't start scaling up as there are no more
  requests coming and it's scaling based on the CPU load. Then suddenly
  you get another burst of requests from the task queue retrying, and
  then again the majority fails and so on.

  So how can we configure a task queue to have a very high rate *but*
  prevent it to get to this rate too fast? I noticed the bucket size
  parameter, but I'm not sure how to use it properly.

  Thanks,

  - Pol

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: How to set up task queues to prevent bursts?

[Pol]

Good point, forcing the tasks to retry every few seconds instead of at
an increasing back off might do it. Keep pounding the system to force
it to scale, but it's not very elegant :)

My original idea was to play with the bucket size parameter for the
queue, as the docs seem to imply that it allows to control bursts, but
it's not clear at all. I reduced it to 1, but I don't understand what
it means in practice if max rate and max concurrency are at 200.

On Oct 30, 10:13 pm, Jeff Schnitzer j...@infohazard.org wrote:
 Interesting problem.  While the sudden load may be undesirable it seems
 that the real problem is that the task queue backoff is too aggressive - if
 it kept trying, it would eventually spin up enough hardware at the external
 system.

 You can configure the retry schedule explicitly - maybe try setting it so
 that requests are retried more often?  You'll still get the initial errors
 but at least you'll get the queues cleared eventually.

 Jeff







 On Sun, Oct 30, 2011 at 10:07 PM, Pol p...@everpix.net wrote:
  Sorry, I realize I didn't explain everything: since the external
  system can handle an arbitrary load (as long as this load doesn't grow
  too fast) and requests get processed in 1 to 2 seconds, we already set
  the processing rate to the maximum (200), as well as the concurrent
  (200).

  The problem is not to limit the number of requests being handled, but
  preventing bursts e.g. scaling up smoothly.

  On Oct 30, 9:22 pm, Nicholas Verne nve...@google.com wrote:
   If your external system is called synchronously by your tasks, you could
   try setting the queue's max_concurrent_requests parameter in
  queue.yaml/xml

   This is documented for Java, but the same general rules apply for Python
  or
   Go applications.
 http://code.google.com/appengine/docs/java/config/queue.html#Defining...

   Nick Verne

   On Mon, Oct 31, 2011 at 3:13 PM, Pol p...@everpix.net wrote:
Hi,

Through a task queue, we manage calls from GAE to an external system,
which is fully-scalable but takes minutes to do so. The task queue
rate is therefore set to 200/s.

So what happens is that sometimes we get bursts of activity, and the
task queue sends 200 requests at once: the vast majority fails in the
external system as it can't handle this sudden request in load. But
the external system doesn't start scaling up as there are no more
requests coming and it's scaling based on the CPU load. Then suddenly
you get another burst of requests from the task queue retrying, and
then again the majority fails and so on.

So how can we configure a task queue to have a very high rate *but*
prevent it to get to this rate too fast? I noticed the bucket size
parameter, but I'm not sure how to use it properly.

Thanks,

- Pol

--
You received this message because you are subscribed to the Google
  Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
  .
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
   http://groups.google.com/group/google-appengine?hl=en.

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Announcing SSL for Custom Domains Trusted Tester Program

[Pol]

That's great news, but I'm wondering: if we put GAE over SSL on
https://example.com, will doing AJAX from https://www.example.com to
https://example.com still be considered cross-domain or not?

On Oct 18, 9:22 am, Jeff Schnitzer j...@infohazard.org wrote:
 +1 to this!

 I *do* need it... because everything through SSL all the time is
 rapidly approaching mandatory.  Firesheep changed everything - now
 users *realize* how insecure the web really is.

 Jeff







 On Mon, Oct 17, 2011 at 2:45 PM, Brandon Wirtz drak...@digerat.com wrote:
  Cayden and GAE Team,

  This is awesome.  I don't need it, but I know a lot of people in the
  community have asked for it, and I know they will be appreciative.  I am
  certain this took a lot of working internally to resolve limitations of
  other products, and likely some politics to get those solutions in place.
  This feature in my view, shows more than most Google's commitment to making
  this an enterprise class solution with as few deal breakers as possible
  for deployments.

  -Brandon

  Brandon Wirtz
  BlackWaterOps: President / Lead Mercenary

  Work: 510-992-6548
  Toll Free: 866-400-4536
  IM: drak...@gmail.com (Google Talk)
  Skype: drakegreene
  BlackWater Ops

  -Original Message-
  From: google-appengine@googlegroups.com
  [mailto:google-appengine@googlegroups.com] On Behalf Of Cayden Meyer
  Sent: Monday, October 17, 2011 1:13 AM
  To: Google App Engine
  Subject: [google-appengine] Announcing SSL for Custom Domains Trusted Tester
  Program

  Hey everyone,

  I am pleased to announce that we are accepting signups for the SSL for
  custom domains Trusted Tester Program. This will allow you to serve secure
  traffic for your App Engine application from your own
  domain(https://your.domain.com) rather than your appspot.com domain
  (https://your-app-id.appspot.com).

  We will be offering two types of SSL service, Server Name Indication
  (SNI) and Virtual IP(VIP). SNI will be significantly less expensive than VIP
  when this service is fully launched, however unlike VIP it does not work
  everywhere SSL is supported, notably it is not supported by IE and Safari on
  Windows XP. Multiple certificates are supported by SNI, while the VIP
  service only supports a single certificate per virtual IP address. Wildcard
  certificates and certificates with alternate names are supported by both SNI
  and VIP.

  Either a Free or Paid Google Apps account is required to use SSL. The use of
  multiple domains is supported via the aliasing feature in Google Apps.

  If you are interesting in signing up to test this feature, please fill in
  the form linked below.

 https://docs.google.com/a/google.com/spreadsheet/viewform?formkey=dHF...
  M1MUNpelFWcVJqcVAwckE6MQ

  Currently we are testing on a limited basis and will not be able to accept
  everybody who applies to the trusted tester program.

  As with all trusted tester programs, documentation is a work in progress.

  This feature is still in testing and as such we would advise against using
  this on production applications.

  If you have any queries, please email google-appengine-ssl-
  feedb...@google.com.

  Cheers,

  Cayden Meyer
  Product Manager, Google App Engine
  Blogger:http://googleappengine.blogspot.com
  Reddit:http://www.reddit.com/r/appengine
  Twitter:http://twitter.com/app_engine

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

  --
  You received this message because you are subscribed to the Google Groups 
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to 
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group 
  athttp://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Announcing SSL for Custom Domains Trusted Tester Program

[Pol]

That's great news, but I'm wondering: if we put GAE over SSL on
https://example.com, will doing AJAX from https://www.example.com to
https://example.com still be considered cross-domain or not?

On Oct 18, 9:22 am, Jeff Schnitzer j...@infohazard.org wrote:
 +1 to this!

 I *do* need it... because everything through SSL all the time is
 rapidly approaching mandatory.  Firesheep changed everything - now
 users *realize* how insecure the web really is.

 Jeff







 On Mon, Oct 17, 2011 at 2:45 PM, Brandon Wirtz drak...@digerat.com wrote:
  Cayden and GAE Team,

  This is awesome.  I don't need it, but I know a lot of people in the
  community have asked for it, and I know they will be appreciative.  I am
  certain this took a lot of working internally to resolve limitations of
  other products, and likely some politics to get those solutions in place.
  This feature in my view, shows more than most Google's commitment to making
  this an enterprise class solution with as few deal breakers as possible
  for deployments.

  -Brandon

  Brandon Wirtz
  BlackWaterOps: President / Lead Mercenary

  Work: 510-992-6548
  Toll Free: 866-400-4536
  IM: drak...@gmail.com (Google Talk)
  Skype: drakegreene
  BlackWater Ops

  -Original Message-
  From: google-appengine@googlegroups.com
  [mailto:google-appengine@googlegroups.com] On Behalf Of Cayden Meyer
  Sent: Monday, October 17, 2011 1:13 AM
  To: Google App Engine
  Subject: [google-appengine] Announcing SSL for Custom Domains Trusted Tester
  Program

  Hey everyone,

  I am pleased to announce that we are accepting signups for the SSL for
  custom domains Trusted Tester Program. This will allow you to serve secure
  traffic for your App Engine application from your own
  domain(https://your.domain.com) rather than your appspot.com domain
  (https://your-app-id.appspot.com).

  We will be offering two types of SSL service, Server Name Indication
  (SNI) and Virtual IP(VIP). SNI will be significantly less expensive than VIP
  when this service is fully launched, however unlike VIP it does not work
  everywhere SSL is supported, notably it is not supported by IE and Safari on
  Windows XP. Multiple certificates are supported by SNI, while the VIP
  service only supports a single certificate per virtual IP address. Wildcard
  certificates and certificates with alternate names are supported by both SNI
  and VIP.

  Either a Free or Paid Google Apps account is required to use SSL. The use of
  multiple domains is supported via the aliasing feature in Google Apps.

  If you are interesting in signing up to test this feature, please fill in
  the form linked below.

 https://docs.google.com/a/google.com/spreadsheet/viewform?formkey=dHF...
  M1MUNpelFWcVJqcVAwckE6MQ

  Currently we are testing on a limited basis and will not be able to accept
  everybody who applies to the trusted tester program.

  As with all trusted tester programs, documentation is a work in progress.

  This feature is still in testing and as such we would advise against using
  this on production applications.

  If you have any queries, please email google-appengine-ssl-
  feedb...@google.com.

  Cheers,

  Cayden Meyer
  Product Manager, Google App Engine
  Blogger:http://googleappengine.blogspot.com
  Reddit:http://www.reddit.com/r/appengine
  Twitter:http://twitter.com/app_engine

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

  --
  You received this message because you are subscribed to the Google Groups 
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to 
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group 
  athttp://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Announcing SSL for Custom Domains Trusted Tester Program

[Pol]

That's great news, but I'm wondering: if we put GAE over SSL on
https://example.com, will doing AJAX from https://www.example.com to
https://example.com still be considered cross-domain or not?

On Oct 18, 9:22 am, Jeff Schnitzer j...@infohazard.org wrote:
 +1 to this!

 I *do* need it... because everything through SSL all the time is
 rapidly approaching mandatory.  Firesheep changed everything - now
 users *realize* how insecure the web really is.

 Jeff







 On Mon, Oct 17, 2011 at 2:45 PM, Brandon Wirtz drak...@digerat.com wrote:
  Cayden and GAE Team,

  This is awesome.  I don't need it, but I know a lot of people in the
  community have asked for it, and I know they will be appreciative.  I am
  certain this took a lot of working internally to resolve limitations of
  other products, and likely some politics to get those solutions in place.
  This feature in my view, shows more than most Google's commitment to making
  this an enterprise class solution with as few deal breakers as possible
  for deployments.

  -Brandon

  Brandon Wirtz
  BlackWaterOps: President / Lead Mercenary

  Work: 510-992-6548
  Toll Free: 866-400-4536
  IM: drak...@gmail.com (Google Talk)
  Skype: drakegreene
  BlackWater Ops

  -Original Message-
  From: google-appengine@googlegroups.com
  [mailto:google-appengine@googlegroups.com] On Behalf Of Cayden Meyer
  Sent: Monday, October 17, 2011 1:13 AM
  To: Google App Engine
  Subject: [google-appengine] Announcing SSL for Custom Domains Trusted Tester
  Program

  Hey everyone,

  I am pleased to announce that we are accepting signups for the SSL for
  custom domains Trusted Tester Program. This will allow you to serve secure
  traffic for your App Engine application from your own
  domain(https://your.domain.com) rather than your appspot.com domain
  (https://your-app-id.appspot.com).

  We will be offering two types of SSL service, Server Name Indication
  (SNI) and Virtual IP(VIP). SNI will be significantly less expensive than VIP
  when this service is fully launched, however unlike VIP it does not work
  everywhere SSL is supported, notably it is not supported by IE and Safari on
  Windows XP. Multiple certificates are supported by SNI, while the VIP
  service only supports a single certificate per virtual IP address. Wildcard
  certificates and certificates with alternate names are supported by both SNI
  and VIP.

  Either a Free or Paid Google Apps account is required to use SSL. The use of
  multiple domains is supported via the aliasing feature in Google Apps.

  If you are interesting in signing up to test this feature, please fill in
  the form linked below.

 https://docs.google.com/a/google.com/spreadsheet/viewform?formkey=dHF...
  M1MUNpelFWcVJqcVAwckE6MQ

  Currently we are testing on a limited basis and will not be able to accept
  everybody who applies to the trusted tester program.

  As with all trusted tester programs, documentation is a work in progress.

  This feature is still in testing and as such we would advise against using
  this on production applications.

  If you have any queries, please email google-appengine-ssl-
  feedb...@google.com.

  Cheers,

  Cayden Meyer
  Product Manager, Google App Engine
  Blogger:http://googleappengine.blogspot.com
  Reddit:http://www.reddit.com/r/appengine
  Twitter:http://twitter.com/app_engine

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

  --
  You received this message because you are subscribed to the Google Groups 
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to 
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group 
  athttp://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Need help tracking datastore transaction or write too big warnings

[Pol]

Hi,

With some clients of our system, we're getting some datastore
transaction or write too big exceptions on transaction commits. The
requests takes 1mn and the process even exceeds the soft size limit
and gets killed, so it's pretty bad.

What are we hitting there? A maximum serialized entity size? What are
these exact limits?

Thanks,

- Pol

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Need help tracking datastore transaction or write too big warnings

[Pol]

Can anyone comment from Google comment on this 2nd point: write limits
on entire transaction? Our entities are 4KB on average so I'm trying
to track down what's happening but debugging on the server in a live
product is really not trivial.

On Oct 14, 12:25 pm, Simon Knott knott.si...@gmail.com wrote:
 The maximum size of an individual entity is 1MB for the datastore 
 -http://code.google.com/appengine/docs/java/datastore/overview.html#Qu...

 Apart from that limit, I'm not sure what write limits there are for the
 datastore for an individual transaction.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] date value out of range exception while committing transaction

[Pol]

Hi,

I just saw some cases of run_in_transaction() throwing an exception
date value out of range. What does this mean and how do I prevent
this?

Thanks,

- Pol

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Getting lots of Exception sending Rollback in the log

[Pol]

Hi,

I'm used to the transaction collision warnings, but it seems we're
getting more and Exception sending Rollback exception reports in the
log. What triggers these exceptions and what is the consequence if any
regarding the data consistency in the HRD?

Exception sending Rollback:
Traceback (most recent call last):
  File /base/python_runtime/python_lib/versions/1/google/appengine/
api/datastore.py, line 2505, in _DoOneTry
new_connection.rollback()
  File /base/python_runtime/python_lib/versions/1/google/appengine/
datastore/datastore_rpc.py, line 1985, in rollback
return rpc.get_result()
  File /base/python_runtime/python_lib/versions/1/google/appengine/
api/apiproxy_stub_map.py, line 592, in get_result
return self.__get_result_hook(self)
  File /base/python_runtime/python_lib/versions/1/google/appengine/
datastore/datastore_rpc.py, line 2007, in __rollback_hook
self.check_rpc_success(rpc)
  File /base/python_runtime/python_lib/versions/1/google/appengine/
datastore/datastore_rpc.py, line 1182, in check_rpc_success
raise _ToDatastoreError(err)
BadRequestError: The referenced transaction has expired or is no
longer valid.

- Pol

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Dashboard says 5.69GB of High Replication Data but Datastore Statistics report 2GB of entities?

[Pol]

Hi,

Datastore Statistics do include the extra Metadata for entities, so
where are these mysterious 3.69GB coming from??

- Pol

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] How to track down urlfetch() Application Error 2?

[Pol]

Hi,

Several times a day we get some ApplicationError: 2 exceptions when
communicating from GAE to EC2 through urlfetch(). How do I get more
info to understand exactly what's happening? I don't see anything
extra in the log and I'm not sure there's more info on the exception
object itself.

...
start = time.time()
try:
  result = urlfetch.fetch(PROCESSOR_URL + ? + query, headers={'Cache-
Control': 'no-cache,max-age=0', 'Pragma': 'no-cache'}, deadline=10)
  if not DEVELOPMENT and result.headers['X-Google-Cache-Control'] !=
'remote-fetch':
raise RuntimeError(Unexpected results headers: %s %
result.headers)
  response = json.loads(result.content)
except Exception, e:
  end = time.time()
  logging.error(EC2 processor failed handling query '%s' in %.1f
seconds: %s % (query, end - start, e))
  return
...

Thanks

- Pierre

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Custom Administration Console pages with admin login requirement result in redirect loops

[Pol]

Hi,

I've added custom admin pages but when I try to follow the links added
to the left pane of the Administration Console, it results in redirect
loops. Looking at the generated HTML, the iframe with the custom admin
page points to https://1755-dot-latest-dot-everpix-alpha.appspot.com/
admin/garbage_collect_backings. Going directly to this page (wether
already logged in as an admin or not) works fine, it's only from
within the Administration Console that it doesn't work.

Here's the first redirect:

Request URL:https://1755-dot-latest-dot-everpix-alpha.appspot.com/
admin/garbage_collect_backings
Request Method:GET
Status Code:302 Found
Response Headersview source
content-length:0
content-type:text/html
date:Sun, 18 Sep 2011 13:12:30 GMT
location:https://www.google.com/a/everpix.net/ServiceLogin?
service=ahpassive=truecontinue=https://appengine.google.com/_ah/
conflogin%3Fcontinue%3Dhttps://1755-dot-latest-dot-everpix-
alpha.appspot.com/admin/
garbage_collect_backingsltmpl=gashdf=ChkLEgZhaG5hbWUaDUV2ZXJwaXggQWxwaGEMEgJhaCIUs8mSvSwhTsHxTKlcPf-1VgYU-
f4oATIU8IshzygSsQah4xFNA5qqA98EfBY
server:Google Frontend
status:302 Found
version:HTTP/1.1

And the second one:

Request URL:https://www.google.com/a/everpix.net/ServiceLogin?
service=ahpassive=truecontinue=https://appengine.google.com/_ah/
conflogin%3Fcontinue%3Dhttps://1755-dot-latest-dot-everpix-
alpha.appspot.com/admin/
garbage_collect_backingsltmpl=gashdf=ChkLEgZhaG5hbWUaDUV2ZXJwaXggQWxwaGEMEgJhaCIUs8mSvSwhTsHxTKlcPf-1VgYU-
f4oATIU8IshzygSsQah4xFNA5qqA98EfBY
Request Method:GET
Status Code:302 Moved Temporarily
Response Headersview source
cache-control:private, max-age=0
content-encoding:gzip
content-length:422
content-type:text/html; charset=UTF-8
date:Sun, 18 Sep 2011 13:12:31 GMT
expires:Sun, 18 Sep 2011 13:12:31 GMT
location:https://www.google.com/accounts/ServiceLogin?continue=https%3A
%2F%2Fappengine.google.com%2F_ah%2Fconflogin%3Fcontinue%3Dhttps%3A%2F
%2F1755-dot-latest-dot-everpix-alpha.appspot.com%2Fadmin
%2Fgarbage_collect_backingsshdf=ChkLEgZhaG5hbWUaDUV2ZXJwaXggQWxwaGEMEgJhaCIUs8mSvSwhTsHxTKlcPf-1VgYU-
f4oATIU8IshzygSsQah4xFNA5qqA98EfBYservice=ahltmpl=gapassive=trueauthuser=0
server:GSE
status:302 Moved Temporarily
version:HTTP/1.1
x-content-type-options:nosniff
x-xss-protection:1; mode=block


- app.yaml -

builtins:
- remote_api: on
- appstats: on

inbound_services:
- xmpp_message
- warmup

admin_console:
  pages:
  - name: Flush Memcache
url: /admin/flush_memcache
  - name: Reprocess Backings
url: /admin/reprocess_backings
  - name: Reprocess Photos
url: /admin/reprocess_photos
  - name: GC Backings
url: /admin/garbage_collect_backings

handlers:
- url: /favicon\.ico
  static_files: static/favicon.ico
  upload: static/favicon\.ico
- url: /mapreduce(/.*)?
  script: mapreduce/main.py
  login: admin
- url: /ereporter.*
  script: $PYTHON_LIB/google/appengine/ext/ereporter/
report_generator.py
  login: admin
- url: /stats.*
  script: $PYTHON_LIB/google/appengine/ext/appstats/ui.py
- url: /_ah/xmpp/.*
  script: xmpp_handlers.py
  login: admin
  secure: never
- url: /cron/.*
  script: cron_handlers.py
  login: admin
  secure: never
- url: /task/.*
  script: task_handlers.py
  login: admin
  secure: never
- url: /api/.*
  script: api_handlers.py
  secure: always
- url: /admin/.*
  script: admin.py
  login: admin
  secure: always
- url: /
  script: main.py
  secure: never
- url: /register
  script: main.py
  secure: never

- Pol

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Need help with App Engine over HTTPS on IE8-9

[Pol]

Hi,

We have a HTML5 front-end on a HTTP CDN (CloudFront) that just talks
over AJAX / HTTPS to Google App Engine. It's fine on Chrome, Safari
and Firefox, but what's the best way to make this work with IE8/9
considering IE's XDomainRequest doesn't let you do HTTPS from a
website hosted on HTTP (protocol mismatch)?

We can switch CDNs to one that allows HTTPS hosting on custom CNAMEs,
but is there any simpler workaround like using Flash or ActiveX for
the communication?

Even if GAE let us do HTTPS with custom CNAMES at some point, it would
still be considered cross-domain from www.everpix.net to
gae.everpix.net, right?

Thanks!

- Pol

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Min idle instances setting not working at all?

[Pol]

Hi Jon,

I don't know where you got that information, the admin logs clearly
reflect our actions and the feedback at the time in the Dashboard:

Originally set to 5:
2011-08-31 23:15:59 ad...@everpix.net   Changed Performance Settings
max_idle_clones=5, min_pending_latency=0.10

Then looking at estimated new-pricing bills, I figured that's looks
very expensive, let's reduce this as much as possible for now:
2011-09-01 00:08:03 ad...@everpix.net   Changed Performance Settings
max_idle_clones=3, min_pending_latency=0.10
2011-09-01 18:44:01 ad...@everpix.net   Changed Performance Settings
max_idle_clones=2, min_pending_latency=0.10
2011-09-03 12:07:44 ad...@everpix.net   Changed Performance Settings
max_idle_clones=1, min_pending_latency=0.10
2011-09-03 12:08:15 ad...@everpix.net   Changed Performance Settings
max_idle_clones=2, min_pending_latency=0.10

Then 2 days before our public launch at TechCrunch Disrupt, change of
mind: now we need a minimum of 5 instances all the time, just in case:
2011-09-10 10:16:23 ad...@everpix.net   Changed Performance Settings
max_idle_clones=5, min_pending_latency=0.10

The next morning, I look at the graph (attached to previous post),
realize it's obviously not working and decide to re-enable always-on
in billing so we at least get 3.
2011-09-11 09:47:15 ad...@everpix.net   Changed Performance Settings
max_idle_clones=5, min_pending_latency=0.10
2011-09-11 09:47:03 ad...@everpix.net   Changed Performance Settings
max_idle_clones=5, min_pending_latency=0.10

And I post to the mailing list a few minutes later at 9:53AM

I guarantee you min-instances was set to 5 in the dashboard for all
the duration of the previously attached graph :)

On Sep 12, 10:36 am, Jon McAlister jon...@google.com wrote:
 Nevermind, I remember, it's everpix-alpha.

 I see that min-idle-instances was removed on 2011/08/31-19:49:59 and
 then set to 3 on 2011/09/11-09:47:38. So, it wasn't on when you emailed but
 is on now. Your instances graph now looks correct.







 On Mon, Sep 12, 2011 at 10:24 AM, Jon McAlister jon...@google.com wrote:
  What's the app-id?

  On Sun, Sep 11, 2011 at 9:53 AM, Pol-Online i...@pol-online.net wrote:

  Hi,

  Because we are about to launch our app very soon, I increased the number
  of idle instances from 1 to 5 yesterday and things looked correct in the
  Dashboard.

  Then this morning, in the Dashboard I see 2 idle instances and this:

  The spikes correspond to cron tasks running hourly I assume.

  Anyway, it looks like either the graph is wrong or the setting doesn't
  work, or the definition of what the setting does is wrong, because clearly
  it not stable at 5.

  I just activated Always On to work around the issue, so now it's back to
  3+1.

  -Pol

  
  Pol-Online
  i...@pol-online.net

   --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.



  PastedGraphic-11.png
 37KViewDownload

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Min idle instances setting not working at all?

[Pol]

I typed too fast, I meant *max* idle instances in the Dashboard. Sorry
about the confusion.

That said, the description of the slider is not very clear: it's
called max which is correct from a billing perspective, but from a
functionality perspective, it really looks like you're setting the
default almost min number of idle instances, especially
considering that Always On disappears in the new system, right?

On Sep 13, 12:48 am, Daniel Florey daniel.flo...@gmail.com wrote:
 Where did you find the min idle instances setting?

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Re-entrancy / deadlock bug with scheduler?

[Pol]

Still happening right now. I can repro pretty easily: schedule ~10
image processing tasks in a queue, and 1 to 3 often fail due to that
re-entrancy bug. Then after they retry automatically and after a few
times usually go through.

5 instances idle / running, no load on the app: only 1 user currently.

It really looks as if the scheduler when getting the callback from the
EC2 box is trying to serve it from the same instance from which
urlfetch() is running: there must be a bug where the scheduler thinks
the instance is not busy or something.

If you look at the distribution of requests, something's clearly wrong
as 1 instance gets the majority of the traffic.

QPS*Latency*RequestsErrors  Age Memory  Availability
0.000   0.0 ms  7   0   0:03:30 17.7 MBytes Dynamic
0.000   0.0 ms  152 1   0:26:56 68.6 MBytes Dynamic
0.000   0.0 ms  6   0   0:32:22 23.9 MBytes Dynamic
0.000   0.0 ms  4   0   0:32:12 24.6 MBytes Dynamic
0.000   0.0 ms  3   0   0:32:09 14.5 MBytes Dynamic

On Sep 9, 1:43 pm, Jon McAlister jon...@google.com wrote:
 Ok, let me know if it recurs.







 On Wed, Sep 7, 2011 at 3:07 PM, Pol i...@pol-online.net wrote:
  Hi Jon,

  Great!

  everpix-alpha

  I still see what appears to be the bug now and then, but I don't have
  a reproducible case just waiting in a task queue anymore.

  On Sep 7, 1:19 pm, Jon McAlister jon...@google.com wrote:
  Hi Pol,

  I think I have a change that will fix this, which should be available
  once 1.5.4 is pushed out. What is the app-id here?

  On Mon, Sep 5, 2011 at 10:41 AM, Pol i...@pol-online.net wrote:
   Hi,

   Our app settings are as follow:
   - Python +HRD
   - Max Idle Instances: ( 2 )
   - Min Pending Latency: ( 100ms )
   As of right now, there are 3 instances alive.

   Without going too much into details, we have GAE integrated with EC2
   on which we run remote image processing tools. The tools are called
   directly using HTTP GETs from GAE and they returned their results as
   JSON (with gzip content encoding).

   There are currently 3 tasks in the processing queue on GAE
   continuously failing: the urlfetch() calls to the EC2 tool reach the
   10 seconds timeout and bail. What doesn't make sense is that calling
   the EC2 tool directly using curl from random machines succeeds in less
   than 1 second.

   But here's the trick: under certain circumstances, the EC2 tool will
   call back to GAE (HEAD request that does a single db.get()) to check
   if the image has already been processed and this happens for these 3
   stuck tasks.

   If calling the EC2 tool from the command line and curl, we have the
   normal behavior:
   - EC2 tool retrieves image from arbitrary URL and computes a hash
   - EC2 tool does a HEAD call to GAE passing this hash to see if image
   has been already processed
    - If yes, return empty JSON
    - If no, process image and return full JSON
   This takes about 1 second.

   The exact same call done from GAE produces this behavior:
   - EC2 tool retrieves image from arbitrary URL and computes a hash
   - EC2 tool does a HEAD call to GAE passing this hash to see if image
   has been already processed
    - HEAD call hangs  --- RE-ENTRANCY /DEADLOCKBUG in GAE
    - urlfetch() from GAE to EC2 reaches 10 seconds deadline and
   aborts  -- BREAKSDEADLOCK
    - HEAD call finally returns
   - EC2 tool completes normally

   GAE logs confirm the bug:

   HEAD call from EC2 / curl origina
   2011-09-05 10:19:52.502 /api/has_backing?
   bid=90e794f348ac76520076f5d02bc67c87c8a9185b8d36affe8377e73fe4820703
   200 368ms 48cpu_ms 8api_cpu_ms 0kb Everpix-Processor

   HEAD call from EC2 / GAE origin
   2011-09-05 10:20:44.670 /api/has_backing?
   bid=90e794f348ac76520076f5d02bc67c87c8a9185b8d36affe8377e73fe4820703
   200 9712ms 48cpu_ms 8api_cpu_ms 0kb Everpix-Processor
   2011-09-05 10:20:44.547 /task/import_photo 500 10348ms 28cpu_ms
   8api_cpu_ms 0kb AppEngine-Google; (+http://code.google.com/appengine)
   (see how the HEAD /api/has_backing call hangs for almost 10 seconds
   and only returns *after* /task/import_photo and its urlfetch() call to
   EC2 has aborted)

   And finally, AppStats confirms that it's not the head() Python
   execution itself that's hanging:

   (1) 2011-09-05 09:16:06.843 HEAD /api/has_backing?
   bid=3bc4aeb08e01d3ba4bfab3282d2a198984a4fc1fab2ad9d1e8a39ee3cddd14da
   200 real=227ms cpu=24ms api=8ms overhead=0ms (1 RPC)
   (2) 2011-09-05 09:15:56.422 POST /task/import_photo 500 real=10002ms
   cpu=33ms api=8ms overhead=0ms (3 RPCs)
   (3) 2011-09-05 09:15:49.328 HEAD /api/has_backing?
   bid=90e794f348ac76520076f5d02bc67c87c8a9185b8d36affe8377e73fe4820703
   200 real=297ms cpu=21ms api=8ms overhead=0ms (1 RPC)

   This issue is currently 100% reproducible.

   - Pol

   --
   You received this message because you are subscribed to the Google 
   Groups Google App Engine group.
   To post to this group, send email

[google-appengine] Min idle instances setting not working at all?

[Pol-Online]

Hi,

Because we are about to launch our app very soon, I increased the number of 
idle instances from 1 to 5 yesterday and things looked correct in the Dashboard.

Then this morning, in the Dashboard I see 2 idle instances and this:



The spikes correspond to cron tasks running hourly I assume.

Anyway, it looks like either the graph is wrong or the setting doesn't work, or 
the definition of what the setting does is wrong, because clearly it not stable 
at 5.

I just activated Always On to work around the issue, so now it's back to 3+1.

-Pol


Pol-Online
i...@pol-online.net




-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

inline: PastedGraphic-11.png

[google-appengine] Re: Min idle instances setting not working at all?

[Pol]

Oops looks like the image was stripped, you can see it here:
http://oi52.tinypic.com/12349hi.jpg

On Sep 11, 9:53 am, Pol-Online i...@pol-online.net wrote:
 Hi,

 Because we are about to launch our app very soon, I increased the number of 
 idle instances from 1 to 5 yesterday and things looked correct in the 
 Dashboard.

 Then this morning, in the Dashboard I see 2 idle instances and this:

 The spikes correspond to cron tasks running hourly I assume.

 Anyway, it looks like either the graph is wrong or the setting doesn't work, 
 or the definition of what the setting does is wrong, because clearly it not 
 stable at 5.

 I just activated Always On to work around the issue, so now it's back to 3+1.

 -Pol

 
 Pol-Online
 i...@pol-online.net

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Why app engine new price model is totally wrong

[Pol]

On Sep 7, 2:53 am, Andrew Cassidy an...@bytz.co.uk wrote:
 I wasn't trying to say 'You can replicate GAE on AWS trivially, GAE is a
 massive beast that does one thing in particular that I haven't experimented
 with yet: automatically creating and destroying running instances. I was
 trying to say you could host your own scaling application on any
 infrastructure you like with a little know how.

Anyone can write a web page, an iPhone app or even drive a car on a
race track with a little know how. That's doesn't you won't crash at
the first serious problem.

Look, we can debate all day long on the perceived value of GAE is and
what a fair price for the service, but putting this type of service
together does require quite a bit of know how.

- Pol

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.

[Pol]

Great to see the problem acknowledged, so why is the HDR line still
all green at http://code.google.com/status/appengine?

The HRD is having problems now (yes, not as bad pas a MS problems, but
that's beside the point as the expectancies have been officially
raised quite a bit for HRD), non-negligible problems, officially
acknowledged problems, so not putting it in the status page is
deceptive to say the least.

That line ought to be orange.

On Sep 6, 11:43 am, Gregory D'alesandre gr...@google.com wrote:
 Hello All,

 This is an issue right now related to higher latency on HRD, particularly
 affecting queries.  We are working on potential fixes to mitigate the issue.
  Once we have determined the full scope of it, we will provide further
 information.  This is not expected behavior for HRD and is indeed a recent
 issue, it has *not* been a slow steady increase based on load or a
 fundamental issue with the way HRD is built.  The issue with M/S was that it
 heavily relied on a fundamentally inconsistent piece of infrastructure, this
 is not true with HRD.

 Greg

 On Tue, Sep 6, 2011 at 9:17 AM, Jason Collins 
 jason.a.coll...@gmail.comwrote:







  Those trends are concerning. We're putting a large effort in to
  convert our app over to HR. Performance consistency is the only reason
  we're doing this.
  j

  On Sep 6, 7:57 am, Mike Wesner mbwes...@gmail.com wrote:
   Thanks for putting those together.  It very clearly shows that
   recently things have changed.

   It would be nice to hear from Greg or someone about this.

   -Mike

   On Sep 6, 7:11 am, Simon Knott knott.si...@gmail.com wrote:

I've attached graphs showing the daily stats for each of the HR
  operations
for the last year, driven from the GAE Status stats.
I'll grab the M/S datastore statistics, when I get the time, to give a
comparison.


 https://lh4.googleusercontent.com/-MV_ixL3Jz0Y/TmYNsX9QEGI/AF...


 https://lh4.googleusercontent.com/-6yrWNd7wfxU/TmYNoldrFWI/AF...


 https://lh6.googleusercontent.com/-NiRHqyRtbro/TmYNgZLeRXI/AF...


 https://lh3.googleusercontent.com/-WGYYZUzXiAg/TmYNk35ZztI/AF...


 https://lh5.googleusercontent.com/-5GRqG-Sy17A/TmYN7E7C60I/AF...

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] App Stats time measurements for GQL queries way off?

[Pol]

Hi,

I'm trying to optimize our Python app and have App Stats installed.

The code to measure (simplified):

query = db.GqlQuery(SELECT * FROM Event WHERE ANCESTOR IS :1
ORDER BY max_date DESC, self.user.key())
if cursor:
  query.with_cursor(cursor)
start_time = time.time()
for event in query.fetch(limit):
  ...
logging.info(DELTA 1 = %.2f seconds % (time.time() -
start_time))
photos = Photo.get(keys)
logging.info(DELTA 2 = %.2f seconds % (time.time() -
start_time))
...
logging.info(DELTA 3 = %.2f seconds % (time.time() -
start_time))

self.write_json_response(...)

logging.info(DELTA 4 = %.2f seconds % (time.time() -
start_time))

From App Stats:

@0ms memcache.Get real=5ms api=0ms
@8ms datastore_v3.RunQuery real=89ms api=1387ms
@537ms datastore_v3.Get real=1574ms api=6250ms  --- Photo.get(...)

From the log:

67.169.78.38 - - [07/Sep/2011:11:04:15 -0700] GET /api/event_list?
keyPhotos=1limit=150cursor=E-
ABAOsB8gEIbWF4X2RhdGX6AQkIgK7os5aSiwLsAYICiAFqD3N-
ZXZlcnBpeC1hbHBoYXJ1CxIEVXNlciIgNWI3NmY0MjhkOGFiMTFlMDgwNWU5YjNmM2U1NDdmMWMMCxIFRXZlbnQiQDlhOWQ4MjljZDhkOTExZTBhZTQ0OWJiMWViNDM3NjUzMTIxNDMxNGVjZmQwNDBjZjlhNDc5YzFmNjFmYmEzNmMMFA==
HTTP/1.1 200 48637 - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8)
AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.220 Safari/
535.1 everpix-alpha.appspot.com ms=4175 cpu_ms=10729
api_cpu_ms=7638 cpm_usd=0.303547 throttle_code=1
instance=00c61b117c52d19855e6bf84b0977f343fba
I 2011-09-07 11:04:11.609
DELTA 1 = 0.45 seconds
I 2011-09-07 11:04:15.172
DELTA 2 = 4.01 seconds
I 2011-09-07 11:04:15.189
DELTA 3 = 4.03 seconds
I 2011-09-07 11:04:15.254
DELTA 4 = 4.09 seconds
I 2011-09-07 11:04:15.286
Saved; key: __appstats__:051100, part: 368 bytes, full: 14629 bytes,
overhead: 0.001 + 0.006; link: 
http://everpix-alpha.appspot.com/_ah/stats/details?time=1315418651153

And the problem:

App Stats states that Photo.get(...) took 1574ms while inline
measurements show a much bigger 3.56s? Does App Stats ignore de-
serialization of the entity properties or something?

- Pol

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: App Stats time measurements for GQL queries way off?

[Pol]

I don't know the exact count for this Photo.get(...) call but it can
be around 2,000.

If it's related to deserialization then my guess is there's a high
overhead due one of the Photo class properties: it contains a
dictionary for EXIF metadata serialized as text.



class DictionaryProperty(db.TextProperty):

  def __init__(self, verbose_name=None, default={}, **kwds):
super(DictionaryProperty, self).__init__(verbose_name,
default=default, **kwds)

  def get_value_for_datastore(self, model_instance):
value = super(DictionaryProperty,
self).get_value_for_datastore(model_instance)
return db.Text(repr(value))

  def make_value_from_datastore(self, value):
value = super(DictionaryProperty,
self).make_value_from_datastore(value)
return eval(str(value))

  def validate(self, value):
if not isinstance(value, dict):
  raise ValueError('Property %s must be a dictionary' % self.name)
return value



I'm not a pro at this stuff yet, can you help modifying this class so
that it does lazy text - dictionary deserialization only the first
time the property is actually accessed instead of when read from the
datastore?


On Sep 7, 12:48 pm, Ikai Lan (Google) ika...@google.com wrote:
 AppStats should only be tracking the RPC, not the time it would take to
 deserialize, so your guess is probably correct. How many entities are you
 retrieving?

 --
 Ikai Lan
 Developer Programs Engineer, Google App Engine
 plus.ikailan.com | twitter.com/ikai







 On Wed, Sep 7, 2011 at 11:12 AM, Pol i...@pol-online.net wrote:
  Hi,

  I'm trying to optimize our Python app and have App Stats installed.

  The code to measure (simplified):

     query = db.GqlQuery(SELECT * FROM Event WHERE ANCESTOR IS :1
  ORDER BY max_date DESC, self.user.key())
     if cursor:
       query.with_cursor(cursor)
     start_time = time.time()
     for event in query.fetch(limit):
       ...
     logging.info(DELTA 1 = %.2f seconds % (time.time() -
  start_time))
     photos = Photo.get(keys)
     logging.info(DELTA 2 = %.2f seconds % (time.time() -
  start_time))
     ...
     logging.info(DELTA 3 = %.2f seconds % (time.time() -
  start_time))

     self.write_json_response(...)

     logging.info(DELTA 4 = %.2f seconds % (time.time() -
  start_time))

  From App Stats:

  @0ms memcache.Get real=5ms api=0ms
  @8ms datastore_v3.RunQuery real=89ms api=1387ms
  @537ms datastore_v3.Get real=1574ms api=6250ms  --- Photo.get(...)

  From the log:

  67.169.78.38 - - [07/Sep/2011:11:04:15 -0700] GET /api/event_list?
  keyPhotos=1limit=150cursor=E-
  ABAOsB8gEIbWF4X2RhdGX6AQkIgK7os5aSiwLsAYICiAFqD3N-

  ZXZlcnBpeC1hbHBoYXJ1CxIEVXNlciIgNWI3NmY0MjhkOGFiMTFlMDgwNWU5YjNmM2U1NDdmMWM 
  MCxIFRXZlbnQiQDlhOWQ4MjljZDhkOTExZTBhZTQ0OWJiMWViNDM3NjUzMTIxNDMxNGVjZmQwND 
  BjZjlhNDc5YzFmNjFmYmEzNmMMFA==
  HTTP/1.1 200 48637 - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8)
  AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.220 Safari/
  535.1 everpix-alpha.appspot.com ms=4175 cpu_ms=10729
  api_cpu_ms=7638 cpm_usd=0.303547 throttle_code=1
  instance=00c61b117c52d19855e6bf84b0977f343fba
  I 2011-09-07 11:04:11.609
  DELTA 1 = 0.45 seconds
  I 2011-09-07 11:04:15.172
  DELTA 2 = 4.01 seconds
  I 2011-09-07 11:04:15.189
  DELTA 3 = 4.03 seconds
  I 2011-09-07 11:04:15.254
  DELTA 4 = 4.09 seconds
  I 2011-09-07 11:04:15.286
  Saved; key: __appstats__:051100, part: 368 bytes, full: 14629 bytes,
  overhead: 0.001 + 0.006; link:
 http://everpix-alpha.appspot.com/_ah/stats/details?time=1315418651153

  And the problem:

  App Stats states that Photo.get(...) took 1574ms while inline
  measurements show a much bigger 3.56s? Does App Stats ignore de-
  serialization of the entity properties or something?

  - Pol

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Re-entrancy / deadlock bug with scheduler?

[Pol]

Hi Jon,

Great!

everpix-alpha

I still see what appears to be the bug now and then, but I don't have
a reproducible case just waiting in a task queue anymore.

On Sep 7, 1:19 pm, Jon McAlister jon...@google.com wrote:
 Hi Pol,

 I think I have a change that will fix this, which should be available
 once 1.5.4 is pushed out. What is the app-id here?







 On Mon, Sep 5, 2011 at 10:41 AM, Pol i...@pol-online.net wrote:
  Hi,

  Our app settings are as follow:
  - Python +HRD
  - Max Idle Instances: ( 2 )
  - Min Pending Latency: ( 100ms )
  As of right now, there are 3 instances alive.

  Without going too much into details, we have GAE integrated with EC2
  on which we run remote image processing tools. The tools are called
  directly using HTTP GETs from GAE and they returned their results as
  JSON (with gzip content encoding).

  There are currently 3 tasks in the processing queue on GAE
  continuously failing: the urlfetch() calls to the EC2 tool reach the
  10 seconds timeout and bail. What doesn't make sense is that calling
  the EC2 tool directly using curl from random machines succeeds in less
  than 1 second.

  But here's the trick: under certain circumstances, the EC2 tool will
  call back to GAE (HEAD request that does a single db.get()) to check
  if the image has already been processed and this happens for these 3
  stuck tasks.

  If calling the EC2 tool from the command line and curl, we have the
  normal behavior:
  - EC2 tool retrieves image from arbitrary URL and computes a hash
  - EC2 tool does a HEAD call to GAE passing this hash to see if image
  has been already processed
   - If yes, return empty JSON
   - If no, process image and return full JSON
  This takes about 1 second.

  The exact same call done from GAE produces this behavior:
  - EC2 tool retrieves image from arbitrary URL and computes a hash
  - EC2 tool does a HEAD call to GAE passing this hash to see if image
  has been already processed
   - HEAD call hangs  --- RE-ENTRANCY / DEADLOCK BUG in GAE
   - urlfetch() from GAE to EC2 reaches 10 seconds deadline and
  aborts  -- BREAKS DEADLOCK
   - HEAD call finally returns
  - EC2 tool completes normally

  GAE logs confirm the bug:

  HEAD call from EC2 / curl origina
  2011-09-05 10:19:52.502 /api/has_backing?
  bid=90e794f348ac76520076f5d02bc67c87c8a9185b8d36affe8377e73fe4820703
  200 368ms 48cpu_ms 8api_cpu_ms 0kb Everpix-Processor

  HEAD call from EC2 / GAE origin
  2011-09-05 10:20:44.670 /api/has_backing?
  bid=90e794f348ac76520076f5d02bc67c87c8a9185b8d36affe8377e73fe4820703
  200 9712ms 48cpu_ms 8api_cpu_ms 0kb Everpix-Processor
  2011-09-05 10:20:44.547 /task/import_photo 500 10348ms 28cpu_ms
  8api_cpu_ms 0kb AppEngine-Google; (+http://code.google.com/appengine)
  (see how the HEAD /api/has_backing call hangs for almost 10 seconds
  and only returns *after* /task/import_photo and its urlfetch() call to
  EC2 has aborted)

  And finally, AppStats confirms that it's not the head() Python
  execution itself that's hanging:

  (1) 2011-09-05 09:16:06.843 HEAD /api/has_backing?
  bid=3bc4aeb08e01d3ba4bfab3282d2a198984a4fc1fab2ad9d1e8a39ee3cddd14da
  200 real=227ms cpu=24ms api=8ms overhead=0ms (1 RPC)
  (2) 2011-09-05 09:15:56.422 POST /task/import_photo 500 real=10002ms
  cpu=33ms api=8ms overhead=0ms (3 RPCs)
  (3) 2011-09-05 09:15:49.328 HEAD /api/has_backing?
  bid=90e794f348ac76520076f5d02bc67c87c8a9185b8d36affe8377e73fe4820703
  200 real=297ms cpu=21ms api=8ms overhead=0ms (1 RPC)

  This issue is currently 100% reproducible.

  - Pol

  --
  You received this message because you are subscribed to the Google Groups 
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to 
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group 
  athttp://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: App Stats time measurements for GQL queries way off?

[Pol]

Looking at the implementation of ReferenceProperty, I was able to
rewrite my DictionaryProperty with a similar design and lazily
deserialize the dictionary, and confirmed it works as expected. The
function make_value_from_datastore is no-op now, no more eval() calls.

But it makes no visible difference in performance on AppEngine. Is
TextProperty just slow? It's just used to store a few hundred bytes of
EXIF as a repr()'d Python dict.

On Sep 7, 1:29 pm, Pol i...@pol-online.net wrote:
 I don't know the exact count for this Photo.get(...) call but it can
 be around 2,000.

 If it's related to deserialization then my guess is there's a high
 overhead due one of the Photo class properties: it contains a
 dictionary for EXIF metadata serialized as text.

 

 class DictionaryProperty(db.TextProperty):

   def __init__(self, verbose_name=None, default={}, **kwds):
     super(DictionaryProperty, self).__init__(verbose_name,
 default=default, **kwds)

   def get_value_for_datastore(self, model_instance):
     value = super(DictionaryProperty,
 self).get_value_for_datastore(model_instance)
     return db.Text(repr(value))

   def make_value_from_datastore(self, value):
     value = super(DictionaryProperty,
 self).make_value_from_datastore(value)
     return eval(str(value))

   def validate(self, value):
     if not isinstance(value, dict):
       raise ValueError('Property %s must be a dictionary' % self.name)
     return value

 

 I'm not a pro at this stuff yet, can you help modifying this class so
 that it does lazy text - dictionary deserialization only the first
 time the property is actually accessed instead of when read from the
 datastore?

 On Sep 7, 12:48 pm, Ikai Lan (Google) ika...@google.com wrote:







  AppStats should only be tracking the RPC, not the time it would take to
  deserialize, so your guess is probably correct. How many entities are you
  retrieving?

  --
  Ikai Lan
  Developer Programs Engineer, Google App Engine
  plus.ikailan.com | twitter.com/ikai

  On Wed, Sep 7, 2011 at 11:12 AM, Pol i...@pol-online.net wrote:
   Hi,

   I'm trying to optimize our Python app and have App Stats installed.

   The code to measure (simplified):

      query = db.GqlQuery(SELECT * FROM Event WHERE ANCESTOR IS :1
   ORDER BY max_date DESC, self.user.key())
      if cursor:
        query.with_cursor(cursor)
      start_time = time.time()
      for event in query.fetch(limit):
        ...
      logging.info(DELTA 1 = %.2f seconds % (time.time() -
   start_time))
      photos = Photo.get(keys)
      logging.info(DELTA 2 = %.2f seconds % (time.time() -
   start_time))
      ...
      logging.info(DELTA 3 = %.2f seconds % (time.time() -
   start_time))

      self.write_json_response(...)

      logging.info(DELTA 4 = %.2f seconds % (time.time() -
   start_time))

   From App Stats:

   @0ms memcache.Get real=5ms api=0ms
   @8ms datastore_v3.RunQuery real=89ms api=1387ms
   @537ms datastore_v3.Get real=1574ms api=6250ms  --- Photo.get(...)

   From the log:

   67.169.78.38 - - [07/Sep/2011:11:04:15 -0700] GET /api/event_list?
   keyPhotos=1limit=150cursor=E-
   ABAOsB8gEIbWF4X2RhdGX6AQkIgK7os5aSiwLsAYICiAFqD3N-

   ZXZlcnBpeC1hbHBoYXJ1CxIEVXNlciIgNWI3NmY0MjhkOGFiMTFlMDgwNWU5YjNmM2U1NDdmMWM

   MCxIFRXZlbnQiQDlhOWQ4MjljZDhkOTExZTBhZTQ0OWJiMWViNDM3NjUzMTIxNDMxNGVjZmQwND
BjZjlhNDc5YzFmNjFmYmEzNmMMFA==
   HTTP/1.1 200 48637 - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8)
   AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.220 Safari/
   535.1 everpix-alpha.appspot.com ms=4175 cpu_ms=10729
   api_cpu_ms=7638 cpm_usd=0.303547 throttle_code=1
   instance=00c61b117c52d19855e6bf84b0977f343fba
   I 2011-09-07 11:04:11.609
   DELTA 1 = 0.45 seconds
   I 2011-09-07 11:04:15.172
   DELTA 2 = 4.01 seconds
   I 2011-09-07 11:04:15.189
   DELTA 3 = 4.03 seconds
   I 2011-09-07 11:04:15.254
   DELTA 4 = 4.09 seconds
   I 2011-09-07 11:04:15.286
   Saved; key: __appstats__:051100, part: 368 bytes, full: 14629 bytes,
   overhead: 0.001 + 0.006; link:
  http://everpix-alpha.appspot.com/_ah/stats/details?time=1315418651153

   And the problem:

   App Stats states that Photo.get(...) took 1574ms while inline
   measurements show a much bigger 3.56s? Does App Stats ignore de-
   serialization of the entity properties or something?

   - Pol

   --
   You received this message because you are subscribed to the Google Groups
   Google App Engine group.
   To post to this group, send email to google-appengine@googlegroups.com.
   To unsubscribe from this group, send email to
   google-appengine+unsubscr...@googlegroups.com.
   For more options, visit this group at
  http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options

[google-appengine] Re-entrancy / deadlock bug with scheduler?

[Pol]

Hi,

Our app settings are as follow:
- Python + HRD
- Max Idle Instances: ( 2 )
- Min Pending Latency: ( 100ms )
As of right now, there are 3 instances alive.

Without going too much into details, we have GAE integrated with EC2
on which we run remote image processing tools. The tools are called
directly using HTTP GETs from GAE and they returned their results as
JSON (with gzip content encoding).

There are currently 3 tasks in the processing queue on GAE
continuously failing: the urlfetch() calls to the EC2 tool reach the
10 seconds timeout and bail. What doesn't make sense is that calling
the EC2 tool directly using curl from random machines succeeds in less
than 1 second.

But here's the trick: under certain circumstances, the EC2 tool will
call back to GAE (HEAD request that does a single db.get()) to check
if the image has already been processed and this happens for these 3
stuck tasks.

If calling the EC2 tool from the command line and curl, we have the
normal behavior:
- EC2 tool retrieves image from arbitrary URL and computes a hash
- EC2 tool does a HEAD call to GAE passing this hash to see if image
has been already processed
  - If yes, return empty JSON
  - If no, process image and return full JSON
This takes about 1 second.

The exact same call done from GAE produces this behavior:
- EC2 tool retrieves image from arbitrary URL and computes a hash
- EC2 tool does a HEAD call to GAE passing this hash to see if image
has been already processed
  - HEAD call hangs  --- RE-ENTRANCY / DEADLOCK BUG in GAE
  - urlfetch() from GAE to EC2 reaches 10 seconds deadline and
aborts  -- BREAKS DEADLOCK
  - HEAD call finally returns
- EC2 tool completes normally

GAE logs confirm the bug:

HEAD call from EC2 / curl origina
2011-09-05 10:19:52.502 /api/has_backing?
bid=90e794f348ac76520076f5d02bc67c87c8a9185b8d36affe8377e73fe4820703
200 368ms 48cpu_ms 8api_cpu_ms 0kb Everpix-Processor

HEAD call from EC2 / GAE origin
2011-09-05 10:20:44.670 /api/has_backing?
bid=90e794f348ac76520076f5d02bc67c87c8a9185b8d36affe8377e73fe4820703
200 9712ms 48cpu_ms 8api_cpu_ms 0kb Everpix-Processor
2011-09-05 10:20:44.547 /task/import_photo 500 10348ms 28cpu_ms
8api_cpu_ms 0kb AppEngine-Google; (+http://code.google.com/appengine)
(see how the HEAD /api/has_backing call hangs for almost 10 seconds
and only returns *after* /task/import_photo and its urlfetch() call to
EC2 has aborted)

And finally, AppStats confirms that it's not the head() Python
execution itself that's hanging:

(1) 2011-09-05 09:16:06.843 HEAD /api/has_backing?
bid=3bc4aeb08e01d3ba4bfab3282d2a198984a4fc1fab2ad9d1e8a39ee3cddd14da
200 real=227ms cpu=24ms api=8ms overhead=0ms (1 RPC)
(2) 2011-09-05 09:15:56.422 POST /task/import_photo 500 real=10002ms
cpu=33ms api=8ms overhead=0ms (3 RPCs)
(3) 2011-09-05 09:15:49.328 HEAD /api/has_backing?
bid=90e794f348ac76520076f5d02bc67c87c8a9185b8d36affe8377e73fe4820703
200 real=297ms cpu=21ms api=8ms overhead=0ms (1 RPC)


This issue is currently 100% reproducible.

- Pol

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] How to configure queue.yaml if you only care about max_concurrent_requests?

[Pol]

Hi,

I just want tasks in the queue to be processed with as little latency
as possible, but have never more than N concurrent ones. So what
values do I use here? There are all short tasks that should take no
more than 1 second in the very worst case.

  rate: ?
  bucket_size: ?
  max_concurrent_requests: N

Thanks,

- Pol

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.

[Pol]

Reviving this thread... GAE team: can we get an official comment on
this really important issue? Thx

On Sep 2, 1:14 pm, Pol i...@pol-online.net wrote:
 Our app has been on HRD since the beginning and indeed, the past few
 days for the time ever, I saw some timeouts. To be clear, not timeouts
 due to contention which we get as expected now and then on
 transactions, but timeout on the db itself.

 Our usage patterns haven't changed though, so it would expect it to be
 on GAE's end.

 On Sep 2, 10:02 am, JH ja...@mhztech.com wrote:







  I agree on both fronts.

  Yes, M/S was a JOKE.  Once moving to HRD I have not seen a datastore
  timeout at all.
  Until earlier this week I saw a small series of them.  I really hope
  we are not going back to what we are all used to on M/S.  The old M/S
  problem days were always written off as well, it's good enough for a
  free service.  Now this is hardly free service.  In fact it is top
  dollar service, so I'd hope that datastore issue DO NOT RETURN.

  On Sep 2, 11:10 am, Robert Kluin robert.kl...@gmail.com wrote:

   I think the point of this post is the trend.  At one time master-slave
   also performed quite well, then we'd see trends where latency
   increased until a blowup.  These graphs show a similar pattern to
   that: increasing latencies over time.  We would like to know that: 1)
   the trend won't continue, 2) we won't start seeing massive blowups.

   As for the SLA, see the exclusions section.  SLAs are fairly worthless
   in my honest opinion anyway, so this probably isn't really something
   to worry about though.
    http://code.google.com/appengine/sla.html

   Robert

   On Fri, Sep 2, 2011 at 10:47, Vivek Puri v...@vivekpuri.com wrote:
I will disagree with this post. We were on MS earlier and now on HRD.
While on MS, we used to have datastore timeouts all the time and it
was driving me insane looking at 20-30k error emails come in. After
the move to HRD, everything is stable and there are no datastore
timeouts anymore. I am happy with the move.

On Sep 2, 11:15 am, Pol i...@pol-online.net wrote:
This is really scary. I'm not so obsessed about the price increase as
this is something you can control, but if the HRD starts becoming
slower and slower, there's absolutely nothing you can do. According to
these official graphs, it's literally 2X slower than 1 year ago. I'm
OK with training some speed against increase reliability, the promise
of HRD, but at some point, you need to draw a line: what tells us it's
not going to take 500ms to do basic queries a year from now?

App Engine team: what's your explanation for this? Are you going to
put clauses in the SLA to guarantee max average get / put / query
latencies? Seems to me that if you want to go enterprise-focus
that's needed.

On Sep 2, 6:23 am, Mike Wesner mbwes...@gmail.com wrote:

 Yes, I agree that it won't (can't) get as bad as m/s did get because
 HRD is awesome.  My concern is mostly over how they provision and 
 that
 the latency is that much higher than it was.  Why does the load have
 such a big impact on latency of datastore query?

 We keep very detailed statistics about the latency of our services 
 and
 its clear by our average latencies that it has creeped up over time.
 It is roughly 200ms higher than it was when we first moved over to
 HRD.

 Mike

 On Sep 2, 8:16 am, Tim Hoffman zutes...@gmail.com wrote:

  Hi

  Actually I would disagree on the M/S front, all of my apps are on 
  M/S and
  even despite the recent issues
  M/S is way more stable than it was in 2008 and 2009, I mean WAY 
  MORE STABLE
  ;-)

  Thats not to say more growing pains won't be felt.  My guess is 
  with a
  supposed imminent exodus of apps, we may say some
  performance improvements across the board ;-)

  Rgds

  Tim

--
You received this message because you are subscribed to the Google 
Groups Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group 
athttp://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: how to reindex a property that once set to indexed = false?

[Pol]

Talking abou this, what do you think of using bcrypt.hashpw(password,
bcrypt.gensalt())? I've read in a few places it was supposed to be a
good solution, but I discovered this morning that the AppEngine
version, having to be pure Python, changes the default log_round for
salt generation from 1024 to 1, otherwise it takes way too long.

In practice, what does this mean for security?

BTW: Since this is so important, you guys should be a Google approved
password hashing function as part of GAE :)

On Sep 5, 5:13 pm, Nick Johnson nickjohn...@google.com wrote:
 2011/9/6 saintthor saintt...@gmail.com

  hash is enough for me. my site is not an e-bank.

 This should not matter. If your password database is compromised, the risk
 is not yours, it's your users'. Many users reuse passwords between sites,
 and if your site provides an easy avenue to determining what those passwords
 are, your users are vulnerable elsewhere, on better secured sites.

 There is no good reason not to use a proper salting and password stretching
 scheme; my site is not important enough is not an excuse.

 -Nick Johnson









  On 9月5日, 下午3时58分, Nick Johnson nickjohn...@google.com wrote:
   On Mon, Sep 5, 2011 at 4:42 PM, saintthor saintt...@gmail.com wrote:
the password was hashed.

i think to query name and password together may take less db ops if
password is wrong. isn't it?

   The number of operations is the same; fewer entities would be returned.
  In
   return, though, you're incurring an extra index entry for every record.
  You
   _should_ be salting your passwords (and preferably using RCrypt, SCrypt
  or
   PBKDF2), though, in which case you can't do an index lookup for the
  password
   anyway.

   -Nick Johnson

On 9月5日, 下午1时43分, Nick Johnson nickjohn...@google.com wrote:
 Also, you don't need to index the password field - just fetch the
  user,
then
 check the password. I sincerely hope you're not storing the password
  in
the
 clear, though!

 -Nick

 On Mon, Sep 5, 2011 at 3:02 PM, Simon Knott knott.si...@gmail.com
wrote:
  Hi,

  You need to retrieve all objects that were written in this period
  out
of
  the datastore and re-put them - single-property indexes are only
written on
  putting the entity.  If you can't work out which entities were
  written
in
  this period, then you will need to retrieve all entities and re-put
them.

  It should be noted that query indexes, which use these
  single-property
  indexes in the background, are re-generated on deployment.

  Cheers,
  Simon

  --
  You received this message because you are subscribed to the Google
Groups
  Google App Engine group.
  To view this discussion on the web visit
 https://groups.google.com/d/msg/google-appengine/-/VYJDjjca21MJ.

  To post to this group, send email to
  google-appengine@googlegroups.com
.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

 --
 Nick Johnson, Developer Programs Engineer, App Engine

--
You received this message because you are subscribed to the Google
  Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
  .
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
   http://groups.google.com/group/google-appengine?hl=en.

   --
   Nick Johnson, Developer Programs Engineer, App Engine

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

 --
 Nick Johnson, Developer Programs Engineer, App Engine

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.

[Pol]

This is really scary. I'm not so obsessed about the price increase as
this is something you can control, but if the HRD starts becoming
slower and slower, there's absolutely nothing you can do. According to
these official graphs, it's literally 2X slower than 1 year ago. I'm
OK with training some speed against increase reliability, the promise
of HRD, but at some point, you need to draw a line: what tells us it's
not going to take 500ms to do basic queries a year from now?

App Engine team: what's your explanation for this? Are you going to
put clauses in the SLA to guarantee max average get / put / query
latencies? Seems to me that if you want to go enterprise-focus
that's needed.

On Sep 2, 6:23 am, Mike Wesner mbwes...@gmail.com wrote:
 Yes, I agree that it won't (can't) get as bad as m/s did get because
 HRD is awesome.  My concern is mostly over how they provision and that
 the latency is that much higher than it was.  Why does the load have
 such a big impact on latency of datastore query?

 We keep very detailed statistics about the latency of our services and
 its clear by our average latencies that it has creeped up over time.
 It is roughly 200ms higher than it was when we first moved over to
 HRD.

 Mike

 On Sep 2, 8:16 am, Tim Hoffman zutes...@gmail.com wrote:







  Hi

  Actually I would disagree on the M/S front, all of my apps are on M/S and
  even despite the recent issues
  M/S is way more stable than it was in 2008 and 2009, I mean WAY MORE STABLE
  ;-)

  Thats not to say more growing pains won't be felt.  My guess is with a
  supposed imminent exodus of apps, we may say some
  performance improvements across the board ;-)

  Rgds

  Tim

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.

[Pol]

Our app has been on HRD since the beginning and indeed, the past few
days for the time ever, I saw some timeouts. To be clear, not timeouts
due to contention which we get as expected now and then on
transactions, but timeout on the db itself.

Our usage patterns haven't changed though, so it would expect it to be
on GAE's end.

On Sep 2, 10:02 am, JH ja...@mhztech.com wrote:
 I agree on both fronts.

 Yes, M/S was a JOKE.  Once moving to HRD I have not seen a datastore
 timeout at all.
 Until earlier this week I saw a small series of them.  I really hope
 we are not going back to what we are all used to on M/S.  The old M/S
 problem days were always written off as well, it's good enough for a
 free service.  Now this is hardly free service.  In fact it is top
 dollar service, so I'd hope that datastore issue DO NOT RETURN.

 On Sep 2, 11:10 am, Robert Kluin robert.kl...@gmail.com wrote:







  I think the point of this post is the trend.  At one time master-slave
  also performed quite well, then we'd see trends where latency
  increased until a blowup.  These graphs show a similar pattern to
  that: increasing latencies over time.  We would like to know that: 1)
  the trend won't continue, 2) we won't start seeing massive blowups.

  As for the SLA, see the exclusions section.  SLAs are fairly worthless
  in my honest opinion anyway, so this probably isn't really something
  to worry about though.
   http://code.google.com/appengine/sla.html

  Robert

  On Fri, Sep 2, 2011 at 10:47, Vivek Puri v...@vivekpuri.com wrote:
   I will disagree with this post. We were on MS earlier and now on HRD.
   While on MS, we used to have datastore timeouts all the time and it
   was driving me insane looking at 20-30k error emails come in. After
   the move to HRD, everything is stable and there are no datastore
   timeouts anymore. I am happy with the move.

   On Sep 2, 11:15 am, Pol i...@pol-online.net wrote:
   This is really scary. I'm not so obsessed about the price increase as
   this is something you can control, but if the HRD starts becoming
   slower and slower, there's absolutely nothing you can do. According to
   these official graphs, it's literally 2X slower than 1 year ago. I'm
   OK with training some speed against increase reliability, the promise
   of HRD, but at some point, you need to draw a line: what tells us it's
   not going to take 500ms to do basic queries a year from now?

   App Engine team: what's your explanation for this? Are you going to
   put clauses in the SLA to guarantee max average get / put / query
   latencies? Seems to me that if you want to go enterprise-focus
   that's needed.

   On Sep 2, 6:23 am, Mike Wesner mbwes...@gmail.com wrote:

Yes, I agree that it won't (can't) get as bad as m/s did get because
HRD is awesome.  My concern is mostly over how they provision and that
the latency is that much higher than it was.  Why does the load have
such a big impact on latency of datastore query?

We keep very detailed statistics about the latency of our services and
its clear by our average latencies that it has creeped up over time.
It is roughly 200ms higher than it was when we first moved over to
HRD.

Mike

On Sep 2, 8:16 am, Tim Hoffman zutes...@gmail.com wrote:

 Hi

 Actually I would disagree on the M/S front, all of my apps are on 
 M/S and
 even despite the recent issues
 M/S is way more stable than it was in 2008 and 2009, I mean WAY MORE 
 STABLE
 ;-)

 Thats not to say more growing pains won't be felt.  My guess is with 
 a
 supposed imminent exodus of apps, we may say some
 performance improvements across the board ;-)

 Rgds

 Tim

   --
   You received this message because you are subscribed to the Google Groups 
   Google App Engine group.
   To post to this group, send email to google-appengine@googlegroups.com.
   To unsubscribe from this group, send email to 
   google-appengine+unsubscr...@googlegroups.com.
   For more options, visit this group 
   athttp://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Problems Deploying App - 500 Internal server Error

[Pol]

From my experience, when these happens they don't happen once, but for
a continuous period of time say 15mn. So they appear to be
reproducible 100% during this period. If you provide an direct email
address people can use next time it happens, then the affected
developer can provide the app id and even attempt to re-deploy several
times so you that can observe things. And hopefully it can finally be
tracked down :)

On Jul 29, 4:49 pm, Ikai Lan (Google) ika...@google.com wrote:
 Can you give it a try now? I'm trying to figure out if these deploy errors
 were ephemeral or if these failures can be reproduced deterministically.

 --
 Ikai Lan
 Developer Programs Engineer, Google App Engine
 plus.ikailan.com | twitter.com/ikai

 On Thu, Jul 28, 2011 at 1:00 PM, Chronos Phaenon Eosphoros 







 cpeospho...@gmail.com wrote:
  I'm receiving this error message when trying to deploy my app.

  I'm kinda new around here, so it may be some mistake I've made, but I
  didn't do anything that rings a bell to me.

  Thanks,

  Unable to update app: Error posting to URL:
 https://appengine.google.com/api/appversion/clonefiles?app_id=inscric...
  500 Internal Server Error

  htmlhead
  meta http-equiv=content-type content=text/html;charset=utf-8
  title500 Server Error/title
  /head
  body text=#00 bgcolor=#ff
  h1Error: Server Error/h1
  h2The server encountered an error and could not complete your
  request.pIf the problem persists, please A HREF=
 http://code.google.com/appengine/community.html;report/A your problem
  and mention this error message and the query that caused it./h2
  h2/h2
  /body/html

  See the deployment console for more details
  Unable to update app: Error posting to URL:
 https://appengine.google.com/api/appversion/clonefiles?app_id=inscric...
  500 Internal Server Error

  htmlhead
  meta http-equiv=content-type content=text/html;charset=utf-8
  title500 Server Error/title
  /head
  body text=#00 bgcolor=#ff
  h1Error: Server Error/h1
  h2The server encountered an error and could not complete your
  request.pIf the problem persists, please A HREF=
 http://code.google.com/appengine/community.html;report/A your problem
  and mention this error message and the query that caused it./h2
  h2/h2
  /body/html

   --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To view this discussion on the web visit
 https://groups.google.com/d/msg/google-appengine/-/hiWxAqlrDpoJ.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Really high request response time for trivial Python handlers (not explained by App Stats)

[Pol]

I didn't look for that info and the logs are gone by now. I'll make
sure to check time.

However, I didn't play with the scheduler settings and I highly doubt
that's relevant as there were *no* users of the app besides me at this
time. There are also 3 resident instances.

On Jul 31, 8:18 am, Robert Kluin robert.kl...@gmail.com wrote:
 Sounds like scheduler issues. Do you see a pending ms in the logs (expand
 the entries)?  Have you adjusted the scheduler knobs under app settings in
 your app's dashboard?

 Robert

 On Jul 29, 2011 12:02 PM, Pol i...@pol-online.net wrote:









  Hi,

  All I'm im doing is reloading a web page that does a couple AJAX
  requests to my app engine app (Python) and I'm often observing insane
  latency:

  2011-07-29 08:52:20.464 /api/user_info 200 10967ms 26cpu_ms 0kb
  Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:5.0.1) Gecko/20100101
  Firefox/5.0.1

  10 seconds for a trivial OPTIONS request?

  class APIHandler(webapp.RequestHandler):

   def initialize(self, request, response):
     super(APIHandler, self).initialize(request, response)
     del self.response.headers['Content-Type']

   def options(self):
     self.response.headers['Access-Control-Allow-Origin'] = '*'
     self.response.headers['Access-Control-Allow-Methods'] = 'POST,
  GET, OPTIONS'
     self.response.headers['Access-Control-Allow-Headers'] = 'X-
  Requested-With, If-Modified-Since, Content-Type, Cache-Control' + ', '
  + VERSION_HEADER + ', ' + AUTHTOKEN_HEADER

  Note that I have 3 resident instances, nobody is using my app besides
  me right now, and no, the log does not report the start of a new
  instance when these very long requests happen.

  What's more incomprehensible is that I have App Stats running on the
  server and it reports really low-latency:

  (1) 2011-07-29 07:48:14.041 GET /api/status 200 real=5ms cpu=46ms
  api=0ms overhead=0ms (1 RPC)
  (2) 2011-07-29 07:48:13.427 OPTIONS /api/status 200 real=0ms
  cpu=1003ms api=0ms overhead=0ms (0 RPCs)
  (3) 2011-07-29 07:48:12.537 GET /api/user_info 401 real=23ms
  cpu=27ms api=12ms overhead=0ms (2 RPCs)
  (4) 2011-07-29 07:48:12.025 OPTIONS /api/user_info 200 real=0ms
  cpu=12ms api=0ms overhead=0ms (0 RPCs)

  Here are the same 4 requests in the dashboard log:

  2011-07-29 08:48:14.053 /api/status 200 133ms 26cpu_ms 0kb Mozilla/5.0
  (Macintosh; Intel Mac OS X 10.6; rv:5.0.1) Gecko/20100101 Firefox/
  5.0.1
  2011-07-29 08:48:13.434 /api/status 200 401ms 213cpu_ms 0kb Mozilla/
  5.0 (Macintosh; Intel Mac OS X 10.6; rv:5.0.1) Gecko/20100101 Firefox/
  5.0.1
  2011-07-29 08:48:12.572 /api/user_info 401 190ms 67cpu_ms 12api_cpu_ms
  0kb Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:5.0.1) Gecko/
  20100101 Firefox/5.0.1
  2011-07-29 08:48:12.068 /api/user_info 200 2446ms 35cpu_ms 0kb Mozilla/
  5.0 (Macintosh; Intel Mac OS X 10.6; rv:5.0.1) Gecko/20100101 Firefox/
  5.0.1

  As you can't see 5~23ms turned into 133~2446ms. What could possibly be
  happening? The latency clearly comes from outside my handler's code.

  Thanks for any help.

  - Pol

  --
  You received this message because you are subscribed to the Google Groups

 Google App Engine group. To post to this group, send email to 
 google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to

 google-appengine+unsubscr...@googlegroups.com. For more options, visit this 
 group at

 http://groups.google.com/group/google-appengine?hl=en.









-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: TransactionFailedError: Is the transaction committed or not?

[Pol]

Hi Jose,

 2) There is no fake-real-transaction, for example, one of the other uncommon
 exception that someone could get in the commit() would be Timeout exception.
 With any distributed system, unless you are willing to wait for an answer
 forever, there is always the possibility that you will get a timeout before
 the remote system responds with the result of the operation you requested
 (this is true in all databases), what GAE warranty is that the transaction
 either commits successfully or it does not, so if you implement some retry
 logic in transactions, having it idempotent will warranty you don't do the
 transaction twice.

Just to be sure, are you saying that for a timeout exception the
transaction has actually been committed or not? In other words, should
a Timeout exception be dealt with the same way a
TransactionFailedError is?

I observe Timeout exceptions now and then where there is too much
contention on the same entity group. But I've only seen a
TransactionFailedError once in 2 months.

 But as I said before, this exceptions are uncommon and it would be good to
 know what are you doing inside the transaction that is causing those
 transactions. Timeout exception could be related to master slaves issues (in
 a M/S app) this could go away by migrating to High replication datastore.

I'm using the HR datastore already, but I'm not surprised with the
Timeouts as in my case there can sometimes, but rarely, be close to 10
transactions / seconds on the same entity group.

 Hope this helps clarifies some points.

Thanks much for your detailed answers indeed!

- Pol

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Really high request response time for trivial Python handlers (not explained by App Stats)

[Pol]

Hi,

All I'm im doing is reloading a web page that does a couple AJAX
requests to my app engine app (Python) and I'm often observing insane
latency:

2011-07-29 08:52:20.464 /api/user_info 200 10967ms 26cpu_ms 0kb
Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:5.0.1) Gecko/20100101
Firefox/5.0.1

10 seconds for a trivial OPTIONS request?

class APIHandler(webapp.RequestHandler):

  def initialize(self, request, response):
super(APIHandler, self).initialize(request, response)
del self.response.headers['Content-Type']

  def options(self):
self.response.headers['Access-Control-Allow-Origin'] = '*'
self.response.headers['Access-Control-Allow-Methods'] = 'POST,
GET, OPTIONS'
self.response.headers['Access-Control-Allow-Headers'] = 'X-
Requested-With, If-Modified-Since, Content-Type, Cache-Control' + ', '
+ VERSION_HEADER + ', ' + AUTHTOKEN_HEADER

Note that I have 3 resident instances, nobody is using my app besides
me right now, and no, the log does not report the start of a new
instance when these very long requests happen.

What's more incomprehensible is that I have App Stats running on the
server and it reports really low-latency:

(1) 2011-07-29 07:48:14.041 GET /api/status 200 real=5ms cpu=46ms
api=0ms overhead=0ms (1 RPC)
(2) 2011-07-29 07:48:13.427 OPTIONS /api/status 200 real=0ms
cpu=1003ms api=0ms overhead=0ms (0 RPCs)
(3) 2011-07-29 07:48:12.537 GET /api/user_info 401 real=23ms
cpu=27ms api=12ms overhead=0ms (2 RPCs)
(4) 2011-07-29 07:48:12.025 OPTIONS /api/user_info 200 real=0ms
cpu=12ms api=0ms overhead=0ms (0 RPCs)

Here are the same 4 requests in the dashboard log:

2011-07-29 08:48:14.053 /api/status 200 133ms 26cpu_ms 0kb Mozilla/5.0
(Macintosh; Intel Mac OS X 10.6; rv:5.0.1) Gecko/20100101 Firefox/
5.0.1
2011-07-29 08:48:13.434 /api/status 200 401ms 213cpu_ms 0kb Mozilla/
5.0 (Macintosh; Intel Mac OS X 10.6; rv:5.0.1) Gecko/20100101 Firefox/
5.0.1
2011-07-29 08:48:12.572 /api/user_info 401 190ms 67cpu_ms 12api_cpu_ms
0kb Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:5.0.1) Gecko/
20100101 Firefox/5.0.1
2011-07-29 08:48:12.068 /api/user_info 200 2446ms 35cpu_ms 0kb Mozilla/
5.0 (Macintosh; Intel Mac OS X 10.6; rv:5.0.1) Gecko/20100101 Firefox/
5.0.1

As you can't see 5~23ms turned into 133~2446ms. What could possibly be
happening? The latency clearly comes from outside my handler's code.

Thanks for any help.

- Pol

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: TransactionFailedError: Is the transaction committed or not?

[Pol]

On Jul 26, 9:40 pm, Jose Montes de Oca jfmontesde...@google.com
wrote:
 Hi Pol,

 What this meas is that even if a transaction throws an exception this does
 not means the transaction failed, thats why you need to make your datastore
 transaction idempotent. So if you retry a transaction because it throws an
 exception, your transaction needs to check if the last
 transaction committed successfully or not.

I don't understand how GAE could not report if the transaction
succeeded or not. Clearly this stuff is deterministic.

You're saying I have to maintain my own journal of transactions to
check against. But then that journal is itself would have to be
updated through transactions, so it needs to be journaled as well and
so on... it's completely unsolvable. How would implement a bank on
such a system? How could you ever 100% guarantee a user's transactions
are all in there or that the account's balance is correct?

Let's even look at a simple example: if you were to implement a simple
counter, how do you make that idempotent?

txn()
  counter = Counter.get(...)
  counter.value += 1
  counter.put()

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: TransactionFailedError: Is the transaction committed or not?

[Pol]

I see two big problems here:

1) There appear to be absolutely no record, say in the dashboard, of
failed transactions that eventually succeeded or really failed. For
some type of apps, I could live with 1 in a million transaction
failure that may or may not be eventually successful, but I need to
know which entity got in a degenerated state.

2) The benefit of the current approach of fake-real-transactions is
completely unknown: does it make GAE massively faster or more reliable
or something?

For instance, if you choose non-redundant storage in Amazon S3:
1) you get notifications for lost objects
2) it's cheaper

On Jul 28, 4:02 pm, Joshua Smith joshuaesm...@charter.net wrote:
 The problem is that google transactions can report an exception, and then go 
 ahead and succeed anyway.

 So the docs recommend that you only write idempotent transactions, which is a 
 completely silly suggestion.  I've yet to see a single example of how one 
 might write an idempotent transaction.  (Unless, I suppose, you create a 
 separate child model in the database which is parented by the object you are 
 transacting on, and then you query the list of children every time you retry 
 your transaction to see if its already in there, but that won't scale.)

 I contend that a DB that cannot tell you reliably whether a transaction 
 succeeded for failed does not support transactions.

 GAE can essentially report 3 possible results from a transaction:
 - Definitely succeeded
 - Definitely failed
 - Beats me

 I contend that third possible result makes it impossible to write software 
 that relies on transactions.

 Therefore, GAE doesn't support transactions.

 On Jul 27, 2011, at 8:58 PM, Tim Hoffman wrote:







  If you always get modify put within a transaction how would it be 
  unreliable?

  Rgds

  Tim

  --
  You received this message because you are subscribed to the Google Groups 
  Google App Engine group.
  To view this discussion on the web 
  visithttps://groups.google.com/d/msg/google-appengine/-/mP_8kv_-LlMJ.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to 
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group 
  athttp://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Can sender address be spoofed in XMPP?

[Pol]

Hi,

I'm not very familiar with the XMPP protocol but from the GAE Python
docs, I couldn't figure the answer to this question: if a message is
received by App Engine claiming to come from a server at:
- a google apps domain,
- a gmail.com,
- a random domain

Can any of these be spoofed like the HTTP headers for web requests?

- Pol

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] 500 deployment errors, here we go again

[Pol]

Anyone else observing? It's been failing for me for the last 20mn.

Google: is the SLA for the entire app engine stack (including
deployment) or just its top (the running app)?

-

Application: XXX; version: 953 (was: 1)
Host: appengine.google.com

Starting update of app: XXX, version: 953
Scanning files on local disk.
Cloning 1 static file.
Cloning 124 application files.
Cloned 100 files.
Rolling back the update.
Error 500: --- begin server output ---

htmlhead
meta http-equiv=content-type content=text/html;charset=utf-8
title500 Server Error/title
/head
body text=#00 bgcolor=#ff
h1Error: Server Error/h1
h2The server encountered an error and could not complete your
request.pIf the problem persists, please A HREF=http://
code.google.com/appengine/community.htmlreport/A your problem and
mention this error message and the query that caused it./h2
h2/h2
/body/html
--- end server output ---

- Pol

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: 500 deployment errors, here we go again

[Pol]

Appears to be working again, indeed.

On Jul 28, 6:47 pm, Chris Copeland ch...@cope360.com wrote:
 +1

 Been happening the last 15 minutes or so.







 On Thu, Jul 28, 2011 at 11:38 AM, Pol i...@pol-online.net wrote:
  Anyone else observing? It's been failing for me for the last 20mn.

  Google: is the SLA for the entire app engine stack (including
  deployment) or just its top (the running app)?

  -

  Application: XXX; version: 953 (was: 1)
  Host: appengine.google.com

  Starting update of app: XXX, version: 953
  Scanning files on local disk.
  Cloning 1 static file.
  Cloning 124 application files.
  Cloned 100 files.
  Rolling back the update.
  Error 500: --- begin server output ---

  htmlhead
  meta http-equiv=content-type content=text/html;charset=utf-8
  title500 Server Error/title
  /head
  body text=#00 bgcolor=#ff
  h1Error: Server Error/h1
  h2The server encountered an error and could not complete your
  request.pIf the problem persists, please A HREF=http://
  code.google.com/appengine/community.htmlreport/A your problem and
  mention this error message and the query that caused it./h2
  h2/h2
  /body/html
  --- end server output ---

  - Pol

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: HR database, ancestor queries and data consistency: docs not 100% clear

[Pol]

On Jul 26, 6:15 am, Nick Johnson (Google) nick.john...@google.com
wrote:

  I understand that, but my problem is with the indexes: even in the
  scope of HR + transactions, the doc is confusing regarding wether or
  not there can be a race-condition on indexes update.

  For instance:
  - transaction A modifies entity X and commits
  - then transaction B happens almost concurrently:
   - at this point, using db.get() X will return the latest X no matter
  what
   - however, using db.GqlQuery() with a predicate that matches X *may
  not* return X because the indexes may not be done updating yet (the A -
   B thing in the doc)

  So is this race-condition possible or not? No matter what, you might
  wanna rewrite a couple sentences in this article :)

 Again, this depends on if you're using an ancestor query. Queries that
 include ancestor filters are strongly consistent, queries that don't aren't.

Since I'm inside a transactions, I am indeed using ancestor queries.
So I'm going to interpret your answer as with HR and ancestor
queries, there can be no race-conditions at all due to index updates
no matter what the doc says about concurrent transactions and small
windows. :)

For the record, I did end up finding a bug in my code that explained
the fact sometimes concurrent transactions with ancestor queries were
apparently returning stalled results.

- Pol

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: HR database, ancestor queries and data consistency: docs not 100% clear

[Pol]

On Jul 25, 7:30 am, Nick Johnson (Google) nick.john...@google.com
wrote:

 The first part specifically refers to ancestor queries. Ancestor queries
 always operate on the latest copy of the data, as they're restricted to an
 entity group; regular queries don't have that guarantee.

I understand that, but my problem is with the indexes: even in the
scope of HR + transactions, the doc is confusing regarding wether or
not there can be a race-condition on indexes update.

For instance:
- transaction A modifies entity X and commits
- then transaction B happens almost concurrently:
  - at this point, using db.get() X will return the latest X no matter
what
  - however, using db.GqlQuery() with a predicate that matches X *may
not* return X because the indexes may not be done updating yet (the A -
 B thing in the doc)

So is this race-condition possible or not? No matter what, you might
wanna rewrite a couple sentences in this article :)

  My understanding of this article is therefore the following (assuming
  an HR datastore and that we are inside a transaction):

  1) Using db.get() / db.put() on an entity always executes on its
  latest version: no race conditions possible, period.
  - Observations in my app appears to confirm that behavior.

 Correct, with the usual caveat about transactionality (if you're not doing
 this inside a transaction, you have a potential race condition).

Yes, I was assuming HR datastore + transaction (see above).

  2) Using db.GqlQuery() / db.put() on entities in the same entity group
  might still be subject to race-conditions: for instance, a previously
  *committed* transaction modified the entities that would be returned
  by the query in the later transactions, but the datastore indexes are
  not up-to-date yet (or something like that), so the query doesn't see
  the updated entities - even if calling db.get() directly on them would
  return the versions that match the query.
  - I've reviewed my code carefully and I think I'm seeing such race
  conditions in my app.

 Correct - but again, any update outside a transaction always has possible
 race conditions.

Same here: I was also assuming HR datastore + transaction + ancestor
queries (by definition since inside a transaction).

So is my code wrong or am I observing some very rare but expected race
conditions?

Thanks

- Pol

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] TransactionFailedError: Is the transaction committed or not?

[Pol]

Hi,

The exception says: TransactionFailedError: The transaction could not
be committed. Please try again.

The doc at 
http://code.google.com/appengine/docs/python/datastore/transactions.html
says: You can receive Timeout, TransactionFailedError, or
InternalError exceptions in cases where transactions have been
committed and eventually will be applied successfully.

So if you get TransactionFailedError, to you need to execute the
transaction again or will it automatically be applied later?

- Pol

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Is anyone seeing deployment 500 server error now?

[Pol]

Same for me: Python SDK deployment from Linux box fails with 500
error:

Scanning files on local disk.
Cloning 1 static file.
Cloning 122 application files.
Cloned 100 files.
Compilation starting.
Compilation completed.
Starting deployment.
Checking if deployment succeeded.
Will check again in 1 seconds.
Checking if deployment succeeded.
Will check again in 2 seconds.
Checking if deployment succeeded.
Deployment successful.
Checking if updated app version is serving.
Completed update of app: XXX, version: 823
Uploading index definitions.
Uploading cron entries.
Uploading task queue entries.
Error 500: --- begin server output ---

htmlhead
meta http-equiv=content-type content=text/html;charset=utf-8
title500 Server Error/title
/head
body text=#00 bgcolor=#ff
h1Error: Server Error/h1
h2The server encountered an error and could not complete your
request.pIf the problem persists, please A HREF=http://
code.google.com/appengine/community.htmlreport/A your problem and
mention this error message and the query that caused it./h2
h2/h2
/body/html
--- end server output ---

- Pol

On Jul 17, 4:28 am, Tom Fishman tom.fish...@dishcrunch.com wrote:
 App deployment has been failing for half an hour or so:

 java.io.IOException: Error posting to 
 URL:https://appengine.google.com/api/appversion/clonefiles?app_id=xxxver...
 500 Internal Server Error

 htmlhead
 meta http-equiv=content-type content=text/html;charset=utf-8
 title500 Server Error/title
 /head
 body text=#00 bgcolor=#ff
 h1Error: Server Error/h1
 h2The server encountered an error and could not complete your
 request.pIf the problem persists, please A
 HREF=http://code.google.com/appengine/community.html;report/A your
 problem and mention this error message and the query that caused it./h2
 h2/h2
 /body/html

 I report it because the error suggest so. It is the same as the
 downtime/maintenance error.

 -T

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Model.get_or_insert() behavior on high-replication datastore

[Pol]

Hi,

The documentation for Model.get_or_insert() indicates this is an
atomic operation but the documentation for the high-replication
datastore says that it is eventually consistent unless you use entity
groups.

So it is correct that Model.get_or_insert() is not atomic on high-
replication datastore if the entity key has no parent?

- Pol

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] What happens to still pending async operations when the request completes (in Python)?

[Pol]

Hi,

I was wondering, if you start an async urlfetch (or even an async db
operation) but don't wait for it to complete before returning from the
handler, what happens? Is the async operation cancelled mid-course or
does it complete normally, except no callback is called to indicate
success or failure?

I'm specifically asking because I have to ping an external URL at the
end of some of the handlers but I don't care about the result of
urlfetch (as long as the HTTP GET request was sent to the remote
server that's fine). Being able to have a fire  forget approach on
this request would save ~100ms.

Thanks,

- Pol

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: About expando properties

[Pol]

Very interesting, thanks for sharing.

If you just need a simple dictionary-like storage, why not just repr()
and eval() - wouldn't that be even faster?

On Jul 13, 8:05 am, someone1 someo...@gmail.com wrote:
 I just looked into this myself. Best bet is to use a Text or Blob
 property to store in a JSON format:

 http://kovshenin.com/archives/pickle-vs-json-which-is-faster/

 I did many tests with 250-500 expando properties and the results were
 horrendous. From digging around I didn't find a way to disable
 indexing expando properties, and this made writing them very
 expensive.

 On Jul 13, 1:35 am, Pol i...@pol-online.net wrote:







  Hi,

  Regarding expando properties:

  1) Can you prevent them from being indexed (I couldn't find anything
  about this, but I'd like official confirmation)?

  2) What's the practical number of expando properties you shouldn't go
  above? If they are all indexed, then surely that can be quite
  expensive, so you should stay below 100 or something?

  3) If you have to be above the limit defined at #2, what's the best
  alternative? Serializing into a blob property, possibly, but what
  would be the most optimal serialization format?

  Thanks!

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: How to write optimal queries?

[Pol]

It's been a few days since the last message on this thread - just
making sure it doesn't fall through the cracks :)

On Jul 8, 9:06 pm, Pol i...@pol-online.net wrote:
 Thanks for the slides, quite interesting - too bad there was no audio,
 it would have helped even more.

 So still, how are zig-zag joins relevant here? I use the Python SDK
 and composite indexes are added automatically to index.yaml anyway, so
 there should be no zig-zagging or am I missing something?

 Anyway, I hope I'm up-to-speed on the terminology now, so I'm looking
 forward to hear your answers about my previously raised questions :)

 On Jul 8, 12:21 pm, Ikai Lan (Google) ika...@google.com wrote:







  Before I try to answer this question, can you take a look at these slides?
  Hopefully these should clarify why things work the way they work:

 http://www.slideshare.net/ikailan/introducing-the-app-engine-datastore

  Ideally, these will raise new questions about how entity groups, index
  scans, etc work.

  Ikai Lan
  Developer Programs Engineer, Google App Engine
  Blog:http://googleappengine.blogspot.com
  Twitter:http://twitter.com/app_engine
  Reddit:http://www.reddit.com/r/appengine

  On Thu, Jul 7, 2011 at 7:37 PM, Pol i...@pol-online.net wrote:
   I'm a little confused by your answer and the concept of zigzag
   joins. In practice, assuming the result set will end up being the
   same, is any of these 2 queries faster / better and why exactly?

   query = db.GqlQuery(SELECT * FROM Photo WHERE ANCESTOR IS :1 AND
   prime = TRUE ORDER BY timestamp DESC, self.user.key())

   query = db.GqlQuery(SELECT * FROM Photo WHERE ANCESTOR IS :1 AND
   initialized = TRUE AND prime = TRUE ORDER BY timestamp DESC,
   self.user.key())

   Regarding the 2nd set of queries I asked about, all things being
   equal, is it better to do the query with or without the ANCESTOR?
   Intuitively, I would expect the ANCESTOR version to perform faster as
   it would only run on 1 machine / entity group, but is this true?

   In the data model, Photo is a child of User and there 1000's more
   photos than users. Otherwise, I haven't measured performance in the
   app yet, I'd rather rely on some official best practices for now :)

   On Jul 7, 6:25 pm, Ikai Lan (Google) ika...@google.com wrote:
Ancestor queries don't add significant overhead, so I'm not going to
consider that factor.

As far as other queries go, in general fewer indexed mean better
   performance
if you are doing zigzag join. However - if an index contains a small
   number
of results, obviously the query will return faster because there are
   simply
less results. Otherwise, you end up zig zagging across more indexes,
   which
probably will result in slower queries. It really depends on the shape 
of
your data. What have your obvservations been?

Ikai Lan
Developer Programs Engineer, Google App Engine
Blog:http://googleappengine.blogspot.com
Twitter:http://twitter.com/app_engine
Reddit:http://www.reddit.com/r/appengine

On Thu, Jul 7, 2011 at 6:21 PM, Pol i...@pol-online.net wrote:
 Hi,

 Assuming the app runs on an HR database and that the number of indexes
 is not a problem:

 Say that because of the data model, these queries are equivalent (i.e.
 return the exact same results), which one should be used to get the
 best performance?

 query = db.GqlQuery(SELECT * FROM Photo WHERE ANCESTOR IS :1 AND
 prime = TRUE ORDER BY timestamp DESC, self.user.key())

 query = db.GqlQuery(SELECT * FROM Photo WHERE ANCESTOR IS :1 AND
 initialized = TRUE AND prime = TRUE ORDER BY timestamp DESC,
 self.user.key())

 Does the response change if somewhere else in the code, there is also
 this query (because of shared indexes or something)?

 query = db.GqlQuery(SELECT * FROM Photo WHERE ANCESTOR IS :1 AND
 initialized = TRUE ORDER BY timestamp DESC, self.user.key())

 Same question, this time with these 2 other queries:

 query = db.GqlQuery(SELECT * FROM Photo WHERE event = :1 AND prime =
 TRUE AND hidden = FALSE ORDER BY timestamp DESC, event.key())

 query = db.GqlQuery(SELECT * FROM Photo WHERE ANCESTOR IS :1 AND
 event = :2 AND prime = TRUE AND hidden = FALSE ORDER BY timestamp
 DESC, event.key().parent(), event.key())

 I understand the 2nd version is an ancestor query which should return
 consistent results (right?) but in this case, it's ok if the results
 are a bit stalled.

 Thanks!

 --
 You received this message because you are subscribed to the Google
   Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com
   .
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

   --
   You received this message because you

[google-appengine] About expando properties

[Pol]

Hi,

Regarding expando properties:

1) Can you prevent them from being indexed (I couldn't find anything
about this, but I'd like official confirmation)?

2) What's the practical number of expando properties you shouldn't go
above? If they are all indexed, then surely that can be quite
expensive, so you should stay below 100 or something?

3) If you have to be above the limit defined at #2, what's the best
alternative? Serializing into a blob property, possibly, but what
would be the most optimal serialization format?

Thanks!

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: How to write optimal queries?

[Pol]

Thanks for the slides, quite interesting - too bad there was no audio,
it would have helped even more.

So still, how are zig-zag joins relevant here? I use the Python SDK
and composite indexes are added automatically to index.yaml anyway, so
there should be no zig-zagging or am I missing something?

Anyway, I hope I'm up-to-speed on the terminology now, so I'm looking
forward to hear your answers about my previously raised questions :)

On Jul 8, 12:21 pm, Ikai Lan (Google) ika...@google.com wrote:
 Before I try to answer this question, can you take a look at these slides?
 Hopefully these should clarify why things work the way they work:

 http://www.slideshare.net/ikailan/introducing-the-app-engine-datastore

 Ideally, these will raise new questions about how entity groups, index
 scans, etc work.

 Ikai Lan
 Developer Programs Engineer, Google App Engine
 Blog:http://googleappengine.blogspot.com
 Twitter:http://twitter.com/app_engine
 Reddit:http://www.reddit.com/r/appengine







 On Thu, Jul 7, 2011 at 7:37 PM, Pol i...@pol-online.net wrote:
  I'm a little confused by your answer and the concept of zigzag
  joins. In practice, assuming the result set will end up being the
  same, is any of these 2 queries faster / better and why exactly?

  query = db.GqlQuery(SELECT * FROM Photo WHERE ANCESTOR IS :1 AND
  prime = TRUE ORDER BY timestamp DESC, self.user.key())

  query = db.GqlQuery(SELECT * FROM Photo WHERE ANCESTOR IS :1 AND
  initialized = TRUE AND prime = TRUE ORDER BY timestamp DESC,
  self.user.key())

  Regarding the 2nd set of queries I asked about, all things being
  equal, is it better to do the query with or without the ANCESTOR?
  Intuitively, I would expect the ANCESTOR version to perform faster as
  it would only run on 1 machine / entity group, but is this true?

  In the data model, Photo is a child of User and there 1000's more
  photos than users. Otherwise, I haven't measured performance in the
  app yet, I'd rather rely on some official best practices for now :)

  On Jul 7, 6:25 pm, Ikai Lan (Google) ika...@google.com wrote:
   Ancestor queries don't add significant overhead, so I'm not going to
   consider that factor.

   As far as other queries go, in general fewer indexed mean better
  performance
   if you are doing zigzag join. However - if an index contains a small
  number
   of results, obviously the query will return faster because there are
  simply
   less results. Otherwise, you end up zig zagging across more indexes,
  which
   probably will result in slower queries. It really depends on the shape of
   your data. What have your obvservations been?

   Ikai Lan
   Developer Programs Engineer, Google App Engine
   Blog:http://googleappengine.blogspot.com
   Twitter:http://twitter.com/app_engine
   Reddit:http://www.reddit.com/r/appengine

   On Thu, Jul 7, 2011 at 6:21 PM, Pol i...@pol-online.net wrote:
Hi,

Assuming the app runs on an HR database and that the number of indexes
is not a problem:

Say that because of the data model, these queries are equivalent (i.e.
return the exact same results), which one should be used to get the
best performance?

query = db.GqlQuery(SELECT * FROM Photo WHERE ANCESTOR IS :1 AND
prime = TRUE ORDER BY timestamp DESC, self.user.key())

query = db.GqlQuery(SELECT * FROM Photo WHERE ANCESTOR IS :1 AND
initialized = TRUE AND prime = TRUE ORDER BY timestamp DESC,
self.user.key())

Does the response change if somewhere else in the code, there is also
this query (because of shared indexes or something)?

query = db.GqlQuery(SELECT * FROM Photo WHERE ANCESTOR IS :1 AND
initialized = TRUE ORDER BY timestamp DESC, self.user.key())

Same question, this time with these 2 other queries:

query = db.GqlQuery(SELECT * FROM Photo WHERE event = :1 AND prime =
TRUE AND hidden = FALSE ORDER BY timestamp DESC, event.key())

query = db.GqlQuery(SELECT * FROM Photo WHERE ANCESTOR IS :1 AND
event = :2 AND prime = TRUE AND hidden = FALSE ORDER BY timestamp
DESC, event.key().parent(), event.key())

I understand the 2nd version is an ancestor query which should return
consistent results (right?) but in this case, it's ok if the results
are a bit stalled.

Thanks!

--
You received this message because you are subscribed to the Google
  Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
  .
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
   http://groups.google.com/group/google-appengine?hl=en.

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit

[google-appengine] How to write optimal queries?

[Pol]

Hi,

Assuming the app runs on an HR database and that the number of indexes
is not a problem:

Say that because of the data model, these queries are equivalent (i.e.
return the exact same results), which one should be used to get the
best performance?

query = db.GqlQuery(SELECT * FROM Photo WHERE ANCESTOR IS :1 AND
prime = TRUE ORDER BY timestamp DESC, self.user.key())

query = db.GqlQuery(SELECT * FROM Photo WHERE ANCESTOR IS :1 AND
initialized = TRUE AND prime = TRUE ORDER BY timestamp DESC,
self.user.key())

Does the response change if somewhere else in the code, there is also
this query (because of shared indexes or something)?

query = db.GqlQuery(SELECT * FROM Photo WHERE ANCESTOR IS :1 AND
initialized = TRUE ORDER BY timestamp DESC, self.user.key())

Same question, this time with these 2 other queries:

query = db.GqlQuery(SELECT * FROM Photo WHERE event = :1 AND prime =
TRUE AND hidden = FALSE ORDER BY timestamp DESC, event.key())

query = db.GqlQuery(SELECT * FROM Photo WHERE ANCESTOR IS :1 AND
event = :2 AND prime = TRUE AND hidden = FALSE ORDER BY timestamp
DESC, event.key().parent(), event.key())

I understand the 2nd version is an ancestor query which should return
consistent results (right?) but in this case, it's ok if the results
are a bit stalled.

Thanks!

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: How to write optimal queries?

[Pol]

I'm a little confused by your answer and the concept of zigzag
joins. In practice, assuming the result set will end up being the
same, is any of these 2 queries faster / better and why exactly?

query = db.GqlQuery(SELECT * FROM Photo WHERE ANCESTOR IS :1 AND
prime = TRUE ORDER BY timestamp DESC, self.user.key())

query = db.GqlQuery(SELECT * FROM Photo WHERE ANCESTOR IS :1 AND
initialized = TRUE AND prime = TRUE ORDER BY timestamp DESC,
self.user.key())

Regarding the 2nd set of queries I asked about, all things being
equal, is it better to do the query with or without the ANCESTOR?
Intuitively, I would expect the ANCESTOR version to perform faster as
it would only run on 1 machine / entity group, but is this true?

In the data model, Photo is a child of User and there 1000's more
photos than users. Otherwise, I haven't measured performance in the
app yet, I'd rather rely on some official best practices for now :)

On Jul 7, 6:25 pm, Ikai Lan (Google) ika...@google.com wrote:
 Ancestor queries don't add significant overhead, so I'm not going to
 consider that factor.

 As far as other queries go, in general fewer indexed mean better performance
 if you are doing zigzag join. However - if an index contains a small number
 of results, obviously the query will return faster because there are simply
 less results. Otherwise, you end up zig zagging across more indexes, which
 probably will result in slower queries. It really depends on the shape of
 your data. What have your obvservations been?

 Ikai Lan
 Developer Programs Engineer, Google App Engine
 Blog:http://googleappengine.blogspot.com
 Twitter:http://twitter.com/app_engine
 Reddit:http://www.reddit.com/r/appengine







 On Thu, Jul 7, 2011 at 6:21 PM, Pol i...@pol-online.net wrote:
  Hi,

  Assuming the app runs on an HR database and that the number of indexes
  is not a problem:

  Say that because of the data model, these queries are equivalent (i.e.
  return the exact same results), which one should be used to get the
  best performance?

  query = db.GqlQuery(SELECT * FROM Photo WHERE ANCESTOR IS :1 AND
  prime = TRUE ORDER BY timestamp DESC, self.user.key())

  query = db.GqlQuery(SELECT * FROM Photo WHERE ANCESTOR IS :1 AND
  initialized = TRUE AND prime = TRUE ORDER BY timestamp DESC,
  self.user.key())

  Does the response change if somewhere else in the code, there is also
  this query (because of shared indexes or something)?

  query = db.GqlQuery(SELECT * FROM Photo WHERE ANCESTOR IS :1 AND
  initialized = TRUE ORDER BY timestamp DESC, self.user.key())

  Same question, this time with these 2 other queries:

  query = db.GqlQuery(SELECT * FROM Photo WHERE event = :1 AND prime =
  TRUE AND hidden = FALSE ORDER BY timestamp DESC, event.key())

  query = db.GqlQuery(SELECT * FROM Photo WHERE ANCESTOR IS :1 AND
  event = :2 AND prime = TRUE AND hidden = FALSE ORDER BY timestamp
  DESC, event.key().parent(), event.key())

  I understand the 2nd version is an ancestor query which should return
  consistent results (right?) but in this case, it's ok if the results
  are a bit stalled.

  Thanks!

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Thread-safe Python Tips

[Pol]

On Jul 1, 10:10 am, Ikai Lan (Google) ika...@google.com wrote:
 It's possible for two operations to update townCache concurrently, but in
 your case it looks like it doesn't really matter. If TownModel is somehow
 updated between reads, it's theoretically possible for you to have an older
 TownModel in the local cache, but if you're going to store something in the
 cache with no expiration, it sounds like you don't care about this case
 anyway.

Are collections thread-safe in Python? Otherwise, townCache[id]
= ... called at the same time on multiple threads would likely
corrupt something.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Inconsistencies in GAE quota documentation?

[Pol]

Hi,

On http://code.google.com/appengine/docs/quotas.html, in the Requests
section, it says:

The total number of requests to the app. The per-minute quotas for
application with billing enabled allow for up to 500 requests per
second—more than one billion requests per month.

But further below, in the table, it also says:

  Billing Enabled Default Quota
  Maximum Rate
  45,200 requests/minute

So is it 30,000 (500 * 60) or 45,000?

Also if you end up requesting a raise for this limit, would it be only
temporary or can it be forever? This limit of 500 requests per second
may be OK for websites but seems like low for desktop or even mobile
apps. For instance, if you have 500 users running a desktop app at the
same time that's doing some batch processing on many instances, you'll
hit the limit immediately.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.