Re: [google-appengine] Python 2.7, threadsafe: true, and deferred.defer
> > What SDK version are you using? > > Cheers, > Brian > I was using 1.5.2 . Using 1.5.5 fixed it. I'm an idiot. Thanks ;-) -- Emlyn http://my.syyn.cc - Synchonise Google+, Facebook, WordPress and Google Buzz posts, comments and all. http://point7.wordpress.com - My blog Find me on Facebook and Buzz -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Python 2.7, threadsafe: true, and deferred.defer
On Fri, Oct 14, 2011 at 3:07 PM, Emlyn wrote:
>>
>> You are specifying a CGI script here. If you want to use CGI then set
>> threadsafe to "no".
>>
>> Cheers,
>> Brian
>
>
> Ah, gotcha. A fog of deep misunderstanding is slowly lifting from my
> brain, possibly. I could still have some Dunning-Kruger going, so
> here's my next attempt:
>
> app.yaml:
> ---
> application: emlynoregan
> version: 1
> runtime: python27
> api_version: 1
> threadsafe: true
>
> builtins:
> - deferred: on
>
> handlers:
> - url: /.*
> script: main.app
> ---
>
> and my main.py now says:
>
> ---
> import webapp2
> from starter import Starter
> app = webapp2.WSGIApplication([('/spinystarter', Starter)], debug=True)
> ---
>
> But I'm still getting the same error. It's related to the built-in
> defer, because when I remove
>
> builtins:
> - deferred: on
>
> from my app.yaml, it's all good. I suspect I need to not use the
> builtin directive, and instead do
>
> app = webapp2.WSGIApplication([('/spinystarter', Starter), (something,
> something else)], debug=True)
>
> but I don't know what "something" and "something else" are supposed to
> be. Does anyone know?
What SDK version are you using?
Cheers,
Brian
>
>>
>>> ---
>>>
>>> Upload via appcfg.py:
>>> ---
>>> Application: emlynoregan; version: 1
>>> Host: appengine.google.com
>>>
>>> Starting update of app: emlynoregan, version: 1
>>> Scanning files on local disk.
>>> Error 400: --- begin server output ---
>>> Error when loading application configuration:
>>> Invalid object:
>>> Threadsafe cannot be enabled with CGI handler:
>>> $PYTHON_LIB/google/appengine/ext/deferred/handler.py
>>> ---
>>>
>>>
>>> On 14 October 2011 00:50, Greg Darke (Google)
>>> wrote:
Have you specified 'runtime: python' or 'runtime: python27'?
If you are using 'runtime: python', then setting 'threadsafe: true' is
not supported.
On 14 October 2011 00:59, Emlyn wrote:
> I am using the builtin handler in app.yaml
>
> On Oct 14, 2011 12:25 AM, "Greg Darke" wrote:
>>
>> On Fri, Oct 14, 2011 at 12:10:09AM +1030, Emlyn wrote:
>> > I'm trying to give python 2.7 a go, with an app that uses
>> > deferred.defer. I'm getting this error when I try to upload it:
>> >
>> > Error 400: --- begin server output ---
>> > Error when loading application configuration:
>> > Invalid object:
>> > Threadsafe cannot be enabled with CGI handler:
>> > $PYTHON_LIB/google/appengine/ext/deferred/handler.py
>> >
>> > Is there a workaround? Or should I even be using deferred.defer in this
>> > context?
>>
>> The easiest way to solve this problem is to remove the explicit handler
>> you are defining, and use the builtin directive as described in
>>
>> http://code.google.com/appengine/docs/python/config/appconfig.html#Builtin_Handlers
>>
>> If you are running the deferred handler at non-default url, you will
>> need to specify the WSGI application in your app.yaml. This is
>> google.appengine.ext.deferred.application
>>
>
>>>
>>>
>>>
>>> --
>>> Emlyn
>>>
>>> http://my.syyn.cc - Synchonise Google+, Facebook, WordPress and Google
>>> Buzz posts,
>>> comments and all.
>>> http://point7.wordpress.com - My blog
>>> Find me on Facebook and Buzz
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "Google App Engine" group.
>>> To post to this group, send email to google-appengine@googlegroups.com.
>>> To unsubscribe from this group, send email to
>>> google-appengine+unsubscr...@googlegroups.com.
>>> For more options, visit this group at
>>> http://groups.google.com/group/google-appengine?hl=en.
>>>
>>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine" group.
>> To post to this group, send email to google-appengine@googlegroups.com.
>> To unsubscribe from this group, send email to
>> google-appengine+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>>
>
>
>
> --
> Emlyn
>
> http://my.syyn.cc - Synchonise Google+, Facebook, WordPress and Google
> Buzz posts,
> comments and all.
> http://point7.wordpress.com - My blog
> Find me on Facebook and Buzz
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this gro
RE: [google-appengine] Re: CloudFlare https traffic blocked?
I suspect they will be OK as long as you don't get too big too fast. I would have gone with Squid running on Amazon, a new scares me when you are looking at using it for secure stuff. From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Owen Wiggins Sent: Thursday, October 13, 2011 10:29 PM To: google-appengine@googlegroups.com Subject: [google-appengine] Re: CloudFlare https traffic blocked? It took a few weeks, but the answer from CloudFlare is: "GAE has apparently closed the port(s) in question & we will search for an alternative solution." Fortunately, Dmitry Ulupov at http://wwwizer.com has launched his SSL solution and got us up and running in about a day. I can't attest to the reliability since we haven't been using it for long, but it's worth a shot for those still looking for a solution. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/r4olIDEKmt0J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: CloudFlare https traffic blocked?
It took a few weeks, but the answer from CloudFlare is: "GAE has apparently closed the port(s) in question & we will search for an alternative solution." Fortunately, Dmitry Ulupov at http://wwwizer.com has launched his SSL solution and got us up and running in about a day. I can't attest to the reliability since we haven't been using it for long, but it's worth a shot for those still looking for a solution. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/r4olIDEKmt0J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: How to determine datastore reads?
Appstats http://googleappengine.blogspot.com/2010/03/easy-performance-profiling-with.html On Thu, Oct 13, 2011 at 10:34 PM, J wrote: > Anyone know a way I can profile my app to see where my datastore reads are > coming from? > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/HohHHRXvFmUJ. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Deadline exceeded on urlfetch calls this afternoon.
My app experienced this too, specifically with URL Fetches from UPS's systems. But it seems to be more likely a problem with UPS's servers, and not GAE. On Thu, Oct 13, 2011 at 5:56 PM, Will Reiher wrote: > Getting a bunch of Deadline Exceeded Application Error 5's from the > urlfetch library this afternoon with code that has been static for weeks. > Thankfully it's failing when checking the shipping rates and not when > processing transactions. I don't see any thing in System Status. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/LCo8WIRYTusJ. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Python 2.7, threadsafe: true, and deferred.defer
>
> You are specifying a CGI script here. If you want to use CGI then set
> threadsafe to "no".
>
> Cheers,
> Brian
Ah, gotcha. A fog of deep misunderstanding is slowly lifting from my
brain, possibly. I could still have some Dunning-Kruger going, so
here's my next attempt:
app.yaml:
---
application: emlynoregan
version: 1
runtime: python27
api_version: 1
threadsafe: true
builtins:
- deferred: on
handlers:
- url: /.*
script: main.app
---
and my main.py now says:
---
import webapp2
from starter import Starter
app = webapp2.WSGIApplication([('/spinystarter', Starter)], debug=True)
---
But I'm still getting the same error. It's related to the built-in
defer, because when I remove
builtins:
- deferred: on
from my app.yaml, it's all good. I suspect I need to not use the
builtin directive, and instead do
app = webapp2.WSGIApplication([('/spinystarter', Starter), (something,
something else)], debug=True)
but I don't know what "something" and "something else" are supposed to
be. Does anyone know?
>
>> ---
>>
>> Upload via appcfg.py:
>> ---
>> Application: emlynoregan; version: 1
>> Host: appengine.google.com
>>
>> Starting update of app: emlynoregan, version: 1
>> Scanning files on local disk.
>> Error 400: --- begin server output ---
>> Error when loading application configuration:
>> Invalid object:
>> Threadsafe cannot be enabled with CGI handler:
>> $PYTHON_LIB/google/appengine/ext/deferred/handler.py
>> ---
>>
>>
>> On 14 October 2011 00:50, Greg Darke (Google)
>> wrote:
>>> Have you specified 'runtime: python' or 'runtime: python27'?
>>>
>>> If you are using 'runtime: python', then setting 'threadsafe: true' is
>>> not supported.
>>>
>>> On 14 October 2011 00:59, Emlyn wrote:
I am using the builtin handler in app.yaml
On Oct 14, 2011 12:25 AM, "Greg Darke" wrote:
>
> On Fri, Oct 14, 2011 at 12:10:09AM +1030, Emlyn wrote:
> > I'm trying to give python 2.7 a go, with an app that uses
> > deferred.defer. I'm getting this error when I try to upload it:
> >
> > Error 400: --- begin server output ---
> > Error when loading application configuration:
> > Invalid object:
> > Threadsafe cannot be enabled with CGI handler:
> > $PYTHON_LIB/google/appengine/ext/deferred/handler.py
> >
> > Is there a workaround? Or should I even be using deferred.defer in this
> > context?
>
> The easiest way to solve this problem is to remove the explicit handler
> you are defining, and use the builtin directive as described in
>
> http://code.google.com/appengine/docs/python/config/appconfig.html#Builtin_Handlers
>
> If you are running the deferred handler at non-default url, you will
> need to specify the WSGI application in your app.yaml. This is
> google.appengine.ext.deferred.application
>
>>>
>>
>>
>>
>> --
>> Emlyn
>>
>> http://my.syyn.cc - Synchonise Google+, Facebook, WordPress and Google
>> Buzz posts,
>> comments and all.
>> http://point7.wordpress.com - My blog
>> Find me on Facebook and Buzz
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine" group.
>> To post to this group, send email to google-appengine@googlegroups.com.
>> To unsubscribe from this group, send email to
>> google-appengine+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
Emlyn
http://my.syyn.cc - Synchonise Google+, Facebook, WordPress and Google
Buzz posts,
comments and all.
http://point7.wordpress.com - My blog
Find me on Facebook and Buzz
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: How to determine datastore reads?
Anyone know a way I can profile my app to see where my datastore reads are coming from? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/HohHHRXvFmUJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Python 2.7, threadsafe: true, and deferred.defer
On Fri, Oct 14, 2011 at 1:48 PM, Emlyn wrote: > app.yaml: > --- > application: emlynoregan > version: 1 > runtime: python27 > api_version: 1 > threadsafe: true > > > builtins: > - deferred: on > > handlers: > - url: /.* > script: main.py You are specifying a CGI script here. If you want to use CGI then set threadsafe to "no". Cheers, Brian > --- > > Upload via appcfg.py: > --- > Application: emlynoregan; version: 1 > Host: appengine.google.com > > Starting update of app: emlynoregan, version: 1 > Scanning files on local disk. > Error 400: --- begin server output --- > Error when loading application configuration: > Invalid object: > Threadsafe cannot be enabled with CGI handler: > $PYTHON_LIB/google/appengine/ext/deferred/handler.py > --- > > > On 14 October 2011 00:50, Greg Darke (Google) wrote: >> Have you specified 'runtime: python' or 'runtime: python27'? >> >> If you are using 'runtime: python', then setting 'threadsafe: true' is >> not supported. >> >> On 14 October 2011 00:59, Emlyn wrote: >>> I am using the builtin handler in app.yaml >>> >>> On Oct 14, 2011 12:25 AM, "Greg Darke" wrote: On Fri, Oct 14, 2011 at 12:10:09AM +1030, Emlyn wrote: > I'm trying to give python 2.7 a go, with an app that uses > deferred.defer. I'm getting this error when I try to upload it: > > Error 400: --- begin server output --- > Error when loading application configuration: > Invalid object: > Threadsafe cannot be enabled with CGI handler: > $PYTHON_LIB/google/appengine/ext/deferred/handler.py > > Is there a workaround? Or should I even be using deferred.defer in this > context? The easiest way to solve this problem is to remove the explicit handler you are defining, and use the builtin directive as described in http://code.google.com/appengine/docs/python/config/appconfig.html#Builtin_Handlers If you are running the deferred handler at non-default url, you will need to specify the WSGI application in your app.yaml. This is google.appengine.ext.deferred.application >>> >> > > > > -- > Emlyn > > http://my.syyn.cc - Synchonise Google+, Facebook, WordPress and Google > Buzz posts, > comments and all. > http://point7.wordpress.com - My blog > Find me on Facebook and Buzz > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Python 2.7, threadsafe: true, and deferred.defer
app.yaml: --- application: emlynoregan version: 1 runtime: python27 api_version: 1 threadsafe: true builtins: - deferred: on handlers: - url: /.* script: main.py --- Upload via appcfg.py: --- Application: emlynoregan; version: 1 Host: appengine.google.com Starting update of app: emlynoregan, version: 1 Scanning files on local disk. Error 400: --- begin server output --- Error when loading application configuration: Invalid object: Threadsafe cannot be enabled with CGI handler: $PYTHON_LIB/google/appengine/ext/deferred/handler.py --- On 14 October 2011 00:50, Greg Darke (Google) wrote: > Have you specified 'runtime: python' or 'runtime: python27'? > > If you are using 'runtime: python', then setting 'threadsafe: true' is > not supported. > > On 14 October 2011 00:59, Emlyn wrote: >> I am using the builtin handler in app.yaml >> >> On Oct 14, 2011 12:25 AM, "Greg Darke" wrote: >>> >>> On Fri, Oct 14, 2011 at 12:10:09AM +1030, Emlyn wrote: >>> > I'm trying to give python 2.7 a go, with an app that uses >>> > deferred.defer. I'm getting this error when I try to upload it: >>> > >>> > Error 400: --- begin server output --- >>> > Error when loading application configuration: >>> > Invalid object: >>> > Threadsafe cannot be enabled with CGI handler: >>> > $PYTHON_LIB/google/appengine/ext/deferred/handler.py >>> > >>> > Is there a workaround? Or should I even be using deferred.defer in this >>> > context? >>> >>> The easiest way to solve this problem is to remove the explicit handler >>> you are defining, and use the builtin directive as described in >>> >>> http://code.google.com/appengine/docs/python/config/appconfig.html#Builtin_Handlers >>> >>> If you are running the deferred handler at non-default url, you will >>> need to specify the WSGI application in your app.yaml. This is >>> google.appengine.ext.deferred.application >>> >> > -- Emlyn http://my.syyn.cc - Synchonise Google+, Facebook, WordPress and Google Buzz posts, comments and all. http://point7.wordpress.com - My blog Find me on Facebook and Buzz -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Green line, "Billed", on Instances graph.
There's a green line, "Billed", on the instances graph. Now you can ignore my blog posts, and just play around with what moves the green line. Brilliant, good stuff AppEngine people. -- Emlyn http://my.syyn.cc - Synchonise Google+, Facebook, WordPress and Google Buzz posts, comments and all. http://point7.wordpress.com - My blog Find me on Facebook and Buzz -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Deadline exceeded on urlfetch calls this afternoon.
Getting a bunch of Deadline Exceeded Application Error 5's from the urlfetch library this afternoon with code that has been static for weeks. Thankfully it's failing when checking the shipping rates and not when processing transactions. I don't see any thing in System Status. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/LCo8WIRYTusJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] Re: Here's your platform...
Enterprise comes in different "sizes". What is "HUGE" to most of us is miniscule for Google. Imagine how big Gmail is. If your Fortune 500 company had an app that got Gmail like traffic you wouldn't outsource the hosting, the mark up would kill you. When you are a certain size not having to have people and resources allocated saves you money, but when you get to a certain level of utilization the markup on other people providing those things starts to kill you. The way "clouds" work follows a lifecycle. Start on an Obfuscated platform, no set up, no management, focus on the code. Move to a Performance of several Specialized platforms. (EC2, Hosted DataBases, CDNs) Start focusing on infrastructure, tuning, and cost optimizations. Move to portions of platforms under your management on your Hardware/Infrastructure. Start Focusing on building resources that run at near 100% utilization, Optimized hardware for Performance, and Network Topologies that align with your specific demographics. Use Cloud Services for Over Flow, Burst Capacity and Point of Presence in regions which are not ideal for full presence. Complete Insource. Minimal failover, burst capacity and peering relationships with external partners. Mostly for "End of world" scenarios, and as part of strategic partnerships. Which of these stages do you think Google spends most of its days in? Most of the people on this list will never get to that final stage. And "Enterprise" like Universities will never get to that last stage. That doesn't mean they aren't enterprise just that there are 5 orders of magnitude between Michigan State Universities 40k user base, and Facebook's 4 Billion. -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Luis Daniel Mesa Velasquez Sent: Thursday, October 13, 2011 1:50 PM To: Google App Engine Subject: [google-appengine] Re: Here's your platform... Yeah, that AND because writing useful software using the many, many, many limitations of GAE is too hard for Google products to be out in time, let other people's products be late. You don't put all your eggs on the basket, you put some of your eggs on the basket and charge other people to put some of their eggs on that same basket. The thing is, Google owns the basket. If Google doesn't trust it's own basket then why should everyone else? It's a (would-be-)great platform. Just not "enterprise" great, no matter how many companies you invite (pay to talk) about how great GAE is for their enterprise. On Oct 13, 12:53 pm, "Brandon Wirtz" wrote: > Josh, > > I sure hope the Googlers on this list don't take your advice, I'd like > them to continue to have employment for a while, and doing as you say > might end that. > > Amazon doesn't run on EC2, Microsoft doesn't run on Azure. Cloud > computing is NOT for Super Enterprise. It is for those still looking > to scale and grow. Those who don't want to deal with infrastructure. > But if you are truly building the Fastest, Most Reliable, Biggest > tools, the obfuscation of the hardware is a bad thing. > > It is like me going to the race track. I'm amateur, I take my Spec > Mini cooper and I run. Sometimes I rent a Lotus. That rental is a > great deal for me, but I can't tune it to my preferences. It is > faster than my Mini, it is well maintained, but compared to what it > could do if I owned it, it is slow and limiting. > > Google does run some projects on GAE, Microsoft runs some projects on Azure. > Certainly the lessons learned from these platforms goes back to other > projects, but you don't put all of your eggs in one basket. > > > > > > > > -Original Message- > From: google-appengine@googlegroups.com > > [mailto:google-appengine@googlegroups.com] On Behalf Of Joshua Smith > Sent: Thursday, October 13, 2011 8:00 AM > To: google-appengine@googlegroups.com > Cc: Peter Magnusson; Gregory D'alesandre > Subject: [google-appengine] Here's your platform... > > I'm sure you've all seen this by now: > > http://siliconfilter.com/google-engineer-google-is-a-prime-example-of... > mplete-failure-to-understand-platforms/ > > (skip the blog entry and go right down to the actual posting at the bottom). > > As I read it, it struck me that GAE totally IS a platform. And it > struck me how much more awesome GAE would be if maps, and docs, and > charts, and everything else was written on it. > > So my advice to you Peter and Gregory is to march into > whoever's-in-charge-these-day's office and say: "He's right. And > here's how to fix it: Require that from this moment forward EVERY > product (except > search) google offers MUST be built on GAE." > > GAE would increase in awesomeness by 1000 fold, because your peers in > google would demand that it does. > > -Joshua > > -- > You received this message because you are subscribed to the Google > Groups "Google App Engine" group. > To post to this group, send email to google-appengi
[google-appengine] HttpSession session timeout and lastAccessedTime
I'm having strange problems with session timeouts on appengine. When
running locally the session timeout appears to work fine. The session
is supposed to be 30 minutes. The browser has a timer setup to check
if the user is still logged in 30min + 1 sec after the last successful
async request to server returns. I had initially tried to not
determine our own timeout logic, as setting the maxInactiveInterval is
supposed to invalidate the session (running locally or on appengine
this would not work properly). Am I missing something that's causing
different behavior between running locally versus hosted in appengine?
Thanks for the help.
In our filter setup for each request we load our own custom context,
which is stored with the session. Here is relevant code:
private static void loadCurrentContext(HttpServletRequest req) {
boolean expire = false;
HttpSession session = req.getSession(false);
if (session == null || session.isNew()) {
registerAccountWithSession(null, req);
} else {
// the sessionTimeoutMillis const is 180L, or 30 min.
expire = FoxUtil.sessionTimeoutMillis <
(System.currentTimeMillis()
- session.getLastAccessedTime());
}
SadfoxServerContext context =
SadfoxServlet.getSadfoxContextFromRequest(req);
if (expire || context == null) {
clearUserSession(req);
registerAccountWithSession(null, req);
context = SadfoxServlet.getSadfoxContextFromRequest(req);
}
if (context != null && context.getActorKey() != null) {
AccountHalper halper = new AccountHalper(context.getActorKey());
halper.populateRolesLists();
context.setActor(halper.getEntity());
}
}
public static SadfoxServerContext
registerAccountWithSession(CloudAccount actor, HttpServletRequest req)
{
HttpSession session = req.getSession(true);
SadfoxServerContext context = new SadfoxServerContext(actor);
context.setClientInfo(req);
// the sessionTimeoutSeconds const is 1800, or 30 min.
session.setMaxInactiveInterval(FoxUtil.sessionTimeoutSeconds);
session.setAttribute(SadfoxServlet.sadfoxContextKey, context);
return context;
}
public static void clearUserSession(HttpServletRequest req) {
registerAccountWithSession(null, req);
HttpSession session = req.getSession(false);
if (session != null) {
session.invalidate();
}
}
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] zipped python code in blobstore?
you can do it in python 2.7 runtime http://code.google.com/intl/en/appengine/docs/python/python27/newin27.html#Bytecode_Modification 2011/10/13 andreas schmid > hi, > > i was wondering if it is actually possible to store zipped python code in > the blobstore and use it after getting it? > > thx > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- С уважением, Кулин Тимофей. Телефон: +7 (4852) 974793 ICQ: 114902104 email: timo...@koolin.ru Blog: http://timofey.koolin.ru -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: debugging app engine
I think you can use logging to log your debug messages
import logging
logging.debug("Your message")
It will then show up in admin console logs
cheers
gipsy
On Oct 13, 4:02 pm, Brian Williams
wrote:
> Hi.
> I am trying to use app engine to email information collected from an app
> inventor app.
> I am using Shivals wolfwebmail2 (for those familiar with app inventor)
> I have asked this question around all the groups in numerous forms.
> Would like to work out what is going on for myself but need some info.
> I see the post request arrives at the engine but nothing arrives at email
> recipient.
> Am trying to work through python code to determine what everything does as i
> am a newbie i am starting from the beginning.
> Is there any way to see what actually arrives at app engine and what code
> uploaded to app engine is doing in a step by step log file?
>
> Thank you for any help you can give.
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Here's your platform...
Yeah, that AND because writing useful software using the many, many, many limitations of GAE is too hard for Google products to be out in time, let other people's products be late. You don't put all your eggs on the basket, you put some of your eggs on the basket and charge other people to put some of their eggs on that same basket. The thing is, Google owns the basket. If Google doesn't trust it's own basket then why should everyone else? It's a (would-be-)great platform. Just not "enterprise" great, no matter how many companies you invite (pay to talk) about how great GAE is for their enterprise. On Oct 13, 12:53 pm, "Brandon Wirtz" wrote: > Josh, > > I sure hope the Googlers on this list don't take your advice, I'd like them > to continue to have employment for a while, and doing as you say might end > that. > > Amazon doesn't run on EC2, Microsoft doesn't run on Azure. Cloud computing > is NOT for Super Enterprise. It is for those still looking to scale and > grow. Those who don't want to deal with infrastructure. But if you are truly > building the Fastest, Most Reliable, Biggest tools, the obfuscation of the > hardware is a bad thing. > > It is like me going to the race track. I'm amateur, I take my Spec Mini > cooper and I run. Sometimes I rent a Lotus. That rental is a great deal for > me, but I can't tune it to my preferences. It is faster than my Mini, it is > well maintained, but compared to what it could do if I owned it, it is slow > and limiting. > > Google does run some projects on GAE, Microsoft runs some projects on Azure. > Certainly the lessons learned from these platforms goes back to other > projects, but you don't put all of your eggs in one basket. > > > > > > > > -Original Message- > From: google-appengine@googlegroups.com > > [mailto:google-appengine@googlegroups.com] On Behalf Of Joshua Smith > Sent: Thursday, October 13, 2011 8:00 AM > To: google-appengine@googlegroups.com > Cc: Peter Magnusson; Gregory D'alesandre > Subject: [google-appengine] Here's your platform... > > I'm sure you've all seen this by now: > > http://siliconfilter.com/google-engineer-google-is-a-prime-example-of... > mplete-failure-to-understand-platforms/ > > (skip the blog entry and go right down to the actual posting at the bottom). > > As I read it, it struck me that GAE totally IS a platform. And it struck me > how much more awesome GAE would be if maps, and docs, and charts, and > everything else was written on it. > > So my advice to you Peter and Gregory is to march into > whoever's-in-charge-these-day's office and say: "He's right. And here's how > to fix it: Require that from this moment forward EVERY product (except > search) google offers MUST be built on GAE." > > GAE would increase in awesomeness by 1000 fold, because your peers in google > would demand that it does. > > -Joshua > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group > athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] debugging app engine
Hi. I am trying to use app engine to email information collected from an app inventor app. I am using Shivals wolfwebmail2 (for those familiar with app inventor) I have asked this question around all the groups in numerous forms. Would like to work out what is going on for myself but need some info. I see the post request arrives at the engine but nothing arrives at email recipient. Am trying to work through python code to determine what everything does as i am a newbie i am starting from the beginning. Is there any way to see what actually arrives at app engine and what code uploaded to app engine is doing in a step by step log file? Thank you for any help you can give. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/ASmqurRSaJMJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] zipped python code in blobstore?
hi, i was wondering if it is actually possible to store zipped python code in the blobstore and use it after getting it? thx -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] Here's your platform...
That "generally speaking" and "and such" are the differences I'm talking about. GAE Runs on Google Edge, and within the "Apps For Domains". Google has as much "Dog Fooding" their products as MSFT and AMZ do. I lived in MSFT for years, I still do a lot of work with them internally. Your understanding of how much they are dog fooding is romanticized. There are a CRAP TON of LAMPs in MSFT deployments. There are a LOT of Squids. There are 100 different Packet Shapers depending on the product teams choices. Much of the Public facing bits of MSFT runs on LimeLight and other CDN's. AMZ uses a standardized Hardware Unit. That is where the EC2-Ness of AMZ's Internal and External infrastructure ends. I don't know but I suspect that GAE runs on Google's Standardized Hardware Unit as well. -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Joshua Smith Sent: Thursday, October 13, 2011 11:23 AM To: google-appengine@googlegroups.com Subject: Re: [google-appengine] Here's your platform... The point of the "leaked" rant was that Amazon does run on the same platform infrastructure they sell (not EC2 exactly, but generally speaking); and MS does, too (not Azure, but Windows Servers and such). Google has exposed a platform interface in GAE which is a good candidate to be the "platform" that google is missing, strategically. I wouldn't argue that it would be the right way to deploy the Maps API today. But I would argue that by making it the right way to deploy the Maps API, they will create something much more wonderful for all of us to use. Google already obfuscates the hardware under their product implementations, I'm sure. But they're using the "people food" version. We're stuck with the "dog food" version. (read the rant if that sentence doesn't mean anything to you) Exposing the people food they have today to outsiders is not realistic. But bringing the quality of the dog food up to the quality people expect is realistic. And using the public platform internally to build products is the way Amazon got there. On Oct 13, 2011, at 1:53 PM, Brandon Wirtz wrote: > Josh, > > I sure hope the Googlers on this list don't take your advice, I'd like > them to continue to have employment for a while, and doing as you say > might end that. > > Amazon doesn't run on EC2, Microsoft doesn't run on Azure. Cloud > computing is NOT for Super Enterprise. It is for those still looking > to scale and grow. Those who don't want to deal with infrastructure. > But if you are truly building the Fastest, Most Reliable, Biggest > tools, the obfuscation of the hardware is a bad thing. > > It is like me going to the race track. I'm amateur, I take my Spec > Mini cooper and I run. Sometimes I rent a Lotus. That rental is a > great deal for me, but I can't tune it to my preferences. It is > faster than my Mini, it is well maintained, but compared to what it > could do if I owned it, it is slow and limiting. > > Google does run some projects on GAE, Microsoft runs some projects on Azure. > Certainly the lessons learned from these platforms goes back to other > projects, but you don't put all of your eggs in one basket. > > -Original Message- > From: google-appengine@googlegroups.com > [mailto:google-appengine@googlegroups.com] On Behalf Of Joshua Smith > Sent: Thursday, October 13, 2011 8:00 AM > To: google-appengine@googlegroups.com > Cc: Peter Magnusson; Gregory D'alesandre > Subject: [google-appengine] Here's your platform... > > I'm sure you've all seen this by now: > > http://siliconfilter.com/google-engineer-google-is-a-prime-example-of- > our-co mplete-failure-to-understand-platforms/ > > (skip the blog entry and go right down to the actual posting at the bottom). > > As I read it, it struck me that GAE totally IS a platform. And it > struck me how much more awesome GAE would be if maps, and docs, and > charts, and everything else was written on it. > > So my advice to you Peter and Gregory is to march into > whoever's-in-charge-these-day's office and say: "He's right. And > here's how to fix it: Require that from this moment forward EVERY > product (except > search) google offers MUST be built on GAE." > > GAE would increase in awesomeness by 1000 fold, because your peers in > google would demand that it does. > > -Joshua > > -- > You received this message because you are subscribed to the Google > Groups "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > > -- > You received this message because you are subscribed to the Google Groups "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this gro
Re: [google-appengine] Here's your platform...
FWIW, the Maps API is one of the better APIs at Google. Jeff On Thu, Oct 13, 2011 at 11:23 AM, Joshua Smith wrote: > The point of the "leaked" rant was that Amazon does run on the same platform > infrastructure they sell (not EC2 exactly, but generally speaking); and MS > does, too (not Azure, but Windows Servers and such). > > Google has exposed a platform interface in GAE which is a good candidate to > be the "platform" that google is missing, strategically. I wouldn't argue > that it would be the right way to deploy the Maps API today. But I would > argue that by making it the right way to deploy the Maps API, they will > create something much more wonderful for all of us to use. > > Google already obfuscates the hardware under their product implementations, > I'm sure. But they're using the "people food" version. We're stuck with the > "dog food" version. (read the rant if that sentence doesn't mean anything to > you) > > Exposing the people food they have today to outsiders is not realistic. But > bringing the quality of the dog food up to the quality people expect is > realistic. And using the public platform internally to build products is the > way Amazon got there. > > On Oct 13, 2011, at 1:53 PM, Brandon Wirtz wrote: > >> Josh, >> >> I sure hope the Googlers on this list don't take your advice, I'd like them >> to continue to have employment for a while, and doing as you say might end >> that. >> >> Amazon doesn't run on EC2, Microsoft doesn't run on Azure. Cloud computing >> is NOT for Super Enterprise. It is for those still looking to scale and >> grow. Those who don't want to deal with infrastructure. But if you are truly >> building the Fastest, Most Reliable, Biggest tools, the obfuscation of the >> hardware is a bad thing. >> >> It is like me going to the race track. I'm amateur, I take my Spec Mini >> cooper and I run. Sometimes I rent a Lotus. That rental is a great deal for >> me, but I can't tune it to my preferences. It is faster than my Mini, it is >> well maintained, but compared to what it could do if I owned it, it is slow >> and limiting. >> >> Google does run some projects on GAE, Microsoft runs some projects on Azure. >> Certainly the lessons learned from these platforms goes back to other >> projects, but you don't put all of your eggs in one basket. >> >> -Original Message- >> From: google-appengine@googlegroups.com >> [mailto:google-appengine@googlegroups.com] On Behalf Of Joshua Smith >> Sent: Thursday, October 13, 2011 8:00 AM >> To: google-appengine@googlegroups.com >> Cc: Peter Magnusson; Gregory D'alesandre >> Subject: [google-appengine] Here's your platform... >> >> I'm sure you've all seen this by now: >> >> http://siliconfilter.com/google-engineer-google-is-a-prime-example-of-our-co >> mplete-failure-to-understand-platforms/ >> >> (skip the blog entry and go right down to the actual posting at the bottom). >> >> As I read it, it struck me that GAE totally IS a platform. And it struck me >> how much more awesome GAE would be if maps, and docs, and charts, and >> everything else was written on it. >> >> So my advice to you Peter and Gregory is to march into >> whoever's-in-charge-these-day's office and say: "He's right. And here's how >> to fix it: Require that from this moment forward EVERY product (except >> search) google offers MUST be built on GAE." >> >> GAE would increase in awesomeness by 1000 fold, because your peers in google >> would demand that it does. >> >> -Joshua >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http:
[google-appengine] please contact the App Engine team - Request was aborted after waiting too long to attempt to service your request
I'm seeing a lot of "Request was aborted after waiting too long to attempt to service your request" for simple tasks that typically take 25-150 ms to be serviced. Most of the failures are for the simplest things that our server does, inspect some arguments and return an XML response. There is no calculating, external lookups or db calls for this. A few weeks ago, I changed the max idle instances to the minimum as user latency isn't a big issue. This app uses next to no processing power, so the change from CPU time to instance hours results in a large percent bill increase, but not a particularly large bill. When I noticed the volume of errors I switched everything back to automatic yesterday, but that has not helped, and the frequency of the errors hasn't slowed down. The long term graph of errors per second shows that this problem seems to have started 4 days ago, and has gotten much worse in the last 24 hours. The python dashboard shows no trouble. this seems like an odd way of getting support but help. -s -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Here's your platform...
The point of the "leaked" rant was that Amazon does run on the same platform infrastructure they sell (not EC2 exactly, but generally speaking); and MS does, too (not Azure, but Windows Servers and such). Google has exposed a platform interface in GAE which is a good candidate to be the "platform" that google is missing, strategically. I wouldn't argue that it would be the right way to deploy the Maps API today. But I would argue that by making it the right way to deploy the Maps API, they will create something much more wonderful for all of us to use. Google already obfuscates the hardware under their product implementations, I'm sure. But they're using the "people food" version. We're stuck with the "dog food" version. (read the rant if that sentence doesn't mean anything to you) Exposing the people food they have today to outsiders is not realistic. But bringing the quality of the dog food up to the quality people expect is realistic. And using the public platform internally to build products is the way Amazon got there. On Oct 13, 2011, at 1:53 PM, Brandon Wirtz wrote: > Josh, > > I sure hope the Googlers on this list don't take your advice, I'd like them > to continue to have employment for a while, and doing as you say might end > that. > > Amazon doesn't run on EC2, Microsoft doesn't run on Azure. Cloud computing > is NOT for Super Enterprise. It is for those still looking to scale and > grow. Those who don't want to deal with infrastructure. But if you are truly > building the Fastest, Most Reliable, Biggest tools, the obfuscation of the > hardware is a bad thing. > > It is like me going to the race track. I'm amateur, I take my Spec Mini > cooper and I run. Sometimes I rent a Lotus. That rental is a great deal for > me, but I can't tune it to my preferences. It is faster than my Mini, it is > well maintained, but compared to what it could do if I owned it, it is slow > and limiting. > > Google does run some projects on GAE, Microsoft runs some projects on Azure. > Certainly the lessons learned from these platforms goes back to other > projects, but you don't put all of your eggs in one basket. > > -Original Message- > From: google-appengine@googlegroups.com > [mailto:google-appengine@googlegroups.com] On Behalf Of Joshua Smith > Sent: Thursday, October 13, 2011 8:00 AM > To: google-appengine@googlegroups.com > Cc: Peter Magnusson; Gregory D'alesandre > Subject: [google-appengine] Here's your platform... > > I'm sure you've all seen this by now: > > http://siliconfilter.com/google-engineer-google-is-a-prime-example-of-our-co > mplete-failure-to-understand-platforms/ > > (skip the blog entry and go right down to the actual posting at the bottom). > > As I read it, it struck me that GAE totally IS a platform. And it struck me > how much more awesome GAE would be if maps, and docs, and charts, and > everything else was written on it. > > So my advice to you Peter and Gregory is to march into > whoever's-in-charge-these-day's office and say: "He's right. And here's how > to fix it: Require that from this moment forward EVERY product (except > search) google offers MUST be built on GAE." > > GAE would increase in awesomeness by 1000 fold, because your peers in google > would demand that it does. > > -Joshua > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] Here's your platform...
Josh, I sure hope the Googlers on this list don't take your advice, I'd like them to continue to have employment for a while, and doing as you say might end that. Amazon doesn't run on EC2, Microsoft doesn't run on Azure. Cloud computing is NOT for Super Enterprise. It is for those still looking to scale and grow. Those who don't want to deal with infrastructure. But if you are truly building the Fastest, Most Reliable, Biggest tools, the obfuscation of the hardware is a bad thing. It is like me going to the race track. I'm amateur, I take my Spec Mini cooper and I run. Sometimes I rent a Lotus. That rental is a great deal for me, but I can't tune it to my preferences. It is faster than my Mini, it is well maintained, but compared to what it could do if I owned it, it is slow and limiting. Google does run some projects on GAE, Microsoft runs some projects on Azure. Certainly the lessons learned from these platforms goes back to other projects, but you don't put all of your eggs in one basket. -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Joshua Smith Sent: Thursday, October 13, 2011 8:00 AM To: google-appengine@googlegroups.com Cc: Peter Magnusson; Gregory D'alesandre Subject: [google-appengine] Here's your platform... I'm sure you've all seen this by now: http://siliconfilter.com/google-engineer-google-is-a-prime-example-of-our-co mplete-failure-to-understand-platforms/ (skip the blog entry and go right down to the actual posting at the bottom). As I read it, it struck me that GAE totally IS a platform. And it struck me how much more awesome GAE would be if maps, and docs, and charts, and everything else was written on it. So my advice to you Peter and Gregory is to march into whoever's-in-charge-these-day's office and say: "He's right. And here's how to fix it: Require that from this moment forward EVERY product (except search) google offers MUST be built on GAE." GAE would increase in awesomeness by 1000 fold, because your peers in google would demand that it does. -Joshua -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Storing date format in dd-mm-yyyy in datastore
I don't know about web2py but it's best with python to just store a
datetime object. That way you can manipulate it in any way needed.
On Oct 13, 10:58 am, WallyDD wrote:
> Hi Rohit,
>
> How the date is stored is largely irrelevant. Store them any way you
> can, as long as it works.
>
> When it comes to displaying the date just configure your output to how
> you would like your dates displayed.
>
> Hope this helps.
>
> On Oct 11, 9:12 pm, Rohit wrote:
>
>
>
>
>
>
>
> > I am developing an online booking application on Google app engine in
> > python using web2py framework where I am storing the date using the
> > date type of web2py, which stores the date in -mm-dd format.I need
> > to store the data in dd-mm- format in the datastore.
>
> > For this I tried to use this :
> > date.requires = IS_DATE(format=T('%d-%m-%Y'))
>
> > Though it takes the date in dd-mm- format it does not store in the
> > dd-mm- format.It still
> > stores in -mm-dd format.
>
> > Please provide suggestion
>
> > Thanks
> > Rohit
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Storing date format in dd-mm-yyyy in datastore
Hi Rohit,
How the date is stored is largely irrelevant. Store them any way you
can, as long as it works.
When it comes to displaying the date just configure your output to how
you would like your dates displayed.
Hope this helps.
On Oct 11, 9:12 pm, Rohit wrote:
> I am developing an online booking application on Google app engine in
> python using web2py framework where I am storing the date using the
> date type of web2py, which stores the date in -mm-dd format.I need
> to store the data in dd-mm- format in the datastore.
>
> For this I tried to use this :
> date.requires = IS_DATE(format=T('%d-%m-%Y'))
>
> Though it takes the date in dd-mm- format it does not store in the
> dd-mm- format.It still
> stores in -mm-dd format.
>
> Please provide suggestion
>
> Thanks
> Rohit
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 6470ms for a memcache get and write out the result?
the same code on another app responds in 14ms
how can this be?
On Oct 13, 2011, at 10:34 AM, andreas schmid wrote:
> hi,
>
> im having serious speed issues on a site.
> my app is a python app and im using torndadoweb to serve requests.
> the actual rendered html is also store in the datastore and memcache.
> a decorator on the GET request checks first if there is a memcache entry if
> it exists it writes the html out,
> if there is no memcache entry it tries to get it from the datastore and only
> if that doesnt exist too it executes the handler.
>
> what i noticed is that the simple get from the memcache until the page
> displays on the browser takes way too long and i actually dont understand how
> this is possible.
>
> here is my stats result:
>
>
> what is actually taking 6seconds here
>
> my decorator looks like this:
>
> def cache_page(with_keywords=False, keywords=None):
>def decorator(method):
>def wrapper(self, *args, **kwargs):
>uri = self.request.uri.strip()
>cachekey = '%s|%s' %(appversion, uri)
>cached = memcache.get(cachekey,
> namespace='cachedpage|%s'%appversion)
>if cached:
>self.write(cached)
>return
>page = CachedObject.all(keys_only=True).filter('appversion =',
> appversion) \
> .filter('uri =', uri).get()
>
>if page:
>p = db.get(str(page))
>self.write(p.html)
>memcache.set(cachekey, p.html,
> namespace='cachedpage|%s'%appversion)
>return
>else:
> ...
>return method(self, *args, **kwargs)
>return wrapper
>return decorator
>
>
> this shouldnt take that long right?
> any thoughts?
>
> thank you
<>--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Here's your platform...
I'm sure you've all seen this by now: http://siliconfilter.com/google-engineer-google-is-a-prime-example-of-our-complete-failure-to-understand-platforms/ (skip the blog entry and go right down to the actual posting at the bottom). As I read it, it struck me that GAE totally IS a platform. And it struck me how much more awesome GAE would be if maps, and docs, and charts, and everything else was written on it. So my advice to you Peter and Gregory is to march into whoever's-in-charge-these-day's office and say: "He's right. And here's how to fix it: Require that from this moment forward EVERY product (except search) google offers MUST be built on GAE." GAE would increase in awesomeness by 1000 fold, because your peers in google would demand that it does. -Joshua -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] 6470ms for a memcache get and write out the result?
hi,
im having serious speed issues on a site.
my app is a python app and im using torndadoweb to serve requests.
the actual rendered html is also store in the datastore and memcache.
a decorator on the GET request checks first if there is a memcache entry if it
exists it writes the html out,
if there is no memcache entry it tries to get it from the datastore and only if
that doesnt exist too it executes the handler.
what i noticed is that the simple get from the memcache until the page displays
on the browser takes way too long and i actually dont understand how this is
possible.
here is my stats result:
what is actually taking 6seconds here
my decorator looks like this:
def cache_page(with_keywords=False, keywords=None):
def decorator(method):
def wrapper(self, *args, **kwargs):
uri = self.request.uri.strip()
cachekey = '%s|%s' %(appversion, uri)
cached = memcache.get(cachekey,
namespace='cachedpage|%s'%appversion)
if cached:
self.write(cached)
return
page = CachedObject.all(keys_only=True).filter('appversion =',
appversion) \
.filter('uri =', uri).get()
if page:
p = db.get(str(page))
self.write(p.html)
memcache.set(cachekey, p.html,
namespace='cachedpage|%s'%appversion)
return
else:
...
return method(self, *args, **kwargs)
return wrapper
return decorator
this shouldnt take that long right?
any thoughts?
thank you<>--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Python 2.7, threadsafe: true, and deferred.defer
Have you specified 'runtime: python' or 'runtime: python27'? If you are using 'runtime: python', then setting 'threadsafe: true' is not supported. On 14 October 2011 00:59, Emlyn wrote: > I am using the builtin handler in app.yaml > > On Oct 14, 2011 12:25 AM, "Greg Darke" wrote: >> >> On Fri, Oct 14, 2011 at 12:10:09AM +1030, Emlyn wrote: >> > I'm trying to give python 2.7 a go, with an app that uses >> > deferred.defer. I'm getting this error when I try to upload it: >> > >> > Error 400: --- begin server output --- >> > Error when loading application configuration: >> > Invalid object: >> > Threadsafe cannot be enabled with CGI handler: >> > $PYTHON_LIB/google/appengine/ext/deferred/handler.py >> > >> > Is there a workaround? Or should I even be using deferred.defer in this >> > context? >> >> The easiest way to solve this problem is to remove the explicit handler >> you are defining, and use the builtin directive as described in >> >> http://code.google.com/appengine/docs/python/config/appconfig.html#Builtin_Handlers >> >> If you are running the deferred handler at non-default url, you will >> need to specify the WSGI application in your app.yaml. This is >> google.appengine.ext.deferred.application >> > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Gae and visualization api
Hi Nick - thanks for the help!
I believe I'm using a relative url, off of localhost. Here's a
representative page:
ESE Pct.
http://www.google.com/jsapi";>
// Load the Visualization API and the ready-made Google table
visualization.
google.load("visualization", "1", {packages:
['corechart,table,annotatedtimeline']});
// Set a callback to run when the API is loaded.
google.setOnLoadCallback(init);
// Send the queries to the data sources.
function init() {
query1 = new google.visualization.Query('ese?url=http://localhost:
8080/gv/11032010_trends_daily_p.csv&pivot=1&dtx=0&tr=0');
query1.send(handleCsvDsResponse1);
}
// Handle the csv data source query response1
function handleCsvDsResponse1(response) {
if (response.isError()) {
alert('Error in query: ' + response.getMessage() + ' ' +
response.getDetailedMessage());
return;
}
var data = response.getDataTable();
var formatter_short = new
google.visualization.DateFormat({formatType: 'short'});
formatter_short.format(data, 0);
var chart = new
google.visualization.LineChart(document.getElementById('chart_div'));
chart.draw(data,{pointSize: 0, title: 'Event/Subevent (ESE)
Analysis - Percentages', hAxis: {slantedTextAngle: 70}});
}
The problem seems to be in the Query. Is there another url entry I
should be using
other than localhost? Also, I'm interested in GAE hosting the
datasource rather than
my host.
Let me know what you think - thanks
John
On Oct 12, 5:33 pm, Nick Johnson wrote:
> Hi,
>
> It sounds like you've embedded the URL of your development environment into
> your code somewhere. Naturally, when deployed, your browser (or the
> Visualization API) can't access files stored on your local machine.
> Instead of hard-coding URLs, you should determine the host header from the
> current request - exactly how to do so depends on your framework - and use
> that. Alternately, you can use relative URLs if the Visualization API
> supports that.
>
> -Nick Johnson
>
>
>
>
>
>
>
>
>
> On Thu, Oct 13, 2011 at 5:39 AM, uover82 wrote:
> > Hi All,
>
> > I'm developing an application using app engine and the google
> > visualization api using java. It currently uses a static file (csv)
> > located in war as a datasource. My app works fine in my local
> > development server but fails with errors like the following after
> > uploading:
>
> > java.io.IOException: Could not fetch URL:http://localhost:8080/...
> >
>
> > Any ideas/thoughts on this?
>
> > Let me know - thanks
>
> > John
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-appengine@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine+unsubscr...@googlegroups.com.
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine?hl=en.
>
> --
> Nick Johnson, Developer Programs Engineer, App Engine
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Python 2.7, threadsafe: true, and deferred.defer
On Fri, Oct 14, 2011 at 12:10:09AM +1030, Emlyn wrote: > I'm trying to give python 2.7 a go, with an app that uses > deferred.defer. I'm getting this error when I try to upload it: > > Error 400: --- begin server output --- > Error when loading application configuration: > Invalid object: > Threadsafe cannot be enabled with CGI handler: > $PYTHON_LIB/google/appengine/ext/deferred/handler.py > > Is there a workaround? Or should I even be using deferred.defer in this > context? The easiest way to solve this problem is to remove the explicit handler you are defining, and use the builtin directive as described in http://code.google.com/appengine/docs/python/config/appconfig.html#Builtin_Handlers If you are running the deferred handler at non-default url, you will need to specify the WSGI application in your app.yaml. This is google.appengine.ext.deferred.application -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Python 2.7, threadsafe: true, and deferred.defer
I'm trying to give python 2.7 a go, with an app that uses deferred.defer. I'm getting this error when I try to upload it: Error 400: --- begin server output --- Error when loading application configuration: Invalid object: Threadsafe cannot be enabled with CGI handler: $PYTHON_LIB/google/appengine/ext/deferred/handler.py Is there a workaround? Or should I even be using deferred.defer in this context? -- Emlyn http://my.syyn.cc - Synchonise Google+, Facebook, WordPress and Google Buzz posts, comments and all. http://point7.wordpress.com - My blog Find me on Facebook and Buzz -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] New Billing: Absolutely make sure you set Max Idle Instances to a fixed value
On 13 October 2011 21:42, Jeff Schnitzer wrote: > Let me summarize: > > You ran an experiment and discovered that GAE left idle instances > running, above and beyond your max-idle-instances setting, for your > application at the time that you ran the test. Ok. Well, that test was in fact designed to force the scheduler's hand. I made the burstiest load that I could (I guess I could have changed the default queue settings to make it even worse). I was testing a conjecture about how the billing works, not the normal performance, and it's wouldn't be right to draw conclusions about normal scheduler behaviour based on that. > Unfortunately there's little reason to believe that this behavior > (lots of free instances) will continue for other applications at other > times; it may just be that you ran at an off-peak period. There's > little reason to believe this behavior will continue as Google > optimizes the scheduler, and a lot of reason to believe that it won't > (free instances are bad for the suddenly-relevant bottom line). (Well, I ran the tests for 20 hours at a time, so not offpeak I think.) In my main real app, changing the scheduler down to min idle instances = 1 more or less halves the idle instances (they're still up at 6 or 7 or so while the active count is below 1). But I don't care too much about that, because it's mostly background processing. A little latency here and there means nothing in that context. > > Should you set the max-idle-instances to a fixed number instead of > "auto"? Depends on your app. If you know how to balance your > traffic, startup latency, and avg request lantecy, then sure. If you > you're more concerned about your bill than your user experience, then > sure. > > Does this mean you don't need to move to a multi-threaded system? No, > because whatever number of idle instances you think you need, you can > divide that by N if you go multithreaded. N is a hard number to > predict because it depends on how much of your app is i/o bound. > "Normal" Java appservers serving typical webapps achieve concurrency > in the hundreds with ease. Ikai once mentioned that the initial > concurrency of Java was hardcoded at 10, but that has probably changed > by now. I don't know what it will be like with Python but really, > it's hard to imagine anything else you can possibly do that will have > an order of magnitude effect on your bill. It can't have an order of magnitude effect on your bill unless you are running with Max Idle Instances on Auto or have set it far too high. > > Jeff Changing Max Idle Instances to 1 for http://my.syyn.cc dropped my projected new billing charges by a factor of 6, with no perceivable performance change (see http://point7.wordpress.com/2011/09/03/the-amazing-story-of-appengine-and-the-two-orders-of-magnitude/ and http://point7.wordpress.com/2011/09/07/appengine-tuning-an-instance-of-success/ For me, to get another order of magnitude would be nice, but is no longer an emergency. And that's something important, because a lot of people rage-quit AppEngine when the new billing appeared, in the usually false belief that they were hosed under the new system. In fact, while I was in the process of introducing AppEngine in my commercial work for our new development, I got some push back specifically around this issue, because people had read that you must have multi-threaded code or else the billing would be sky high (probably reading your article in fact). And the perception was that this would make the whole thing too hard, and why don't we look at some other platform? So I'm *not* saying there's no point to a multi-threaded code base. I'm saying that it is not the only way to fix instance pricing woes, or even the best way (especially from a developer effort POV). And the idea that multi-threading is the best and only way is turning small time developers off the platform. But I'll tell you what; the effect of multi-threading on appengine python app instance pricing warrants a good hard look, with numbers and tables and graphs and whatnot. Reading your stuff in detail has convinced me to put it on my shortlist, particularly with an eye to developing some simple techniques for web apps, for people who would otherwise be intimidated by heavy concurrency work. So I apologise for raising your hackles, and thank you for the interaction, and the motivation to take a look at multi-threaded Python. > > On Thu, Oct 13, 2011 at 3:32 AM, Emlyn wrote: >> On 13 October 2011 19:46, Jeff Schnitzer wrote: >>> I'm afraid you are still confused. >> >> Possibly not. >> >>> You have ignored the entire point >>> of the "max idle instances" slider in the first place. >> >> I might be accused of ignoring its intended function and focusing on >> its practical function. >> >>> GAE keeps idle >>> instances around so that sudden bursts of traffic don't cause users to >>> sit around waiting while your django/spring app spends 5+ seconds >>> loading. >> >> Sure,
Re: [google-appengine] New Billing: Absolutely make sure you set Max Idle Instances to a fixed value
Let me summarize: You ran an experiment and discovered that GAE left idle instances running, above and beyond your max-idle-instances setting, for your application at the time that you ran the test. Ok. Unfortunately there's little reason to believe that this behavior (lots of free instances) will continue for other applications at other times; it may just be that you ran at an off-peak period. There's little reason to believe this behavior will continue as Google optimizes the scheduler, and a lot of reason to believe that it won't (free instances are bad for the suddenly-relevant bottom line). Should you set the max-idle-instances to a fixed number instead of "auto"? Depends on your app. If you know how to balance your traffic, startup latency, and avg request lantecy, then sure. If you you're more concerned about your bill than your user experience, then sure. Does this mean you don't need to move to a multi-threaded system? No, because whatever number of idle instances you think you need, you can divide that by N if you go multithreaded. N is a hard number to predict because it depends on how much of your app is i/o bound. "Normal" Java appservers serving typical webapps achieve concurrency in the hundreds with ease. Ikai once mentioned that the initial concurrency of Java was hardcoded at 10, but that has probably changed by now. I don't know what it will be like with Python but really, it's hard to imagine anything else you can possibly do that will have an order of magnitude effect on your bill. Jeff On Thu, Oct 13, 2011 at 3:32 AM, Emlyn wrote: > On 13 October 2011 19:46, Jeff Schnitzer wrote: >> I'm afraid you are still confused. > > Possibly not. > >> You have ignored the entire point >> of the "max idle instances" slider in the first place. > > I might be accused of ignoring its intended function and focusing on > its practical function. > >> GAE keeps idle >> instances around so that sudden bursts of traffic don't cause users to >> sit around waiting while your django/spring app spends 5+ seconds >> loading. > > Sure, so that's going to matter if traffic is very bursty, and you > need to service it immediately. True of some apps, not true of all by > any means. > > For instance, a lot of my work is with systems whose major work is in > the background in tasks. Latency is a non-issue there. > > Also, that's only loosely related to the Max Idle Instances setting. > >> Will turning max-idle-instances down to 1 reduce your bill? Of >> course. But only at the expense of user experience. > > Only true if I have no idle instances ready in reality. > >> The second user >> to arrive is going to wait for five seconds. Maybe you'll get lucky >> and Google will actually leave your idle instances running for longer >> than you've decided you're willing to pay for. Or maybe they won't >> and your users will wonder why your site is broken and go to your >> competitor's site instead. If you have a lot of idle instances >> running, it's because you're lucky that memory pressure hasn't evicted >> your spares, not because you're paying for service. > > True. I've tuned based on testing, not based on theory. The current > scheduler seems to kick off instances like crazy, no matter how you > set Max Idle Instances. So having enough idle instances to service > requests isn't currently a problem. > > Note that I'm not actually recommending setting Max Idle Instances to > 1, but setting it to a fixed number. How high? It depends on the size > of traffic bursts you absolutely must service immediately. How big is > that number? Up to your situation. I bet it's smaller than infinity > (which is what Automatic means). > >> My point in the "unofficial faq" is that this slider does not have a >> single "correct" setting, and if there was, it would certainly not be >> 1. You're basically paying for RAM. If you have a single-threaded >> server, your 128 MB frontend will run one concurrent request at a >> time. If you have an efficient multi-threaded server, your 128 MB >> frontend can serve an order of magnitude more requests - possibly more >> depending on how I/O bound your process is. > > I haven't seen anyone's numbers from the java side (and I haven't > tried python 2.7 yet); is there actual data floating around on how > many requests you can expect to serve concurrently per instance, using > multi-threading? > > I guess what's crucial here is, if you use multi-threaded request > processing code, do you see a serious decrease in the number of > instances the scheduler wants to kick off? > > - If Max Idle Instances is set to Automatic, a serious decrease will > mean a serious billing decrease, because you are paying for all of > those instances > - If Max Idle Instances is set to a fixed number which lands Active > Instances + Max Idle Instances lower than Total Instances, then you > are paying for that setting + your activity, ie: no difference even if > your code is multi-threaded. > > Am I
Re: [google-appengine] App Engine Weekly Community Update #11
I've made some (rough) updates. Jeff On Wed, Oct 12, 2011 at 12:25 PM, Johan Euphrosine wrote: > This should now be fixed, let me know if it works. > On Wed, Oct 12, 2011 at 8:09 PM, Jeff Schnitzer wrote: >> >> I can't seem to edit or comment in any way. >> >> Jeff >> >> On Wed, Oct 12, 2011 at 8:18 AM, Johan Euphrosine >> wrote: >> > Hi, >> > Thanks a lot both for your feedback, I think this topic deserve a proper >> > article covering all the view / technique about caching, based on what >> > you >> > said I started to draft one here: >> > >> > https://docs.google.com/document/d/16OEn3-7pWBv6PopKjk9Onk9cF1B0PtOQcyDd3q6YAc0/edit?hl=en_US >> > Feel free to contribute or add comments. >> > On Sun, Oct 9, 2011 at 12:35 AM, Brandon Wirtz >> > wrote: >> >> >> >> It also doesn't talk about using Instance memory, which is a HUGE part >> >> of >> >> my >> >> caching strategy. Even from a beginners guide I'm not sure this is an >> >> article I would promote, I think more people will be unhappy with this >> >> solution than will benefit from it. >> >> >> >> >> >> -Original Message- >> >> From: google-appengine@googlegroups.com >> >> [mailto:google-appengine@googlegroups.com] On Behalf Of Jeff Schnitzer >> >> Sent: Saturday, October 08, 2011 2:21 PM >> >> To: google-appengine@googlegroups.com >> >> Subject: Re: [google-appengine] App Engine Weekly Community Update #11 >> >> >> >> On Sat, Oct 8, 2011 at 5:16 AM, Johan Euphrosine >> >> wrote: >> >> > >> >> > Nice Article about caching strategies for App Engine apps by +Chris >> >> > Bunch >> >> > >> >> > http://www.byzantinereality.com/2011/8/Two-Caching-Strategies-for-App- >> >> > Engine-apps >> >> >> >> That page doesn't seem to allow comments, so I'll have to rant here: >> >> >> >> This caching strategy is a terrible idea. Trust me, it's taken a long >> >> time >> >> to work out the kinks in Objectify's transparent memcache. This >> >> caching >> >> strategy will go out of sync under contention. You need to use CAS >> >> operations to avoid race conditions. >> >> >> >> Jeff >> >> >> >> -- >> >> You received this message because you are subscribed to the Google >> >> Groups >> >> "Google App Engine" group. >> >> To post to this group, send email to google-appengine@googlegroups.com. >> >> To unsubscribe from this group, send email to >> >> google-appengine+unsubscr...@googlegroups.com. >> >> For more options, visit this group at >> >> http://groups.google.com/group/google-appengine?hl=en. >> >> >> >> >> >> -- >> >> You received this message because you are subscribed to the Google >> >> Groups >> >> "Google App Engine" group. >> >> To post to this group, send email to google-appengine@googlegroups.com. >> >> To unsubscribe from this group, send email to >> >> google-appengine+unsubscr...@googlegroups.com. >> >> For more options, visit this group at >> >> http://groups.google.com/group/google-appengine?hl=en. >> >> >> > >> > >> > >> > -- >> > Johan Euphrosine (proppy) >> > Developer Programs Engineer >> > Google Developer Relations >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups >> > "Google App Engine" group. >> > To post to this group, send email to google-appengine@googlegroups.com. >> > To unsubscribe from this group, send email to >> > google-appengine+unsubscr...@googlegroups.com. >> > For more options, visit this group at >> > http://groups.google.com/group/google-appengine?hl=en. >> > >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > > > -- > Johan Euphrosine (proppy) > Developer Programs Engineer > Google Developer Relations > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] New Billing: Absolutely make sure you set Max Idle Instances to a fixed value
On 13 October 2011 19:46, Jeff Schnitzer wrote: > I'm afraid you are still confused. Possibly not. > You have ignored the entire point > of the "max idle instances" slider in the first place. I might be accused of ignoring its intended function and focusing on its practical function. > GAE keeps idle > instances around so that sudden bursts of traffic don't cause users to > sit around waiting while your django/spring app spends 5+ seconds > loading. Sure, so that's going to matter if traffic is very bursty, and you need to service it immediately. True of some apps, not true of all by any means. For instance, a lot of my work is with systems whose major work is in the background in tasks. Latency is a non-issue there. Also, that's only loosely related to the Max Idle Instances setting. > Will turning max-idle-instances down to 1 reduce your bill? Of > course. But only at the expense of user experience. Only true if I have no idle instances ready in reality. > The second user > to arrive is going to wait for five seconds. Maybe you'll get lucky > and Google will actually leave your idle instances running for longer > than you've decided you're willing to pay for. Or maybe they won't > and your users will wonder why your site is broken and go to your > competitor's site instead. If you have a lot of idle instances > running, it's because you're lucky that memory pressure hasn't evicted > your spares, not because you're paying for service. True. I've tuned based on testing, not based on theory. The current scheduler seems to kick off instances like crazy, no matter how you set Max Idle Instances. So having enough idle instances to service requests isn't currently a problem. Note that I'm not actually recommending setting Max Idle Instances to 1, but setting it to a fixed number. How high? It depends on the size of traffic bursts you absolutely must service immediately. How big is that number? Up to your situation. I bet it's smaller than infinity (which is what Automatic means). > My point in the "unofficial faq" is that this slider does not have a > single "correct" setting, and if there was, it would certainly not be > 1. You're basically paying for RAM. If you have a single-threaded > server, your 128 MB frontend will run one concurrent request at a > time. If you have an efficient multi-threaded server, your 128 MB > frontend can serve an order of magnitude more requests - possibly more > depending on how I/O bound your process is. I haven't seen anyone's numbers from the java side (and I haven't tried python 2.7 yet); is there actual data floating around on how many requests you can expect to serve concurrently per instance, using multi-threading? I guess what's crucial here is, if you use multi-threaded request processing code, do you see a serious decrease in the number of instances the scheduler wants to kick off? - If Max Idle Instances is set to Automatic, a serious decrease will mean a serious billing decrease, because you are paying for all of those instances - If Max Idle Instances is set to a fixed number which lands Active Instances + Max Idle Instances lower than Total Instances, then you are paying for that setting + your activity, ie: no difference even if your code is multi-threaded. Am I right that you are proposing that multi-threaded request handling will seriously lower your total instance count (in reality, not just in theory), and that you will therefore pay much less, with Max Idle Instances set to Automatic, than a single threaded app with a low Max Idle Instances setting would pay under the same load? > I'm not saying you shouldn't set your max-idle-instances to 1 right > now. For the time being, it *might* give you great performance at a > great price. I'm just saying this isn't any kind of a real solution - > you won't be able to complain when your latency goes to hell. Maximizing future complaint possibilities is not the same as actually lowering your bill. Right now, it's absolutely a great solution to set that number low. Everyone should do it. Everyone should also monitor their total instance count and keep an eye out for a change in scheduler behaviour that will render this advice incorrect, of course. But you should do that anyway, because Google could change their scheduler's behaviour without warning. If Max Idle Instances is set to Automatic, you'd pay real money for that. > Put it this way: If your multi-threaded system has an avg concurrency > of 10, setting max-idle-instance to 1 is the equivalent of setting it > to 10 on a single-threaded system. If you care about your user > experience, you want that number high. > > Jeff Sure, if you can get average concurrency of 10, then that's excellent. But that's a performance consideration, not a price consideration. Note that we are in a context where people are not complaining about poor performance, they're complaining about increased costs. Also, testing would indicate that moving Max Idl
[google-appengine] Re: Large number of datastore puts
Thanks a lot for the reply -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/8hHDnGaLZzkJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] New Billing: Absolutely make sure you set Max Idle Instances to a fixed value
I'm afraid you are still confused. You have ignored the entire point of the "max idle instances" slider in the first place. GAE keeps idle instances around so that sudden bursts of traffic don't cause users to sit around waiting while your django/spring app spends 5+ seconds loading. Will turning max-idle-instances down to 1 reduce your bill? Of course. But only at the expense of user experience. The second user to arrive is going to wait for five seconds. Maybe you'll get lucky and Google will actually leave your idle instances running for longer than you've decided you're willing to pay for. Or maybe they won't and your users will wonder why your site is broken and go to your competitor's site instead. If you have a lot of idle instances running, it's because you're lucky that memory pressure hasn't evicted your spares, not because you're paying for service. My point in the "unofficial faq" is that this slider does not have a single "correct" setting, and if there was, it would certainly not be 1. You're basically paying for RAM. If you have a single-threaded server, your 128 MB frontend will run one concurrent request at a time. If you have an efficient multi-threaded server, your 128 MB frontend can serve an order of magnitude more requests - possibly more depending on how I/O bound your process is. I'm not saying you shouldn't set your max-idle-instances to 1 right now. For the time being, it *might* give you great performance at a great price. I'm just saying this isn't any kind of a real solution - you won't be able to complain when your latency goes to hell. Put it this way: If your multi-threaded system has an avg concurrency of 10, setting max-idle-instance to 1 is the equivalent of setting it to 10 on a single-threaded system. If you care about your user experience, you want that number high. Jeff On Thu, Oct 13, 2011 at 1:21 AM, Emlyn wrote: > np Johan. > > I was confused earlier on by posts such as > > http://blorn.com/post/10013293300/the-unofficial-google-app-engine-price-change-faq > > which focuses on multithreading to get pricing down, which I think is > just wrong. There are lots of good reasons to write multithreaded > code, but AppEngine pricing isn't really one of them. > > On 13 October 2011 18:22, Johan Euphrosine wrote: >> Hi Emlyn, >> >> Thanks for sharing those articles, it is very nice that you were able >> to backup the billing formula with hard facts. >> >> As it was discussed in the groups during the pricing model change the >> billing formula under the new model will be: >> billable_instances_rate = min(active_instances_rate + >> max_idle_instances, total_instances_rate) >> >> where in the dashboard: >> - active_instances_rates is the yellow line >> - total_instances_rate is the blue line >> - max_idle_instances is the upper bound of "Idle Instances" performance >> settings >> >> If you set max_idle_instances to automatic, it's equivalent to setting >> it to a very large number making the formula essentially become: >> billable_instances_rate = total_instances_rate >> >> See the following threads where Jon McAlister commented about the >> billing formula: >> https://groups.google.com/d/msg/google-appengine/zuRXAphGnPk/UiTgTIIesL0J >> https://groups.google.com/d/msg/google-appengine/W-17IhgwrLI/05Wti7I39EUJ >> https://groups.google.com/d/msg/google-appengine/T-dJtXmOO8U/npM69XZAJFcJ >> >> On Wed, Oct 12, 2011 at 3:10 PM, Emlyn wrote: >>> I've been testing this hypothesis: >>> >>> Hypothesis: Ignoring the 15 minute cost for spinning up new instances, >>> the price we pay is the moment by moment minimum of (total instances) >>> and (active instances + Max Idle Instances). If Max Idle Instances is >>> set to Automatic, then we pay for the moment by moment total >>> instances. >>> >>> It holds. That is, if you leave the default Max Idle Instances setting >>> (Automatic), you'll be billed for every bit of instance time the >>> scheduler chooses to run. If you set it to a fixed number, you'll have >>> a fixed cost cap based on the actual work you are doing (a lot more >>> like cpu time), in most cases a lot lower. >>> >>> Here are a couple of posts in detail, with graphs, billing numbers, >>> pictures! >>> >>> The Spiny Norman Test >>> http://appenginedevelopment.blogspot.com/2011/10/spiny-norman-test.html >>> >>> Go Spiny Norman, Go >>> http://appenginedevelopment.blogspot.com/2011/10/go-spiny-norman-go.html >>> >>> -- >>> Emlyn >>> >>> http://my.syyn.cc - Synchonise Google+, Facebook, WordPress and Google >>> Buzz posts, >>> comments and all. >>> http://point7.wordpress.com - My blog >>> Find me on Facebook and Buzz >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Google App Engine" group. >>> To post to this group, send email to google-appengine@googlegroups.com. >>> To unsubscribe from this group, send email to >>> google-appengine+unsubscr...@googlegroups.com. >>> For more options, visit this group at >>
Re: [google-appengine] New Billing: Absolutely make sure you set Max Idle Instances to a fixed value
np Johan. I was confused earlier on by posts such as http://blorn.com/post/10013293300/the-unofficial-google-app-engine-price-change-faq which focuses on multithreading to get pricing down, which I think is just wrong. There are lots of good reasons to write multithreaded code, but AppEngine pricing isn't really one of them. On 13 October 2011 18:22, Johan Euphrosine wrote: > Hi Emlyn, > > Thanks for sharing those articles, it is very nice that you were able > to backup the billing formula with hard facts. > > As it was discussed in the groups during the pricing model change the > billing formula under the new model will be: > billable_instances_rate = min(active_instances_rate + > max_idle_instances, total_instances_rate) > > where in the dashboard: > - active_instances_rates is the yellow line > - total_instances_rate is the blue line > - max_idle_instances is the upper bound of "Idle Instances" performance > settings > > If you set max_idle_instances to automatic, it's equivalent to setting > it to a very large number making the formula essentially become: > billable_instances_rate = total_instances_rate > > See the following threads where Jon McAlister commented about the > billing formula: > https://groups.google.com/d/msg/google-appengine/zuRXAphGnPk/UiTgTIIesL0J > https://groups.google.com/d/msg/google-appengine/W-17IhgwrLI/05Wti7I39EUJ > https://groups.google.com/d/msg/google-appengine/T-dJtXmOO8U/npM69XZAJFcJ > > On Wed, Oct 12, 2011 at 3:10 PM, Emlyn wrote: >> I've been testing this hypothesis: >> >> Hypothesis: Ignoring the 15 minute cost for spinning up new instances, >> the price we pay is the moment by moment minimum of (total instances) >> and (active instances + Max Idle Instances). If Max Idle Instances is >> set to Automatic, then we pay for the moment by moment total >> instances. >> >> It holds. That is, if you leave the default Max Idle Instances setting >> (Automatic), you'll be billed for every bit of instance time the >> scheduler chooses to run. If you set it to a fixed number, you'll have >> a fixed cost cap based on the actual work you are doing (a lot more >> like cpu time), in most cases a lot lower. >> >> Here are a couple of posts in detail, with graphs, billing numbers, pictures! >> >> The Spiny Norman Test >> http://appenginedevelopment.blogspot.com/2011/10/spiny-norman-test.html >> >> Go Spiny Norman, Go >> http://appenginedevelopment.blogspot.com/2011/10/go-spiny-norman-go.html >> >> -- >> Emlyn >> >> http://my.syyn.cc - Synchonise Google+, Facebook, WordPress and Google >> Buzz posts, >> comments and all. >> http://point7.wordpress.com - My blog >> Find me on Facebook and Buzz >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> >> > > > > -- > Johan Euphrosine (proppy) > Developer Programs Engineer > Google Developer Relations > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Emlyn http://my.syyn.cc - Synchonise Google+, Facebook, WordPress and Google Buzz posts, comments and all. http://point7.wordpress.com - My blog Find me on Facebook and Buzz -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] New Billing: Absolutely make sure you set Max Idle Instances to a fixed value
Hi Emlyn, Thanks for sharing those articles, it is very nice that you were able to backup the billing formula with hard facts. As it was discussed in the groups during the pricing model change the billing formula under the new model will be: billable_instances_rate = min(active_instances_rate + max_idle_instances, total_instances_rate) where in the dashboard: - active_instances_rates is the yellow line - total_instances_rate is the blue line - max_idle_instances is the upper bound of "Idle Instances" performance settings If you set max_idle_instances to automatic, it's equivalent to setting it to a very large number making the formula essentially become: billable_instances_rate = total_instances_rate See the following threads where Jon McAlister commented about the billing formula: https://groups.google.com/d/msg/google-appengine/zuRXAphGnPk/UiTgTIIesL0J https://groups.google.com/d/msg/google-appengine/W-17IhgwrLI/05Wti7I39EUJ https://groups.google.com/d/msg/google-appengine/T-dJtXmOO8U/npM69XZAJFcJ On Wed, Oct 12, 2011 at 3:10 PM, Emlyn wrote: > I've been testing this hypothesis: > > Hypothesis: Ignoring the 15 minute cost for spinning up new instances, > the price we pay is the moment by moment minimum of (total instances) > and (active instances + Max Idle Instances). If Max Idle Instances is > set to Automatic, then we pay for the moment by moment total > instances. > > It holds. That is, if you leave the default Max Idle Instances setting > (Automatic), you'll be billed for every bit of instance time the > scheduler chooses to run. If you set it to a fixed number, you'll have > a fixed cost cap based on the actual work you are doing (a lot more > like cpu time), in most cases a lot lower. > > Here are a couple of posts in detail, with graphs, billing numbers, pictures! > > The Spiny Norman Test > http://appenginedevelopment.blogspot.com/2011/10/spiny-norman-test.html > > Go Spiny Norman, Go > http://appenginedevelopment.blogspot.com/2011/10/go-spiny-norman-go.html > > -- > Emlyn > > http://my.syyn.cc - Synchonise Google+, Facebook, WordPress and Google > Buzz posts, > comments and all. > http://point7.wordpress.com - My blog > Find me on Facebook and Buzz > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Johan Euphrosine (proppy) Developer Programs Engineer Google Developer Relations -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
