Re: [google-appengine] Python 3 local development: fast startup or static assets, choose one?

[Ryan Barrett]

thanks for the reply!

On Fri, Nov 8, 2019 at 2:40 PM 'Andrew Gorcester' via Google App Engine <
google-appengine@googlegroups.com> wrote:

> I would recommend trying to solve the first problem and serving static
> files locally.
>

understood, but using a different static file server is not great. it makes
local development diverge significantly from production. local no longer
uses or tests app.yaml's static file handlers, but prod does, so bugs will
eventually pop up that only happen in production, and are thus harder to
test for and debug.

for that reason, i'm actually going the other way and using dev_appserver
right now, soi still have some confidence that my app will behave more or
less the same in production that it does locally. the minute long delay on
startup is definitely painful though.


At first glance I think the most straightforward ways to do that would be
> to either:
>
> - Use the Flask development server instead of gunicorn and leverage
> Flask's integrated /static directory
> https://flask.palletsprojects.com/en/1.1.x/tutorial/static/
> - If you prefer to use gunicorn or you need more serving options, add
> WhiteNoise middleware to your app; ideally, have your app detect whether
> it's running in production or in development mode and conditionally add
> WhiteNoise in development only: http://whitenoise.evans.io/en/stable/
>
>
> On Fri, Nov 8, 2019 at 2:21 PM Ryan B  wrote:
>
>> Hi all! My Python 3 topic for the day is local development. How are you
>> all doing it? Any tips?
>>
>> The two main ways seem to be a third party server, eg Flask or Gunicorn,
>> or dev_appserver.py. I have both working, but each one has a fatal flaw
>> that makes it basically unusable for me so far.
>>
>> When I run my app with eg gunicorn -b :8080 app:application - the way
>> Google recommends
>> 
>> - it works, but doesn't serve any assets from static file handlers in my
>> app.yaml. This is widely reported on StackOverflow
>> 
>> and elsewhere. Responses are often some variation of, "why do you need
>> static assets for dynamic requests?" The obvious reason is that pretty much
>> all webapps use images, CSS, and/or JS, among other static files. They're
>> generally not usable without those assets.
>>
>> When I run my app with  dev_appserver.py, it creates a new virtualenv
>> and installs all of my dependencies from requirements.txt. This is nice,
>> but also extremely slow for any nontrivial app. My current medium sized app
>> has 155 packages, and even when they're all cached locally, installing them
>> takes around 60s. I often switch between different apps and other stacks,
>> so I can't often keep dev_appserver running for hours or days on end to
>> avoid this. (I've filed this feature request for dev_appserver to reuse
>> an existing virtualenv.
>> )
>>
>> Let me know if you have any ideas!
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to google-appengine+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/google-appengine/eff3cf33-3a7b-4cc4-8b8c-2a46b2c56c45%40googlegroups.com
>> 
>> .
>>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Google App Engine" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/google-appengine/BJDE8y2KISM/unsubscribe
> .
> To unsubscribe from this group and all its topics, send an email to
> google-appengine+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/google-appengine/CANtYgF%2BPRh3vU%3Dpp3708sRAFE8%2BYxC%3Dm%2BmC8oGVqHyLw3yKBhw%40mail.gmail.com
> 
> .
>


-- 
https://snarfed.org/

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/CA%2BcaGh-8rd4NVj2HmcS0t0PRAAJ%2BiWo3huzjXcbJSDxyJtsAPw%40mail.gmail.com.

Re: [google-appengine] Instagram blocking App Engine's urlfetch/sockets IP block

[Ryan Barrett]

On Fri, May 6, 2016 at 9:47 AM, 'Nick (Cloud Platform Support)' via
Google App Engine  wrote:
> Hey Ryan,
>
> Glad to be of assistance, and I really want to get to the bottom of this.
> Reviewing the infrastructure used by UrlFetch, this absolutely does make
> sense, when we consider this tantalizing detail from the documentation:
>
>> The URL Fetch service uses an HTTP/1.1 compliant proxy to fetch the
>> result.

yup. if urlfetch is behind a small or even medium sized set of VIPs or
IP blocks, and instagram rate limits their www based on IP (individual
or block), that's it. you could data mine urlfetch's logs and find the
offending app(s), if any, and play the abuse whack-a-mole game,
but...meh.

> which sort of proxy configuration you're using, the type of proxy, the average
> request load, etc.?

sure! it's dirt simple, just apache mod_proxy with these lines in httpd.conf:

SSLProxyEngine on

ProxyPass "https://www.instagram.com/";


my load is miniscule and pretty constant, roughly 1-2qm on average.
most of that is to profile URLs (evenly spread across ~500 users), the
rest to individual photo URLs like eg
https://www.instagram.com/p/BE4xLpmABFz/.

i used to do 3-5x that much before i throttled down recently. i
haven't tried, but i expect i could ramp back up to that on the
reverse proxy and not get 429ed.


> On Thursday, May 5, 2016 at 6:36:12 PM UTC-4, Ryan Barrett wrote:
>>
>> thanks for going above and beyond, nick! much appreciated. i'm
>> currently working around it by using a reverse proxy outside of app
>> engine, so that my requests are charged to a different IP and isolated
>> from other app engine apps. glad this info is here now for other
>> people too.
>>
>>
>> On Thu, May 5, 2016 at 2:13 PM, 'Nick (Cloud Platform Support)' via
>> Google App Engine  wrote:
>> >
>> > After some extensive testing, I've determined that the 429 you're
>> > receiving
>> > is expected behaviour from instagram, and it does relate to a windowing
>> > average, although it may not be the same as that published in their
>> > documentation for APIs. After sending a few thousand requests in a span
>> > of
>> > ~15 seconds, I began to receive 429 responses, with some 200's
>> > intermixed.
>> >
>> > Cheers,
>> >
>> > Nick
>> > Cloud Platform Community Support
>> >
>> > On Wednesday, May 4, 2016 at 3:21:46 PM UTC-4, Ryan Barrett wrote:
>> >>
>> >> On Wed, May 4, 2016 at 12:02 PM, 'Nick (Cloud Platform Support)' via
>> >> Google App Engine  wrote:
>> >> > So, you're attempting merely to fetch http://www.instagram.com/, and
>> >> > you
>> >> > receive 429 on the first request, and you're not launching many other
>> >> > requests at the same time? It seems odd that a rate-limit response
>> >> > would
>> >> > come without a condition being reached requiring rate-limiting... Let
>> >> > me
>> >>
>> >> i'm actually fetching profile URLs, not the front page. eg `import
>> >> urllib2; urllib2.urlopen('https://www.instagram.com/kevin/')` in
>> >> https://shell-hrd.appspot.com/ gets 429ed even though i'm not fetching
>> >> that particular URL in any of my apps.
>> >>
>> >> it definitely seems odd, agreed. i only suspect rate limiting/blocking
>> >> at the IP level because i exhaused the other obvious causes. i'd be
>> >> happy to be proven wrong!
>> >>
>> >>
>> >> > know what you think in your reply.
>> >> >
>> >> > Cheers,
>> >> >
>> >> > Nick
>> >> > Cloud Platform Community Support
>> >> >
>> >> > On Wednesday, May 4, 2016 at 1:09:35 PM UTC-4, Ryan Barrett wrote:
>> >> >>
>> >> >> thanks for the replies! i should have emphasized that this is for
>> >> >> www.instagram.com, not the API. API requests are working fine.
>> >> >>
>> >> >> you're right that IP blocking wouldn't usually be the first culprit
>> >> >> in
>> >> >> general, especially for 429s. i tried from a few different apps,
>> >> >> though,
>> >> >> including shell-hrd (log in my first post), which pretty much never
>> >> >> uses
>> >> >> urlfetch otherwise based on its quota numbers, so i doubt it's
>&

Re: [google-appengine] Instagram blocking App Engine's urlfetch/sockets IP block

[Ryan Barrett]

thanks for the replies! i should have emphasized that this is for 
www.instagram.com, not the API. API requests are working fine.

you're right that IP blocking wouldn't usually be the first culprit in 
general, especially for 429s. i tried from a few different apps, though, 
including shell-hrd (log in my first post), which pretty much never uses 
urlfetch otherwise based on its quota numbers, so i doubt it's User-Agent 
blocking. i tried an entirely new www.instagram.com URL and still got a 
429, so it's probably not specific URLs, at least due to my own traffic. 
and i can fetch the same URL fine from my local machine. hence my IP 
suspicion.

i've already worked around this, so it's not urgent. just figured you all 
might want to know. thanks again!

On Monday, May 2, 2016 at 10:52:11 AM UTC-7, Nick (Cloud Platform Support) 
wrote:
>
> Hey Ryan,
>
> I'm unsure that this indicates that App Engine specifically is being 
> rate-limited. It's likely that the 429 response is directly related to the 
> frequency with which you're making requests, regardless of the origin of 
> those requests. While not impossible, I suppose, it would be surprising if 
> they were keeping track of App Engine IP ranges and applying a different 
> rate-limit, and would require some thorough A/B testing to prove. So, I 
> recommend just checking their documentation 
> <https://www.instagram.com/developer/limits/> or, if the rate-limit is 
> undocumented, benchmarking to attempt to determine it, and try to fly under 
> it. Generally, exponential-backoff 
> <https://en.wikipedia.org/wiki/Exponential_backoff> is a good tactic when 
> dealing with rate-limiting.
>
> Sincerely,
>
> Nick
> Cloud Platform Community Support
>
> On Monday, May 2, 2016 at 11:57:15 AM UTC-4, Nickolas Daskalou wrote:
>>
>> Hi Ryan,
>>
>> It seems to be working fine for us (SocialPage.me 
>> <https://socialpage.me/>).
>>
>> Are you accessing their API using separate access tokens for each user?
>>
>> Nick
>>
>>
>> On 2 May 2016 at 14:30, Ryan Barrett > 
>> wrote:
>>
>>> hi all! just FYI, it looks like Instagram is blocking/rate limiting App 
>>> Engine's IPs from fetching www.instagram.com, both urlfetch and 
>>> sockets, across apps. e.g. this session from 
>>> https://shell-hrd.appspot.com/ :
>>>
>>> >>> urllib2.urlopen('https://www.instagram.com/snarfed/')
>>> Traceback (most recent call last):
>>> ...
>>>   File 
>>> "/base/data/home/runtimes/python/python_dist/lib/python2.5/urllib2.py", 
>>> line 506, in http_error_default
>>> raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
>>> HTTPError: HTTP Error 429: Unknown
>>>
>>> it's not 100% consistent - i occasionally see requests make it through - 
>>> but the majority get 429ed.
>>>
>>> not holding my breath, but i figured you all might want to know, 
>>> especially in case cloud support people have lines of communication open 
>>> with instagram/facebook for this kind of thing.
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Google App Engine" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to google-appengi...@googlegroups.com .
>>> To post to this group, send email to google-a...@googlegroups.com 
>>> .
>>> Visit this group at https://groups.google.com/group/google-appengine.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/google-appengine/be7f6ead-fe34-45c4-9ee0-00956b5f89de%40googlegroups.com
>>>  
>>> <https://groups.google.com/d/msgid/google-appengine/be7f6ead-fe34-45c4-9ee0-00956b5f89de%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/df130e09-f87b-40a7-bd4c-9ab0dc6fad0e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Instagram blocking App Engine's urlfetch/sockets IP block

[Ryan Barrett]

hi all! just FYI, it looks like Instagram is blocking/rate limiting App 
Engine's IPs from fetching www.instagram.com, both urlfetch and sockets, 
across apps. e.g. this session from https://shell-hrd.appspot.com/ :

>>> urllib2.urlopen('https://www.instagram.com/snarfed/')
Traceback (most recent call last):
...
  File 
"/base/data/home/runtimes/python/python_dist/lib/python2.5/urllib2.py", 
line 506, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 429: Unknown

it's not 100% consistent - i occasionally see requests make it through - 
but the majority get 429ed.

not holding my breath, but i figured you all might want to know, especially 
in case cloud support people have lines of communication open with 
instagram/facebook for this kind of thing.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/be7f6ead-fe34-45c4-9ee0-00956b5f89de%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: twitter API suddenly returning error 32 “Could not authenticate you.” only in prod GAE

[Ryan Barrett]

On Friday, May 1, 2015 at 4:13:03 PM UTC-7, Waleed wrote:
>
> My app was also rolled back to 1.9.19 yesterday. Then later it was back on 
> 1.9.20, but the error seems to have been fixed. I also got email 
> confirmations from customers that they're not seeing this problem any more. 
> Great team work. 
>

confirmed! same with my app. thanks for the heads up! 

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/cae65195-1706-472e-bd64-1a3632611ad1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: twitter API suddenly returning error 32 “Could not authenticate you.” only in prod GAE

[Ryan Barrett]

...on an unrelated note, 1.9.20 was *way* better with utilization and 
scheduling instances efficiently for my app. app engine team, i hope that part 
survives!

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/246911e0-0ff0-4ecf-b4aa-4f056a1f3e15%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: twitter API suddenly returning error 32 “Could not authenticate you.” only in prod GAE

[Ryan Barrett]

On Thursday, April 30, 2015 at 1:10:23 PM UTC-7, Waleed wrote:
>
> Another update. It seems that the issue is happening in App Engine release 
> 1.9.20. Some of the instances in my app are on this release, and others are 
> on the previous 1.9.19. We didn't a full scan, but based on manual 
> inspection it seems that all the errors are happening on requests that are 
> on the 1.9.20 release. 
>

nice sleuthing! i can confirm that app engine 1.9.19 doesn't exhibit this 
behavior. my app was rolled back to 1.9.19 earlier today, and i'm now able 
to post a tweet with the problematic characters again.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/af18fba1-e46b-4705-be37-6dd18b45e137%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: twitter API suddenly returning error 32 “Could not authenticate you.” only in prod GAE

[Ryan Barrett]

On Wednesday, April 29, 2015 at 11:29:34 AM UTC-7, Ryan Barrett wrote:
>
> as another data point, there may be more to it than just punctuation 
> characters. i see the error with pretty much any tweet content and 
> username. i reproduced it just now by attempting to tweet "foo" (just those 
> three characters) as @schnarfed. sigh.
>

sorry guys, disregard this, i was mistaken. i was accidentally including 
parentheses in the tweet text after all. tweeting just 'foo' does work for 
me, so it probably is indeed the difference in URL-encoding those 
characters.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/4352ac3a-2278-4f65-ae77-bb7360f55512%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: twitter API suddenly returning error 32 “Could not authenticate you.” only in prod GAE

[Ryan Barrett]

thanks for the comprehensive debugging, Dave and David! it definitely 
sounds like new header munging on app engine's part may be the problem. 
Dave, your idea of sending the API calls to a different endpoint and 
examining the headers sounds like a great lead. i'm looking forward to the 
results!
as another data point, there may be more to it than just punctuation 
characters. i see the error with pretty much any tweet content and 
username. i reproduced it just now by attempting to tweet "foo" (just those 
three characters) as @schnarfed. sigh.

On Wednesday, April 29, 2015 at 9:58:38 AM UTC-7, Dave Loomer wrote:
>
> (what I was getting it with the previous post is: if any of you need this 
> for your day jobs, you are welcome to try the same yourselves in the 
> meantime ;) )
>
> On Wednesday, April 29, 2015 at 11:56:52 AM UTC-5, Dave Loomer wrote:
>>
>> All: I won't have time to do this in the next several hours (my app 
>> engine app is not my day job), but I think tonight I'll try changing the 
>> api.twitter.com endpoint to point to a URL I own, and then examine the 
>> URLs / query strings and headers received from my prod vs. dev code to see 
>> if the encoding is any different.
>>
>> I figure that could help point to whether it's a urlfetch issue. Only 
>> other thing I can think of, if it's not urlfetch, is that the oauth 
>> signatures are being generated differently in prod vs. dev, due to a 
>> difference in some built-in python library on the prod server, and that 
>> will be much more difficult to debug since the timestamp and nonce prevent 
>> you from comparing dev vs prod requests character-by-character.
>>
>>
>> On Wednesday, April 29, 2015 at 9:36:17 AM UTC-5, Ryan Barrett wrote:
>>>
>>> hi all! a number of us have started seeing twitter API errors in prod 
>>> GAE in the last week or so. write calls and some search calls are returning 
>>> HTTP 401 {"errors":[{"code":32,"message":"Could not authenticate 
>>> you."}]} for all users with no code changes on our end. the 
>>> particularly odd part is that the same calls with the same code, twitter 
>>> app key/secret, and even *the exact same user access token key/secret* 
>>> work fine in dev_appserver.
>>>
>>> we're on python and mostly using tweepy. our current theory is that 
>>> twitter has blacklisted or graylisted app engine's external facing IPs. can 
>>> you think of any other ideas?
>>>
>>> more details in this twitter dev forum post. 
>>> <https://twittercommunity.com/t/post-to-statuses-update-json-started-hitting-error-32-could-not-authenticate-you-with-no-code-changes/36495>
>>>  
>>> thanks in advance!
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/75642a3b-ec93-4a89-a549-2fd9280bf410%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] twitter API suddenly returning error 32 “Could not authenticate you.” only in prod GAE

[Ryan Barrett]

 

hi all! a number of us have started seeing twitter API errors in prod GAE 
in the last week or so. write calls and some search calls are returning 
HTTP 401 {"errors":[{"code":32,"message":"Could not authenticate you."}]} 
for all users with no code changes on our end. the particularly odd part is 
that the same calls with the same code, twitter app key/secret, and even *the 
exact same user access token key/secret* work fine in dev_appserver.

we're on python and mostly using tweepy. our current theory is that twitter 
has blacklisted or graylisted app engine's external facing IPs. can you 
think of any other ideas?

more details in this twitter dev forum post. 

 
thanks in advance!

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/28123aa6-30f5-4102-be74-16088af87b22%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] 1.9.19 raised string property limit from 500 to 1500 chars

[Ryan Barrett]

looks like yesterday's 1.9.19 release raised the limit on string properties 
in the datastore from 500 to 1500 chars. awesome!

from google/appengine/api/datastore_types.py:

_MAX_STRING_LENGTH = 1500

tested in https://shell-27.appspot.com/ :

>>> from google.appengine.ext import db
>>> class Foo(db.Expando):
  pass
>>> key = Foo(long='x'*1490).put()
>>> foo = Foo.get(key)
>>> len(foo.long)
1490

app engine team, might be worth adding this to the release notes and docs?

https://code.google.com/p/googleappengine/wiki/SdkReleaseNotes
https://cloud.google.com/appengine/docs/python/ndb/properties#types

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/846b9ec2-2621-450b-806a-d7d8c9175d96%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: 1.9.13 causing some requests to crash instances?

[Ryan Barrett]

...nd started again a few hours ago, i'm guessing due to the 
introduction of 1.9.15.

apologies for harping on this; it just continues to be an interesting 
new(ish) pattern, at least to me.

On Tuesday, October 21, 2014 5:30:11 PM UTC-7, Ryan Barrett wrote:
>
> ...and stopped again, as of 12:30 PST.
>
> On Monday, October 20, 2014 11:28:25 AM UTC-7, Ryan Barrett wrote:
>>
>> just a heads up: this started happening again a few hours ago. my 
>> instances are on 1.9.14 now, and the timing and situation are about the 
>> same, so maybe the same root cause with the new release?
>>
>> On Tuesday, October 7, 2014 4:39:03 PM UTC-7, Ryan Barrett wrote:
>>>
>>> looks like this has stopped as of around 1pm PST today. app engine team, 
>>> if this was a 1.9.13 bug that got fixed, thank you!
>>>
>>> On Monday, October 6, 2014 8:54:04 AM UTC-7, Ryan Barrett wrote:
>>>>
>>>> hi all! starting around 3PM PST yesterday, my instances have started 
>>>> crashing on a small fraction of requests, much more often and consistently 
>>>> than before. (app id s~brid-gy.) the log message is the usual, "A problem 
>>>> was encountered with the process that handled this request, causing it to 
>>>> exit... (Error code 204)"
>>>>
>>>> when this started, i hadn't deployed any new changes since 10/2, and i 
>>>> haven't found anything suspicious and new in my workload or usage pattern. 
>>>> my instances are on 1.9.13, which hasn't been announced yet, so i wonder 
>>>> if 
>>>> the 1.9.13 rollout got to my app yesterday and may have caused this.
>>>>
>>>> has anyone seen a similar bump in instance crashes in their apps since 
>>>> yesterday?
>>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: 1.9.13 causing some requests to crash instances?

[Ryan Barrett]

...and stopped again, as of 12:30 PST.

On Monday, October 20, 2014 11:28:25 AM UTC-7, Ryan Barrett wrote:
>
> just a heads up: this started happening again a few hours ago. my 
> instances are on 1.9.14 now, and the timing and situation are about the 
> same, so maybe the same root cause with the new release?
>
> On Tuesday, October 7, 2014 4:39:03 PM UTC-7, Ryan Barrett wrote:
>>
>> looks like this has stopped as of around 1pm PST today. app engine team, 
>> if this was a 1.9.13 bug that got fixed, thank you!
>>
>> On Monday, October 6, 2014 8:54:04 AM UTC-7, Ryan Barrett wrote:
>>>
>>> hi all! starting around 3PM PST yesterday, my instances have started 
>>> crashing on a small fraction of requests, much more often and consistently 
>>> than before. (app id s~brid-gy.) the log message is the usual, "A problem 
>>> was encountered with the process that handled this request, causing it to 
>>> exit... (Error code 204)"
>>>
>>> when this started, i hadn't deployed any new changes since 10/2, and i 
>>> haven't found anything suspicious and new in my workload or usage pattern. 
>>> my instances are on 1.9.13, which hasn't been announced yet, so i wonder if 
>>> the 1.9.13 rollout got to my app yesterday and may have caused this.
>>>
>>> has anyone seen a similar bump in instance crashes in their apps since 
>>> yesterday?
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: 1.9.13 causing some requests to crash instances?

[Ryan Barrett]

just a heads up: this started happening again a few hours ago. my instances 
are on 1.9.14 now, and the timing and situation are about the same, so 
maybe the same root cause with the new release?

On Tuesday, October 7, 2014 4:39:03 PM UTC-7, Ryan Barrett wrote:
>
> looks like this has stopped as of around 1pm PST today. app engine team, 
> if this was a 1.9.13 bug that got fixed, thank you!
>
> On Monday, October 6, 2014 8:54:04 AM UTC-7, Ryan Barrett wrote:
>>
>> hi all! starting around 3PM PST yesterday, my instances have started 
>> crashing on a small fraction of requests, much more often and consistently 
>> than before. (app id s~brid-gy.) the log message is the usual, "A problem 
>> was encountered with the process that handled this request, causing it to 
>> exit... (Error code 204)"
>>
>> when this started, i hadn't deployed any new changes since 10/2, and i 
>> haven't found anything suspicious and new in my workload or usage pattern. 
>> my instances are on 1.9.13, which hasn't been announced yet, so i wonder if 
>> the 1.9.13 rollout got to my app yesterday and may have caused this.
>>
>> has anyone seen a similar bump in instance crashes in their apps since 
>> yesterday?
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: 1.9.13 causing some requests to crash instances?

[Ryan Barrett]

looks like this has stopped as of around 1pm PST today. app engine team, if 
this was a 1.9.13 bug that got fixed, thank you!

On Monday, October 6, 2014 8:54:04 AM UTC-7, Ryan Barrett wrote:
>
> hi all! starting around 3PM PST yesterday, my instances have started 
> crashing on a small fraction of requests, much more often and consistently 
> than before. (app id s~brid-gy.) the log message is the usual, "A problem 
> was encountered with the process that handled this request, causing it to 
> exit... (Error code 204)"
>
> when this started, i hadn't deployed any new changes since 10/2, and i 
> haven't found anything suspicious and new in my workload or usage pattern. 
> my instances are on 1.9.13, which hasn't been announced yet, so i wonder if 
> the 1.9.13 rollout got to my app yesterday and may have caused this.
>
> has anyone seen a similar bump in instance crashes in their apps since 
> yesterday?
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] 1.9.13 causing some requests to crash instances?

[Ryan Barrett]

hi all! starting around 3PM PST yesterday, my instances have started 
crashing on a small fraction of requests, much more often and consistently 
than before. (app id s~brid-gy.) the log message is the usual, "A problem 
was encountered with the process that handled this request, causing it to 
exit... (Error code 204)"

when this started, i hadn't deployed any new changes since 10/2, and i 
haven't found anything suspicious and new in my workload or usage pattern. 
my instances are on 1.9.13, which hasn't been announced yet, so i wonder if 
the 1.9.13 rollout got to my app yesterday and may have caused this.

has anyone seen a similar bump in instance crashes in their apps since 
yesterday?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: too many appengine frontend instances

[Ryan Barrett]

me too. it seems like many dynamic requests, if not all, are spawning new 
instances. very odd. seems like the scheduler is unhappy.

On Thursday, May 8, 2014 10:00:59 PM UTC-7, James Foster wrote:
>
> I'm seeing the same thing. Between 10x and 30x the regular number of 
> instances, even though the number of requests is the same as usual. Perhaps 
> a problem with v1.9.5?
>
> Status page seems to be showing everything but datastore as normal, though.
>
> https://code.google.com/status/appengine
>
> If anyone files an issue, please link it here so that we can star it.
>
>
> On Friday, May 9, 2014 1:40:19 PM UTC+10, blackdew wrote:
>>
>> my server make too many Frontend instances
>>
>> may be it make instance per request
>>
>> from 2 hours ago
>>
>> and my last deploy was yesterday may be 24 hours ago
>>
>> so, help and give me any advice
>>
>> thanks
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] oauth-dropins: a collection of drop-in OAuth client flows for Python App Engine

[Ryan Barrett]

Hi all! If you're on Python App Engine and you need to use an 
OAuth-protected API on another site, check out oauth-dropins. It’s a 
collection of drop-in OAuth client flows for many popular sites, including 
Facebook, Twitter, Google+, Instagram, Dropbox, Blogger, Tumblr, and 
WordPress.com, with more on the way.

https://github.com/snarfed/oauth-dropins

You can try it right now in the interactive demo app: 
https://oauth-dropins.appspot.com/

Feedback and pull requests are welcome. Happy hacking!

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: Get the root entity for a given Model

[Ryan Barrett]

On Feb 19, 11:26 am, Rein Petersen  wrote:
>
> root() Returns a model instance for the root (top parent) entity
> within the entity chain of this instance, or None if this instance
> does not have a parent

interesting! that would definitely be useful. you don't need a
dedicated new app engine feature, though; you can do it yourself, in
user land. i haven't tested this, but it should work:

class Foo(db.Model):
  ...

  def root_key(self):
root_key = self.parent_key()

while root_key.parent() is not None:
  root_key = root_key.parent()

return root_key

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: GqlQuery bind method - is it safe to share the same query instance between requests?

[Ryan Barrett]

hi alex! you're right to be cautious, but happily, requests are not
handled by different threads. our python interpreters are single
threaded, and handle only a single request at a time. more:

http://code.google.com/appengine/docs/python/sandbox.html
http://groups.google.com/group/google-appengine/browse_thread/thread/3f8cfeaf7dc2eb72/7a5918d69158ae1b?#7a5918d69158ae1b

given that, you don't need to worry about concurrent accesses to the
query global variable, so this should be safe. even better, fetch() re-
runs the query from the beginning, so if the parameters are the same
across requests, you don't even need to bind() each time.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---