[google-appengine] Very high request latency periods on App Engine
Last week I started noticing periods of very high request latency in my app lasting for many hours and even days. These are not correlated to the traffic patterns in my app, and so I assume it's an issue that affects all of app engine. Looking at the logs, it seems that some requests are spending way too much time in the pending queue, despite my app's max pending latency set to Automatic – 200ms (min - max). I filed an issue for this: http://code.google.com/p/googleappengine/issues/detail?id=6447 This is pretty bad. Could anyone on the team comment whether this is an App Engine problem or something wrong with my app's configuration? Thanks, Alex -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: DeadlineExceededError on HRD and sky-high latency (python27+threadsafe)
I've been experiencing the same issues lately. I'm on py 2.5 and MS datastore. Didn't see this thread, so started my own: http://groups.google.com/group/google-appengine/browse_thread/thread/31d1c228070ac09b# Also filed an issue: http://code.google.com/p/googleappengine/issues/detail?id=6447 I also don't think it's a matter of having more instances. My app seems to have tons of idle instances sitting around (despite my max idle instance setting being 1), and still I'm seeing tons of pending latency. On Dec 5, 5:37 pm, Andrius A andriu...@gmail.com wrote: I can confirm that as well, don't think its related with python 2.7 or HRD. I am seeing high latencies similar to yours with python 2.5 and MS just right now, even GAE console takes ages to load and some times throws errors. I noticed that similar slowness happens few times per week, I wonder will it ever become stable? On 5 December 2011 22:32, sergio.jar...@gmail.com sergio.jar...@gmail.comwrote: The issue is that I've seen sky-high latencies *without* the app engine even trying to spin other instances! Just right now I captured this screenshot from the management console: http://i42.tinypic.com/bhzyf7.png The above screenshot is not normal! BTW, around the same time I took the screenshot I got a 500-Internel server error accessing the app engine dashboard at appspot.com. Most of the time I've got ~ 0.2 QPS and 100-200ms latencies that's why I have just one idle instance. On Dec 5, 9:43 pm, Brandon Wirtz drak...@digerat.com wrote: Allow more than one idle instance. How long is your spin-up? If you get 8 simultaneous requests and each request takes 1 second, and your spin up time is 5 seconds, then you will have time outs with the settings you describe. Consider running full auto for 24 hours and then clamping down. Also consider reading the tuning guides published on other threads. -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of sergio.jar...@gmail.com Sent: Monday, December 05, 2011 11:15 AM To: Google App Engine Subject: [google-appengine] DeadlineExceededError on HRD and sky-high latency (python27+threadsafe) Hi! I've successfully migrated from master-slave to hdd with python 27 and threadsafe however I've been getting sporadically a few deadline exceeded errors (bellow stack trace excerpt). I also have configured min idle requests to automatic and maximum to 1 and pending latency to 800ms, I've observed on occasion latencies of up to 63000(!!!)ms and without app engine spinning extra instances to serve requests. Any thoughts? I can provide app-id off-list if someone from google wants to have a look. Thanks! File /base/python27_runtime/python27_lib/versions/1/google/appengine/ ext/db/__init__.py, line 3588, in __get__ instance = get(reference_id) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/ext/db/__init__.py, line 1515, in get return get_async(keys, **kwargs).get_result() File /base/python27_runtime/python27_lib/versions/1/google/ appengine/api/apiproxy_stub_map.py, line 592, in get_result return self.__get_result_hook(self) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/datastore/datastore_rpc.py, line 1437, in __get_hook entities = rpc.user_data(entities) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/api/datastore.py, line 623, in local_extra_hook return extra_hook(result) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/ext/db/__init__.py, line 1485, in extra_hook model = cls1.from_entity(entity) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/ext/db/__init__.py, line 1419, in from_entity return cls(None, _from_entity=entity, **entity_values) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/ext/db/__init__.py, line 1800, in __init__ super(Expando, self).__init__(parent, key_name, _app, **kwds) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/ext/db/__init__.py, line 945, in __init__ prop.__set__(self, value) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/ext/db/__init__.py, line 600, in __set__ setattr(model_instance, self._attr_name(), value) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/ext/db/__init__.py, line 1826, in __setattr__ check_reserved_word(key) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/ext/db/__init__.py, line 300, in check_reserved_word if datastore_types.RESERVED_PROPERTY_NAME.match(attr_name): DeadlineExceededError -- You received this message because you are subscribed to the Google Groups Google App
[google-appengine] Question about Write Ops cost for updating an entity with changed properties
Alfred wrote an excellent explanation of how write ops are charged ( http://groups.google.com/group/google-appengine/msg/ca47eb7678d50372? ). My question is how does the the datastore determine whether a property has actually changed - does it compare the new value to the saved value or does the python db.Model class mark a property as changed every time it's assigned (even if the values are the same). -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Any way to specify sort direction for indexed properties?
Suppose you have an entity: class Foo(db.Model): x = db.IntegerProperty() I believe that putting an entity of this type will incur 2 write ops for the property x, because indexes in both sort directions will be written. If you don't need to sort on x in both directions, is there any way to only have 1 index for it? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Lots of import errors today
I'm seeing lots of DeadlineExceededErrors in production logs in recent hours where the app seems to be stuck on loading various imports. Examples: 1) File /base/python_runtime/python_lib/versions/third_party/ django-0.96/django/template/loaders/filesystem.py, line 17, in load_template_source return (open(filepath).read(), filepath) 2) File /base/python_runtime/python_lib/versions/1/google/appengine/ ext/appstats/datamodel_pb.py, line 20, in module from google.net.proto import ProtocolBuffer 3) File /base/data/home/apps/typeracerdata/62.354361981982300313/ encoding.py, line 1, in module import time etc.. I've never seen this before - in the past deadlines were usually reached while waiting for datastore operations and that sort of thing. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Write Ops incurred during Model.put()
Thanks for the definitive clarification, Alfred. It would be awesome if you guys could show the actual Read/Write Ops incurred by each URI under the Current Load section of the live dashboard. On Sat, Sep 24, 2011 at 4:00 PM, Alfred Fuller arfuller+appeng...@google.com wrote: Yes, the write ops for changing an entity depends on how many indexed properties you change. It is: 1 op for the entity 2 ops for each changed index value (1 to remove the old value and 1 to add the new) Since it is not a fixed value and depends on what you actually modify, we do not (can not) show it in the dev_appserver's Datastore Viewer. On Tue, Sep 20, 2011 at 11:09 AM, Alex Epshteyn alexander.epsht...@gmail.com wrote: So the Write Ops value in the new dev_appserver's Datastore Viewer applies only to creating new entities of a kind, updating the entity i is presumably much cheaper if only a few properties are changed? On Tue, Sep 20, 2011 at 12:27 PM, Simon Knott knott.si...@gmail.com wrote: Hi, According to Alfred, who's a Googler who appears to know lots about the datastore, only the updated values cause index write operations - see https://groups.google.com/d/msg/google-appengine/mjnSqQWOfqU/cgPVeHbrR8oJ for more info. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/OcSq1WpF5noJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Another perspective on the upcoming datastore prices
Consider an average dynamic page in any web application. It might do one entity write and fetch a small result set. Under the new GAE billing model, this might cost 25 Write Ops, and 21 Read Ops (a pretty conservative estimate). Those amount to ~ $40 per million page views, which is more than the average revenue from Google AdSense for the same million pageviews! There's something really wrong with this picture. The reason AdSense does so well is that web hosting in this day and age costs a lot less than what AdSense earns. But now GAE hosting is about to cost much more than what AdSense earns (and so far I only counted just the datastore ops, so the true costs might be 2-3x higher than $40 per million pageviews). The bottom line: developers will be losing money by hosting an ad- supported application on GAE! What's going on here? Google App Engine was supposed to be more cost- effective than the alternatives, but these new prices seem to be totally out of whack with the reality of current web economics. To the management at Google who came up with these prices: please consider consulting the AdSense team before these prices go into effect! -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Another perspective on the upcoming datastore prices
Correction: my math is probably off by a factor of about 10 on the AdSense revenue estimate, and I apologize for my quickness in jumping to my conclusion. But still, we're talking about some pretty narrow margins here. On Sep 23, 1:58 pm, Alex Epshteyn alexander.epsht...@gmail.com wrote: Consider an average dynamic page in any web application. It might do one entity write and fetch a small result set. Under the new GAE billing model, this might cost 25 Write Ops, and 21 Read Ops (a pretty conservative estimate). Those amount to ~ $40 per million page views, which is more than the average revenue from Google AdSense for the same million pageviews! There's something really wrong with this picture. The reason AdSense does so well is that web hosting in this day and age costs a lot less than what AdSense earns. But now GAE hosting is about to cost much more than what AdSense earns (and so far I only counted just the datastore ops, so the true costs might be 2-3x higher than $40 per million pageviews). The bottom line: developers will be losing money by hosting an ad- supported application on GAE! What's going on here? Google App Engine was supposed to be more cost- effective than the alternatives, but these new prices seem to be totally out of whack with the reality of current web economics. To the management at Google who came up with these prices: please consider consulting the AdSense team before these prices go into effect! -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Another perspective on the upcoming datastore prices
How do you know what your costs will be with Python 2.7, Brandon? I don't believe it is available yet, so are you using it as a trusted tester? If so, I'd be curious to know how many concurrent threads you can get per instance with Python 2.7. On Fri, Sep 23, 2011 at 4:50 PM, Brandon Wirtz drak...@digerat.com wrote: I monetize quite a few of my applications with Adsense, GAE Drastically lowered my hosting costs on the old billing, and since moving to Python 2.7 they are in check for the new billing as well. If I could have old billing and 2.7 I'd be really excited... but 2.7 brings my costs inline with what I had anticipated post beta to look like. -Brandon -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Alex Epshteyn Sent: Friday, September 23, 2011 10:59 AM To: Google App Engine Cc: appengine_updated_pric...@google.com Subject: [google-appengine] Another perspective on the upcoming datastore prices Consider an average dynamic page in any web application. It might do one entity write and fetch a small result set. Under the new GAE billing model, this might cost 25 Write Ops, and 21 Read Ops (a pretty conservative estimate). Those amount to ~ $40 per million page views, which is more than the average revenue from Google AdSense for the same million pageviews! There's something really wrong with this picture. The reason AdSense does so well is that web hosting in this day and age costs a lot less than what AdSense earns. But now GAE hosting is about to cost much more than what AdSense earns (and so far I only counted just the datastore ops, so the true costs might be 2-3x higher than $40 per million pageviews). The bottom line: developers will be losing money by hosting an ad- supported application on GAE! What's going on here? Google App Engine was supposed to be more cost- effective than the alternatives, but these new prices seem to be totally out of whack with the reality of current web economics. To the management at Google who came up with these prices: please consider consulting the AdSense team before these prices go into effect! -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Write Ops incurred during Model.put()
So the Write Ops value in the new dev_appserver's Datastore Viewer applies only to creating new entities of a kind, updating the entity i is presumably much cheaper if only a few properties are changed? On Tue, Sep 20, 2011 at 12:27 PM, Simon Knott knott.si...@gmail.com wrote: Hi, According to Alfred, who's a Googler who appears to know lots about the datastore, only the updated values cause index write operations - see https://groups.google.com/d/msg/google-appengine/mjnSqQWOfqU/cgPVeHbrR8oJ for more info. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/OcSq1WpF5noJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] What happens under the hood when a Property is switched from indexed to unindexed
Suppose you have lots of entities of a certain kind in the datastore and you want to change some of their properties (e.g. of type StringProperty) to unindexed (whereas they were previously indexed). Once you set indexed=False on the property, what happens to the existing index? Does it get deleted, lowering your storage bill, or does it remain? Also what happens if you change the property back to indexed (indexed=True)? Do you have to fetch and re-put all the existing entities to get them re-indexed? In other words, do these changes apply to entities already stored or do they only apply to new entities stored after the change was made? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Write Ops incurred during Model.put()
Does a Model.put() call incur Write Ops for every indexed property in an entity regardless of whether the actual property values have changed? In other words, if I update a stored entity having only changed one property, does the datastore update just that one index with 2 write ops (asc desc) or does it perform write ops to update all the properties in the entity regardless of whether their values have changed? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Datastore between apps.
Running both python and java runtimes (under different version strings) is an intriguing idea. Is it possible to have both the python and java local dev appservers to use the same local datastore? (If not, it seems difficult to properly test the apps.) Are there any other specific worms in this can anyone can think of? On Nov 10, 6:51 pm, Ikai L (Google) ika...@google.com wrote: You won't be able to have direct access to the same datastore from both applications, however, you can write both a Python and Java application running with the same appspot ID:http://googleappengine.blogspot.com/2009/06/10-things-you-probably-di In general, though, if you use this approach, you'd probably opening a can of worms. A better approach if you absolutely need to write your application using two languages is to have two applications, with one application using URLFetch APIs to post to the application hosting the authoritative data store. Is there a reason you need to write your application in both Java and Python? It may be simpler just to write the entire application in a single language, but even then there is another option: running any JVM version of a language. The community has been successful running Scala, Jython, JRuby, Groovy amongst other languages on the Java flavor of App Engine in conjunction with Java (though there are some caveats). As far as SLAs go, we do not have any SLAs for applications running on App Engine. On Tue, Nov 10, 2009 at 3:19 PM, geniesis geniesis...@gmail.com wrote: My eventual solution/implmentation will consist of two apps. The first app is the web-frontend which provides the user interface and eye-candy for users to view data in the datastore. The second app is mainly for the backend support software. There will be a bunch of servers that will be periodically posting data to this second app. The second app will process this data and store it in the datastore. The issue here is, can the two apps share a common datastore?? Or does it need to all exist in the one app. I ask this because the frontend will more than likely be in java, but the backend app will be in python. Considering you can only have 1 lang running at any one time on an app, this because an issue. Also, are there SLA's on App Engine? -- Ikai Lan Developer Programs Engineer, Google App Engine -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Can you switch billing to monthly instead of daily
Could be weekly but with multiple apps it seems like I'm getting billed almost every day. Are the apps not consolidated under the same billing account? On 9/16/09, Nick Johnson (Google) nick.john...@google.com wrote: Hi Alex, Normal billing is weekly. Are you sure you're getting billing emails daily? What did you set your daily limit to? -Nick On Wed, Sep 16, 2009 at 5:24 AM, Alex Epshteyn alexander.epsht...@gmail.com wrote: It's a bit annoying to get daily billing emails and having 30 tiny charges on my credit card bill every month instead of one. In addition to being annoying, it complicates the accounting. I think every other hosting company charges on a monthly basis. Any chance App Engine could start using the traditional billing model? -- Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- Sent from my mobile device --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Can you switch billing to monthly instead of daily
It's a bit annoying to get daily billing emails and having 30 tiny charges on my credit card bill every month instead of one. In addition to being annoying, it complicates the accounting. I think every other hosting company charges on a monthly basis. Any chance App Engine could start using the traditional billing model? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Appengine down again
Everything is completely down right now. This sucks. My site is losing users every minute. On Jul 2, 2:08 pm, stelg stelg.best...@gmail.com wrote: See for this issue this informationhttp://groups.google.com/group/google-appengine-downtime-notify/brows... On Jul 2, 8:01 pm, stelg stelg.best...@gmail.com wrote: Are all Google Apps down I cannot access my apps anymore while nothing has been changed by me. There is no status of the service available === only a blank page onhttp://code.google.com/status/appengine I guess there is a major breakdown of the Google App Engine infrastructure. 2 july 20:00 hrs CET On Jul 2, 6:56 pm, vishpat vish...@gmail.com wrote: http://code.google.com/status/appengine Gives an internal error. Very frustrating when your apps go down in spite of paying for them. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Outage this morning
Another thing to note is that the system status dashboard did not pick this up: http://code.google.com/status/appengine On Jul 1, 3:10 pm, Alex Epshteyn alexander.epsht...@gmail.com wrote: My app's URL, typeracerdata.appspot.com was completely unreachble for 10 minutes and then mostly unreachable for an hour today between Wed Jul 01 13:43:16 UTC 2009 and Wed Jul 01 14:51:09 UTC 2009 I'd like to request a post-mortem to be conducted on this particular outage. I'm concerned because these kinds of things happen very often on App Engine (at least once a week, maybe more). We need an explanation for why the service is so unreliable. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Your application is exceeding a quota: Datastore Indices Count
I just started seeing this message as well today after I enabled billing on my app (id = typeracerdata) On Jun 29, 8:20 pm, Jeff S (Google) j...@google.com wrote: This quota has been reset, apologies for the inconvenience. Happy coding, Jeff On Mon, Jun 29, 2009 at 1:35 PM, gae123 pa...@gae123.com wrote: I see this with my app with appid: neatschool after I uploaded a new version of the app that I have not made the current version yet. Could you please reset the count? Thanks --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Elevated quota dropped a week ahead of schedule - not cool
A recent email from the App Engine team said: -- * On June 22nd, as previously announced, standard free quota levels are being adjusted: http://code.google.com/appengine/docs/quotas.html#Free_Changes * On July 6th, two weeks later, your application's special elevated free quota will revert to the new standard free quota levels. -- Well, my app's elevated quota got dropped today (June 30, at 7:51 PDT). This caused a few hours of downtime for my users. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Problem in dashboard expanding log entries using Firefox 3.5 (Windows XP)
Confirmed. I'm also on XP with FF 3.5. Filed a bug for this: http://code.google.com/p/googleappengine/issues/detail?id=1799 On Jun 30, 11:13 pm, Tim Hoffman zutes...@gmail.com wrote: Hi I am just trying out Firefox 3.5 (windows xp - yeah I know ;-) and noticed that in the appengine dashboard that I am unable to expand log entries in the logviewer (ie clicking on the + sign at the top or on the log entry itself) Works fine on ff 3.0.11 on my machine. Can anyone else check it to see if its just me ? Thanks Tim --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Too many versions error during upload
Uploads are working again for me as of this morning. On Jun 28, 6:59 pm, Alex Epshteyn alexander.epsht...@gmail.com wrote: I've been having the same problem since yesterday (with a Java app), Judging by how many messages there are on the groups about this the past couple of days, this is most likely a global app engine issue happening this weekend. I don't think it's related to the actual number of versions people currently have deployed nor their deployment quota (both of mine are fine). Cross posted: http://groups.google.com/group/google-appengine-java/browse_thread/th... On Jun 27, 12:11 am, Stephen Mayer stephen.ma...@gmail.com wrote: Howmanydeployments are you currently using? If you delete a few old ones it will probably let you add a new one. Or you could overwrite an old version and then deploy under that version since it probably takes some time before your quota resets once you reach the max number ofversions. See the Deployments in your Quota Details in AppEngine Control Panel. Hope that helps! Stephen On Jun 26, 7:29 pm, manuelaraoz manuelar...@gmail.com wrote: PLEASE HELP this is urgent when trying to deploy, I get this error message: Rolling back the update. Error403: --- begin server output --- TooManyVersions(403) The application already has the maximum number ofversions. --- end server output --- WHAT DOES THAT MEAN I really need to deploy today, it's urgent please please please someone help me I've tried deleting oldversionsof my app and deploying a new one (changing version id) but nothing... I'm getting desperate!!! I REALLY HAVE TO DEPLOY TODAY thanks! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Too many versions error during upload
I've been having the same problem since yesterday (with a Java app), Judging by how many messages there are on the groups about this the past couple of days, this is most likely a global app engine issue happening this weekend. I don't think it's related to the actual number of versions people currently have deployed nor their deployment quota (both of mine are fine). Cross posted: http://groups.google.com/group/google-appengine-java/browse_thread/thread/5083c89588258546/fba352f293600709#fba352f293600709 On Jun 27, 12:11 am, Stephen Mayer stephen.ma...@gmail.com wrote: Howmanydeployments are you currently using? If you delete a few old ones it will probably let you add a new one. Or you could overwrite an old version and then deploy under that version since it probably takes some time before your quota resets once you reach the max number ofversions. See the Deployments in your Quota Details in AppEngine Control Panel. Hope that helps! Stephen On Jun 26, 7:29 pm, manuelaraoz manuelar...@gmail.com wrote: PLEASE HELP this is urgent when trying to deploy, I get this error message: Rolling back the update. Error403: --- begin server output --- TooManyVersions(403) The application already has the maximum number ofversions. --- end server output --- WHAT DOES THAT MEAN I really need to deploy today, it's urgent please please please someone help me I've tried deleting oldversionsof my app and deploying a new one (changing version id) but nothing... I'm getting desperate!!! I REALLY HAVE TO DEPLOY TODAY thanks! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] GqlQuery bind method - is it safe to share the same query instance between requests?
I'm tempted to define an instance of GqlQuery in a global variable, then while handling each request, to call the query's bind() method to bind to it the parameters for the user who issued the request. I'm relatively new to Python, but in a language like Java, this would be a terrible idea - you shouldn't share the same instance of a mutable object between threads. Is this a concern on App Engine? In other words, could a scenario like this occur? QUERY.bind() called in request1 QUERY.bind() called in request2 QUERY.fetch() called in request1 (but QUERY is bound to request2's parameters now!) QUERY.fetch() called in request2 In this scenario, both request would fetch the data that was intended to be fetched only by request2, which is unacceptable... --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: GqlQuery bind method - is it safe to share the same query instance between requests?
Ah, I see. It's safe because a new request will not be processed by the same instance of the python interpreter until the previous request has fully completed, right? Thanks, Ryan! On Dec 16, 7:37 pm, Ryan Barrett goo...@ryanb.org wrote: hi alex! you're right to be cautious, but happily, requests are not handled by different threads. our python interpreters are single threaded, and handle only a single request at a time. more: http://code.google.com/appengine/docs/python/sandbox.htmlhttp://groups.google.com/group/google-appengine/browse_thread/thread/... given that, you don't need to worry about concurrent accesses to the query global variable, so this should be safe. even better, fetch() re- runs the query from the beginning, so if the parameters are the same across requests, you don't even need to bind() each time. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Please reopen Issue 182: Mail API: python email module import fails when source code is changed ...
This dev_appserver issue has not gone away, and is still rather annoying during development. http://code.google.com/p/googleappengine/issues/detail?id=182 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Datastore Timeouts Must Die
As an example, of how frequently datastore write timeouts are happening on app engine, here are the last 20 entries in my production error log, showing 20 timeouts in 15 minutes! I'm hoping the Google team will break their silence about this and at least acknowledge the problem. Developers: please star issue http://code.google.com/p/googleappengine/issues/detail?id=764 if you are also experiencing this problem. 10-07 02:20PM 48.172 /games 500 4040ms 11285mcycles 0kb 10-07 02:20PM 48.172 /games 500 4040ms 11285mcycles 0kb 10-07 02:16PM 43.763 /games 500 2813ms 8253mcycles 0kb 10-07 02:16PM 43.410 /games 500 3078ms 8909mcycles 0kb 10-07 02:16PM 40.958 /games 500 3084ms 8915mcycles 0kb 10-07 02:15PM 03.499 /games 500 4109ms 12077mcycles 0kb 10-07 02:15PM 02.584 /games 500 4110ms 12365mcycles 0kb 10-07 02:14PM 21.732 /games 500 3071ms 8900mcycles 0kb 10-07 02:10PM 52.865 /games 500 4045ms 11287mcycles 0kb 10-07 02:10PM 51.875 /games 500 4041ms 11285mcycles 0kb 10-07 02:10PM 41.774 /games 500 4036ms 11282mcycles 0kb 10-07 02:10PM 36.958 /games 500 4033ms 11279mcycles 0kb 10-07 02:10PM 23.148 /games 500 4039ms 11174mcycles 0kb 10-07 02:08PM 31.288 /games 500 4068ms 10555mcycles 0kb 10-07 02:08PM 30.842 /games 500 4449ms 12217mcycles 0kb 10-07 02:07PM 27.179 /games 500 4047ms 11285mcycles 0kb 10-07 02:07PM 24.828 /games 500 4047ms 11284mcycles 0kb 10-07 02:07PM 20.139 /games 500 4061ms 11281mcycles 0kb 10-07 02:07PM 18.532 /games 500 4042ms 11246mcycles 0kb 10-07 02:07PM 08.776 /games 500 4048ms 11285mcycles 0kb 10-07 02:05PM 58.206 /games 500 4047ms 11019mcycles 0kb On Oct 6, 1:53 am, Alex Epshteyn [EMAIL PROTECTED] wrote: Please see:http://code.google.com/p/googleappengine/issues/detail?id=764 There is a recent thread on this group titled Why Google App Engine is broken and what Google must do to fix it. I personally don't think that any of the topics raised by the OP of that thread imply that anything is broken - they are just feature requests. What *is* broken, however, is writing to the Datastore. Lots of developers, not just me, have been complaining about this here for months. I don't think there's been any real response from the Google team about this. This issue needs to be investigated and hopefully fixed before App Engine can be used for production. The probability of failure for any given data write request is just too high right now. I actually *am* using App Engine in production and somewhat regretting it, mostly due to this issue. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: What changes were recently made to request.charset?
I'm not sure what has changed, but I just fixed my issue by using msg.encode(utf8) before passing msg to hmacSha1. It was working without this step in the past. Alex On Mon, Oct 6, 2008 at 5:41 AM, Sylvain [EMAIL PROTECTED] wrote: I've seen many issues with charset (Unicode) like this one : http://code.google.com/p/googleappengine/issues/detail?id=573 Could you explain more your issue/code that seems broken. These issues are very annoying. Regards On 6 oct, 08:50, Alex Epshteyn [EMAIL PROTECTED] wrote: I had an issue with European characters in my application that I fixed two months ago. However, the issue has recently returned. I started seeing UnicodeEncodeError: 'ascii' codec can't encode character u'\xe3' in position 180: ordinal not in range(128) in the production logs about a week ago and after I upgraded the dev app server, I started seeing it locally as well. The release notes for 1.1.5 say Set the webapp Request charset property more accurately from CONTENT_TYPE. This must have been done to match some change in production. My question is - what has changed in production regarding request charset handling since August? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Datastore Timeouts Must Die
Please see: http://code.google.com/p/googleappengine/issues/detail?id=764 There is a recent thread on this group titled Why Google App Engine is broken and what Google must do to fix it. I personally don't think that any of the topics raised by the OP of that thread imply that anything is broken - they are just feature requests. What *is* broken, however, is writing to the Datastore. Lots of developers, not just me, have been complaining about this here for months. I don't think there's been any real response from the Google team about this. This issue needs to be investigated and hopefully fixed before App Engine can be used for production. The probability of failure for any given data write request is just too high right now. I actually *am* using App Engine in production and somewhat regretting it, mostly due to this issue. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Release of AppEngine 1.1.4 SDK
What is the proper upgrade procedure for Windows? I've just been installing over the previous files (same directory). Is this correct or not? Good to see releases coming out so frequently. Alex On Sep 27, 1:12 am, Rafe [EMAIL PROTECTED] wrote: Hello, This evening we have released the AppEngine 1.1.4 SDK. It addresses some of the problems that Windows users have had with static files on dev_appserver. If you are working on Windows and an existing application started telling you something like the following message, this fix is for you: configuration file invalid: regex does not compile: bogus escape: '\\xa' You can download it from the project site here: http://code.google.com/p/googleappengine/downloads/list - Rafe Kaplan --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Timeout - operation took too long
I've been seeing Timeouts on data put operations about 30-80 times a
day (which accounts for as much as 1-3% of all write requests) ever
since my app went into production on August 18. This is happening
every day and it's very annoying. It's worse during some periods,
(e.g. when the GAE team reports outages), but it's been a constant
factor regardless of these issues.
It would be awesome if the App Engine team could investigate and put
together a set of best practices for coding and data modeling to
reduce the chance of timeouts, if this is at all possible.
Certainly it's a good idea to code defensively and expect these sorts
of exceptions, but there has to be a way to lower the timeout rate.
Everyone's logs are filling up with garbage because of this and making
it hard to find any other important failures in the logs.
Here are some example stack traces I've been getting:
1) On Model.get_or_insert()
a).
File /base/python_lib/versions/1/google/appengine/ext/db/
__init__.py, line 862, in get_or_insert
return run_in_transaction(txn)
File /base/python_lib/versions/1/google/appengine/api/
datastore.py, line 1407, in RunInTransaction
result = function(*args, **kwargs)
File /base/python_lib/versions/1/google/appengine/ext/db/
__init__.py, line 857, in txn
entity = cls.get_by_key_name(key_name,
parent=kwds.get('parent'))
File /base/python_lib/versions/1/google/appengine/ext/db/
__init__.py, line 779, in get_by_key_name
return get(*keys)
File /base/python_lib/versions/1/google/appengine/ext/db/
__init__.py, line 974, in get
entities = datastore.Get(keys)
File /base/python_lib/versions/1/google/appengine/api/
datastore.py, line 211, in Get
_MaybeSetupTransaction(req, keys)
File /base/python_lib/versions/1/google/appengine/api/
datastore.py, line 1502, in _MaybeSetupTransaction
tx.handle)
File /base/python_lib/versions/1/google/appengine/api/
apiproxy_stub_map.py, line 46, in MakeSyncCall
stub.MakeSyncCall(service, call, request, response)
File /base/python_lib/versions/1/google/appengine/runtime/
apiproxy.py, line 245, in MakeSyncCall
rpc.Wait()
File /base/python_lib/versions/1/google/appengine/runtime/
apiproxy.py, line 161, in Wait
rpc_completed =
_apphosting_runtime___python__apiproxy.Wait(self)
DeadlineExceededError
b).
File /base/python_lib/versions/1/google/appengine/ext/db/
__init__.py, line 862, in get_or_insert
return run_in_transaction(txn)
File /base/python_lib/versions/1/google/appengine/api/
datastore.py, line 1431, in RunInTransaction
tx.handle, resp)
File /base/python_lib/versions/1/google/appengine/api/
apiproxy_stub_map.py, line 46, in MakeSyncCall
stub.MakeSyncCall(service, call, request, response)
File /base/python_lib/versions/1/google/appengine/runtime/
apiproxy.py, line 245, in MakeSyncCall
rpc.Wait()
File /base/python_lib/versions/1/google/appengine/runtime/
apiproxy.py, line 161, in Wait
rpc_completed = _apphosting_runtime___python__apiproxy.Wait(self)
File /base/python_lib/versions/1/google/appengine/runtime/
apiproxy.py, line 216, in __MakeCallDone
exception_entry[1] % (self.package, self.call))
DeadlineExceededError
c).
File /base/python_lib/versions/1/google/appengine/ext/db/
__init__.py, line 862, in get_or_insert
return run_in_transaction(txn)
File /base/python_lib/versions/1/google/appengine/api/
datastore.py, line 1407, in RunInTransaction
result = function(*args, **kwargs)
File /base/python_lib/versions/1/google/appengine/ext/db/
__init__.py, line 860, in txn
entity.put()
File /base/python_lib/versions/1/google/appengine/ext/db/
__init__.py, line 618, in put
return datastore.Put(self._entity)
File /base/python_lib/versions/1/google/appengine/api/
datastore.py, line 162, in Put
raise _ToDatastoreError(err)
File /base/python_lib/versions/1/google/appengine/api/
datastore.py, line 1627, in _ToDatastoreError
raise errors[err.application_error](err.error_detail)
Timeout
d). (this is a very frequent trace)
File /base/python_lib/versions/1/google/appengine/ext/db/
__init__.py, line 862, in get_or_insert
return run_in_transaction(txn)
File /base/python_lib/versions/1/google/appengine/api/
datastore.py, line 1441, in RunInTransaction
raise _ToDatastoreError(err)
File /base/python_lib/versions/1/google/appengine/api/
datastore.py, line 1627, in _ToDatastoreError
raise errors[err.application_error](err.error_detail)
Timeout
2). On Model.save()
a).
File /base/python_lib/versions/1/google/appengine/ext/db/
__init__.py, line 618, in put
return datastore.Put(self._entity)
File /base/python_lib/versions/1/google/appengine/api/
datastore.py, line 160, in Put
apiproxy_stub_map.MakeSyncCall('datastore_v3', 'Put', req, resp)
File /base/python_lib/versions/1/google/appengine/api/
apiproxy_stub_map.py, line 46, in MakeSyncCall
[google-appengine] Re: Question about effect of ListProperty on index size and put times
Hi Ryan, Thanks for your explanation! Actually, upon further analysis my write timeout rate is more like 1-3% and timeouts happen every 5 - 10 minutes throughout the day. Although I have retry logic to deal with this, I feel uneasy with so many errors in my logs. It doesn't sound like you think that indexing a 10-valued list is a big contributor to the timeouts, so I'll leave it alone. Do you have any other ideas that could reduce the timeout rate? So I gather the best way to replace a property in App Engine, is actually to leave the old property in the model spec but just not write anything to it in the future, to avoid exceptions, right? Alex On Tue, Sep 30, 2008 at 1:06 PM, ryan [EMAIL PROTECTED] wrote: hi alex! serializing the list property into json shouldn't make much of a difference in the timeout rate, but you're welcome to try it. honestly, though, .01% is noticeable, but i wouldn't exactly call it large. regardless of how your data is shaped, you'll at least see a small timeout rate. .01% is in the ballpark. as for deleting a property in your model definition, that's backward compatible. the existing data in the datastore will remain unchanged. when you load entities with the new model definition, they'll just ignore the old property, and not populate it in new entities. other changes, e.g. changing a property type or adding a required property, will cause exceptions when you fetch entities created with the old model definition. you can use Expando and optional properties to help with schema changes like these. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Lots of Datastore timeouts today - what keeps causing these?
A more general question first: my app has been experiencing more than 20 datastore write timeouts per day on average with an average of approx. 30K daily data write requests. I understand that it's only a 0.07% failure rate, but that's still too high for my liking. Some of my users' data has already been corrupted because my code didn't foresee every possible fault scenario. I've been spending lots of time trying to make my code and data bulletproof over the past month but it's an uphill battle. I'd like to know more about the cause of these mysterious failures. They mostly happen on data writes (in my case on Entity.save and Entity.get_or_insert), but sometimes also reads (in Entity.get_by_key_name). Could they be recovered if App Engine allowed waiting longer than 4000 ms? Or is it fatal when you've already waited this long? (when the same request succeeds, it does so under 200ms) It would be really awesome if App Engine could provide more fault- tolerance. Perhaps App Engine could time out these things sooner and do a couple of retries automatically before throwing an exception? It's too late for the handler code to trap these exception and retry, right? It's pretty frustrating seeing my logs filled with these errors and have to keep worrying about the affected users. Now regarding today: almost a hundred datastore timeouts between 9:30 and 9:41 am. Relevant stack trace: File /base/python_lib/versions/1/google/appengine/ext/db/ __init__.py, line 618, in put return datastore.Put(self._entity) File /base/python_lib/versions/1/google/appengine/api/ datastore.py, line 162, in Put raise _ToDatastoreError(err) File /base/python_lib/versions/1/google/appengine/api/ datastore.py, line 1627, in _ToDatastoreError raise errors[err.application_error](err.error_detail) Timeout --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
