Re: [google-appengine] Best Practices for PHP
On 5 December 2013 17:30, Gary Mort garyam...@gmail.com wrote: I was wondering if there are any best practice web sites on using PHP on Google App Engine. While working through installing the Joomla! CMS on GAE I've run into some weird oddities and initially have taken a rather brute force approach to work through them. Being unhappy with the brute force method[which requires modifying core files in the open source code in order to accommodate GAE] I have continued to cycle on it in the back burner and am trying an alternate approach right now. Specifically, in Joomla there are 3 main entree points to an application: http://mydomain.com/index.php http://mydomain.com/administrator/index.php http://mydomain.com/installation/index.php So far I've run into 2 issues: 1) Joomla uses XML files in order to define html form's. PHP is used to parse those files and then build the forms. GAE disables the ability to load /remote/ XML files in PHP by default. It is possible to allow this by add libxml_disable_entity_loader to the list of enabled functions and then to call libxml_disable_entity_loader(false); at some point in the code. For some reason, despite the fact that these are LOCAL xml files Jooma attempts to read, I still need to use this fix to allow Joomla to read the local files. 2) Joomla defaults to saving sessions in files, GAE defaults to saving sessions in memcache. Attempt to save session files using the default save_path does not work since the path does not exist. Making a small hack to make memcache the default does not work because Joomla checks for the existence of the Memcache extension[via extension_loaded()] which does not exist in GAE. I've filed a bug for this: https://code.google.com/p/googleappengine/issues/detail?id=10371 Cheers, Brian A slightly more involved solution required me adding a new GaeMemcache class to override that check, and modifying a core class to force it to enable GaeMemcache. None of this is bad...it's just inelegant and messy. It strikes me that this was the wrong way to go about it. Because of the way GAE is configured through app.yaml to match URI patterns to individual php files - instead of changing the core code, I can instead provide a pre-loader to process the file. So my new file structure will be as follows: /gae/joomla-site.php /gae/joomla-install.php /gae/joomla-admin.php /gae/lib/gaememcache.php /gae/appl.yaml /joomla-cms : submodule git repository pointer for the Joomla-CMS repository With this layout, I can now define my 3 entree points so that instead of loading the various index.php files directly, I can proxy the call first by the /gae/joomla* php file. That gives me the ability to make any Google App Engine modifications needed[defining a special memcache handler for GAE, making my libxml_disable_entity_loader function call - etc. No core hacks needed - now I can just setup a special pre-processors to handle everything. The configuration ability of GAE is extremely fascinating.. In many ways it is overly cumbersome for simply usage due to the options available - but the options also allow for neat little workarounds[I also realized that instead of hacking the core file, I could have made a copy of the core file under my gae directory and used the upload path overrides to overwrite the core files only when uploaded/deployed to GAE] What I am somewhat curious about is if there are any best practices tutorials out there since this sort of situation seems like it should be common, and the various answers are intuitive once you start groking the GAE deployment system. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: how to set GAE_SDK_ROOT environment variable?
How are you invoking dev_appserver.py? Is it through a symlink? Cheers, Brian On 23 November 2013 12:24, Martin Caceres Murrie mcacer...@gmail.com wrote: I am getting the same error when running dev_appserver.py . Any clue? El sábado, 23 de noviembre de 2013 12:40:28 UTC-3, saintthor escribió: i didn't update my app for weeks. today, when i try to update, the appcfg.py dosn't work. info as: urllib2.URLError: urlopen error [Errno 111] Connection refused i downloaded a new version package. then update. it shows me: ValueError: Could not determine SDK root; please set GAE_SDK_ROOT environment variable. what dose it mean? i think there may be some new settings in new version. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: gae urlfetch down
Hi Andy, Are you sure that you aren't attempting to catch InvalidURLError without having imported that name? Python does not check that exception patterns have existing names until an exception (*any* exception is thrown) e.g. try: ... 1 ... except Foo: ... pass ... 1 try: ... 1/0 ... except Foo: ... pass ... Traceback (most recent call last): File stdin, line 3, in module NameError: name 'Foo' is not defined Cheers, Brian On 27 July 2013 16:20, Andy Warner andy.war...@gmail.com wrote: Looks like it started at exactly 4pm Pacific. I have requests moments before that were fine and everything after is throwing a name error for InvalidURLError, so it is not just fetches failing, urlfetch is missing libraries in webapp2. except InvalidURLError: NameError: global name 'InvalidURLError' is not defined On Saturday, July 27, 2013 4:06:27 PM UTC-7, Adam Sah wrote: ouch! adam -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Modules and runtimes
Hi Ronoaldo, Yes! Cheers, Brian On 25 July 2013 20:15, Ronoaldo Pereira ronoa...@gmail.com wrote: Can I have an application module (or backend) using the Go runtime and keep my front-ends using Java runtime? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Please help! Invalid runtime or the current user is not authorized to use it.
Hi, The PHP runtime is currently in Limited Preview - so before you can upload your application you'll need to register it at https://gaeforphp.appspot.com/ Cheers, Brian On 23 July 2013 15:03, Nitin Manchanda nitinmanchanda.pu...@gmail.comwrote: Hi, I'm a registered user of google apps engine and i have created an application which i am using in my app.yaml configuration file, but when i am trying to deploy my php app on production box, it shows the following error and stops me there: *$ appcfg.py update . Host: appengine.google.com Application: my-app-id; version: 1 Starting update of app: my-app-id, version: 1 Getting current resource limits. Scanning files on local disk. Error 400: --- begin server output --- Invalid runtime or the current user is not authorized to use it. --- end server output ---* -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Puzzled by promotion of Go for high performance web serving
On 17 June 2013 16:39, Andrew Gerrand a...@google.com wrote: On 18 June 2013 09:28, Andrew Gerrand a...@google.com wrote: Yet C++ style guide forbids the use of exceptions. Perhaps our systems aren't large or commercial enough for us to see the value. ;-) That's supposed to say Google's C++ guide. My apologies. While what you say is true, that's mostly a historical artifact: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml#Exceptions Cheers, Brian Andrew -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] New Development Server URLFetch doesn't seem multithreaded
Hi John, On 23 May 2013 10:45, John Wheeler j...@highvolumeseller.com wrote: Hello, I am using the new python development server with the app engine 1.8.0 release. The Google Code page for said dev server here: https://code.google.com/p/appengine-devappserver2-experiment/ states - Multithreaded serving for *better* *performance* for complex applications and more correct semantics e.g. accessing your own application through urlfetch no longer deadlocks. However, and I don't know if I'm getting the meaning mixed up, but it doesn't appear like it can run URLFetches concurrently with one another. I have a task the fans-out a whole bunch of URLFetches, and they process one at a time, very slowly in the new development server. I want them to process at the same time. Could you explain exactly what you mean by I have a task the fans-out a whole bunch of URLFetches Do you have many task queue tasks that make synchronous calls using urlfetch, a single task that uses asynchronous urlfetch, or something else? Because tasks do *not* (currently) execute in parallel. Cheers, Brian Furthermore it appears each one starts a new instance because in the dev admin console, I refresh the page and there's only one instance listed with a request count of 1 at any given time. This is in the new dev server that ships with the SDK - this doesn't happen if I clone the Google repo above and run from that--but the URLFetches still process serially in either case. John -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] New Development Server URLFetch doesn't seem multithreaded
Hi John, On 23 May 2013 15:41, John Wheeler j...@highvolumeseller.com wrote: I have a task that spawns more tasks, each potentially spawning even more tasks, and they all do URLFetches. Thank you for clarifying this. You're saying it's not that we can't do concurrent URLFetch, we can't do concurrent tasks. That's right. Is concurrent tasks on the roadmap by any chance? I don't know of any short-term plans to add concurrent task execution to the development server. Cheers, Brian Thank you, John On Thursday, May 23, 2013 1:38:01 PM UTC-7, Brian Quinlan wrote: Hi John, On 23 May 2013 10:45, John Wheeler jo...@highvolumeseller.com wrote: Hello, I am using the new python development server with the app engine 1.8.0 release. The Google Code page for said dev server here: https://code.google.com/p/**appengine-devappserver2-**experiment/https://code.google.com/p/appengine-devappserver2-experiment/ states - Multithreaded serving for *better* *performance* for complex applications and more correct semantics e.g. accessing your own application through urlfetch no longer deadlocks. However, and I don't know if I'm getting the meaning mixed up, but it doesn't appear like it can run URLFetches concurrently with one another. I have a task the fans-out a whole bunch of URLFetches, and they process one at a time, very slowly in the new development server. I want them to process at the same time. Could you explain exactly what you mean by I have a task the fans-out a whole bunch of URLFetches Do you have many task queue tasks that make synchronous calls using urlfetch, a single task that uses asynchronous urlfetch, or something else? Because tasks do *not* (currently) execute in parallel. Cheers, Brian Furthermore it appears each one starts a new instance because in the dev admin console, I refresh the page and there's only one instance listed with a request count of 1 at any given time. This is in the new dev server that ships with the SDK - this doesn't happen if I clone the Google repo above and run from that--but the URLFetches still process serially in either case. John -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@**googlegroups.com. To post to this group, send email to google-a...@googlegroups.**com. Visit this group at http://groups.google.com/** group/google-appengine?hl=enhttp://groups.google.com/group/google-appengine?hl=en . For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out . -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: Breakpoints broken in new dev_appserver (1.7.6)?
Hi Renzo, On 23 March 2013 01:20, Renzo Nuccitelli ren...@gmail.com wrote: This is related with the local server improvements. So keep the 1.7.5 with you want to debug your program until they fix it. 1. You don't need to keep 1.7.5 - the old new dev_appserver is available in 1.7.6 with the name old_dev_appserver.py 2. The multi-process now used by dev_appserver.py means that debuggers will have to work differently to inter-operate with it. This document explains how that will work in 1.7.7: https://docs.google.com/document/d/1CCSaRiIWCLgbD3OwmuKsRoHHDfBffbROWyVWWL0ZXN4/edit#heading=h.fj44xnkhr0gr A preview of those debugging features can be found here: https://code.google.com/p/appengine-devappserver2-experiment/downloads/detail?name=devappserver2-v0.8.zipcan=2q=#makechanges Cheers, Brian Renzo On Wednesday, March 20, 2013 12:21:47 AM UTC-3, Dylan Sale wrote: Hi, I have updated to 1.7.6 and noticed the new dev_appserver. Nice! I did notice that PyDev (Eclipse plugin) breakpoints no longer work when using it. Does anyone know how to get breakpoints working again (with or without pydev). I guess this could be a question for the PyDev people, but I thought I would ask it here in case someone had encountered a similar problem. I assume it is due to the new server spawning new processes or something. For now I am using old_dev_appserver.py but I assume this will be deprecated at some point. Thanks -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: Important for Python developers - Updates to the Development Server in 1.7.6
The Python 2.5 deprecation blog post is here: http://googleappengine.blogspot.com.au/2013/03/python-25-thanks-for-good-times.html Cheers, Brian On 22 March 2013 05:58, Darien Caldwell darien.caldw...@gmail.com wrote: On Tuesday, March 19, 2013 5:57:02 PM UTC-7, Andrew Jessup wrote: Python 2.5 applications are no longer officially supported on the new Development Server in line with our deprecation announcement (**although they may still be run) I must have missed that notification. I know 2.7 was brought online, but I've seen nothing to indicate 2.5 was being depreciated or removed. Where/when was this announcement? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] warmup service and 1.7.6
Hi, On 22 March 2013 05:46, GAEfan ken...@gmail.com wrote: Hi. What happened to warmup service and 1.7.6? First, I do not remember warmup being called in the dev environment in the past. Is that new, and how do I turn off warmup in development? Warmup requests may be done in the development server now. That is a feature that we added to cause it to behave more like the production App Engine environment. There is currently no option to disable warmup requests in the development environment, could you explain why you want to do so? Second, /_ah/warmup is no longer a valid url. Am I missing something in the docs? Is this no longer the proper call in app.yaml: inbound_services: - warmup I don't understand what you mean here. Could you explain what you mean by: /_ah/warmup is no longer a valid url. Cheers, Brian ? Thank you. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] warmup service and 1.7.6
Hi, I still don't fully understand. You are opting into warmup requests so you must want to receive these requests in production App Engine. What is different about the development environment that you don't want these requests there? Cheers, Brian On 22 March 2013 07:25, GAEfan ken...@gmail.com wrote: Thanks for the reply, Brian. I am seeing 301's from /_ah/warmup in the logs. There is no url mapping for that url, so it redirects to my default page, which triggers all kinds of other processes. So, I would like to turn it off in dev, or at least find the proper mapping for /_ah/warmup. Thanks again. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] warmup service and 1.7.6
On 22 March 2013 10:01, GAEfan ken...@gmail.com wrote: Hi Brian: Is /_ah/warmup supposed to map to any particular response? Right now, there is no mapping for that url, and it gives a 301 to my default view. No, the mapping is up to you. Have you read: https://developers.google.com/appengine/docs/python/config/appconfig#Warmup_Requests I undertand the benefits of having the devserver match production, to test the actual performance (and thank you for that!). I am just not used to seeing the 301's in my logs. Should I map /_ah/warmup to a simple http response? Wouldn't that keep the instance warm without sending a request to an irrelevant view? The idea is that your /_ah/warmup handler should load whatever state you need to make subsequent requests faster. That might mean doing imports, loading template files, fetching configuration settings out of datastore or files, etc. If your warmup handler doesn't load any interesting state then there is no reason to opt into warmup requests. I am just seeing my development logs polluted with extra messages spawned by a view that I did not request. I can learn to live with it. You might want to change your app because you could have the same issue when you deploy. Thanks. BTW, will you be bringing more chocolate bars to I/O? Sorry, some things are classified ;-) Cheers, Brian -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] What I learned while migrating to the new dev app server for python
Hey PK, Thanks for this great write-up! On 18 March 2013 05:15, PK p...@gae123.com wrote: [snipped] 1.3 In addition to the datastore data and the blob data, logs now end up in some structured storage. I did not realize that at the beginning and running multiple configuration side by side was causing corruptions and weird crashes. So make sure you set the *--logs_path* to point to where the logs sqlite file should be located. I am not sure what the default is. By default all of the persisted state or an application (e.g. blobstore, datastore, logs) is stored in the directory specified by *--storage_path*. If that flag isn't specified, then the default is to create new directory in /tmp like appengine.appid.userid. The name and location is different on Windows. 1.4 dev_appserver.py would not run the tasks by default if you set the * --enable_tasks=no* but you could manually run them from the admin console. In the new server this option turns off the tasks completely, not clear if this is a bug or a new feature. Bug. 1.5 -d does not cut it for debug, you now needs to say *--log_level=debug* 2. Observed performance is about the same as before both from my experience and the measurements (it took a while to get there but it seems that now we are there) 3. I was using remote_api without specifying both the host and port of my server before. devappserver2.py is pickier. It took Sam and I several interaction to nail this issue. 4. The new server uses different options for log_levels of your code vs the dev server code (--*dev_appserver_log_level* vs --*log_level*). This is awesome but might be confusing!! --*dev_appserver_log_level* should not be used by most people - it is most useful in debugging dev_appserver behavior, not application behavior. 5. The new server does not seem to pick up application file modifications. Please star issue https://code.google.com/p/appengine-devappserver2-experiment/issues/detail?id=54 if you experience the same issue. This definitely doesn't effect everyone and we are still investigating why it is effecting you. Thanks again for the great feedback! Hopefully we'll be able to sort your issues out soon. Cheers, Brian Best, PK http://www.gae123.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: What I learned while migrating to the new dev app server for python
Hi Kaan, On 18 March 2013 08:36, Kaan Soral kaanso...@gmail.com wrote: Thanks a lot for sharing this. I see that I'm only missing the log_path (I remember checking the log file once, it was an sqlite too, really useful if you ask me, but it's not used anywhere yet, wishing it will soon be) I starred your issue too, 1.1 trigger a code path not sure what you mean by this, but I also, very rarely, see modifications not live, when you add a blank line and save, it starts showing, very rarely happens If you rarely see this problem and it resolves itself when you save again, it is likely working as intended (though maybe not as well as the old development server). Here is the full explanation. In the original development server, requests were processed sequentially so file modifications were checked with each request. The new development server supports concurrent and long-running requests, which makes this model problematic - you don't want to kill an instance that is currently handling another requests if a change is detected. So the change detection is done asynchronously about every 2 seconds. If a change is detected then the development server waits until all currently active requests finish and then restarts the instance. So it is expected that sometimes you may have to wait 2-3 seconds for your changes to be seen. We can reduce that time but it will result in more CPU usage. Cheers, Brian On Sunday, March 17, 2013 8:15:32 PM UTC+2, PK wrote: As you have probably read 1.7.6 is slated to ship with a new development server. Last weekend, I finally moved over our infrastructure to the new development server for python after a few months of on and off efforts. I was already using HDR sqlite so no surprises there. Many thanks to Brian and Sam for fixes and assistance that goes back to December. Here are some things that might help your transition and wish I knew on day one: (Unfortunately a productivity related blocker bug see point 5 below still keeps me on the old server, but all the regression tests now run with the new server) 1. The new server is not command line option for option backwards compatible. It seems that the Google team took a chance to clean up a little bit, so expect that you will need to make some changes in your scripts. Take a look at the new options before you do anything else. In particular: 1.1 The host name is now specified using *--host* instead of -a or --address 1.2 The admin part now runs as a different server so if you run multiple configurations on the same machine make sure you use the options * --admin_host* and *--admin_port* to set them 1.3 In addition to the datastore data and the blob data, logs now end up in some structured storage. I did not realize that at the beginning and running multiple configuration side by side was causing corruptions and weird crashes. So make sure you set the *--logs_path* to point to where the logs sqlite file should be located. I am not sure what the default is. 1.4 dev_appserver.py would not run the tasks by default if you set the * --enable_tasks=no* but you could manually run them from the admin console. In the new server this option turns off the tasks completely, not clear if this is a bug or a new feature. 1.5 -d does not cut it for debug, you now needs to say *--log_level=debug * 2. Observed performance is about the same as before both from my experience and the measurements (it took a while to get there but it seems that now we are there) 3. I was using remote_api without specifying both the host and port of my server before. devappserver2.py is pickier. It took Sam and I several interaction to nail this issue. 4. The new server uses different options for log_levels of your code vs the dev server code (--*dev_appserver_log_level* vs --*log_level*). This is awesome but might be confusing!! 5. The new server does not seem to pick up application file modifications. Please star issue https://code.google.com/** p/appengine-devappserver2-**experiment/issues/detail?id=54https://code.google.com/p/appengine-devappserver2-experiment/issues/detail?id=54 ** if you experience the same issue. Best, PK http://www.gae123.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com.
[google-appengine] Re: New experimental development server for Python
We have released a new version of the Python development server. A descendant of this version will be included in the App Engine 1.7.7 SDK release so any testing or other feedback would be greatly appreciated! To download the cross-platform package for Python: https://appengine-devappserver2-experiment.googlecode.com/files/devappserver2-v0.8.zip New Features: - Cloud Endpoints (see http://goo.gl/m2GjG) - Integration points for Python tools (see http://goo.gl/3oSBp) - Add support for tasks that target backends Bug fixes: - Fix the size display in blobstore viewer - Fix the styling of the Purge button in the taskqueue UI - Fix orphan runtime processes on shutdown - Make --log_level=debug display debug-level logging Cheers, Brian On 25 February 2013 16:28, Brian Quinlan bquin...@google.com wrote: We have released a new version of the Python development server. This version will be included in the App Engine 1.7.6 SDK release so any testing or other feedback would be greatly appreciated! Aside from a long list of bug fixes, this version adds experimental Go support! To download the cross-platform package for Python: https://code.google.com/p/appengine-devappserver2-experiment/downloads/detail?name=devappserver2-v0.7.zip To download the Linux amd64-only package for Python and Go: https://code.google.com/p/appengine-devappserver2-experiment/downloads/detail?name=devappserver2-v0.7_linux_amd64.zip Changes: - Fixes testbed support - Tests now pass on Windows and Mac OS X - Multiple set-cookies headers no longer combined - Some minor UI improvements - Invalid login cookies are ignored - Go runtime support! - Use a threadsafe subprocess implementation - Fixed Google Cloud SQL support on windows - Performance: - use mtime checking for config file update checks - switch to one thread per async work item - set the backlog for WsgiServer sockets to 100 Thanks for your help! Cheers, Brian -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Re: New experimental development server for Python
We have released a new version of the Python development server. This version will be included in the App Engine 1.7.6 SDK release so any testing or other feedback would be greatly appreciated! Aside from a long list of bug fixes, this version adds experimental Go support! To download the cross-platform package for Python: https://code.google.com/p/appengine-devappserver2-experiment/downloads/detail?name=devappserver2-v0.7.zip To download the Linux amd64-only package for Python and Go: https://code.google.com/p/appengine-devappserver2-experiment/downloads/detail?name=devappserver2-v0.7_linux_amd64.zip Changes: - Fixes testbed support - Tests now pass on Windows and Mac OS X - Multiple set-cookies headers no longer combined - Some minor UI improvements - Invalid login cookies are ignored - Go runtime support! - Use a threadsafe subprocess implementation - Fixed Google Cloud SQL support on windows - Performance: - use mtime checking for config file update checks - switch to one thread per async work item - set the backlog for WsgiServer sockets to 100 Thanks for your help! Cheers, Brian -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Data queries returning with partial data after migration
Hi Eran, Non-ancestor queries are eventually consistent. See: https://developers.google.com/appengine/docs/python/datastore/queries#Data_Consistency Cheers, Brian On Fri, Feb 15, 2013 at 5:50 AM, Eran Schaffer eran.schaf...@gmail.com wrote: Hi, I finished migrating my DB from m/s to high replication, and while I am seeing all the data using datastore viewer and GQL , when my website's code does a similar query I only get partial results, meaning not all entries in the DB get back. Did anyone experience something similar? FYI, I deployed my app before migration , and the relevant indexes for these classes are tagged as Serving Thanks, ES -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] markupsafe removed in 1.7.5?
Hi Thomas, markupsafe was not removed in 1.7.5. Could you file a bug at: https://code.google.com/p/googleappengine/issues Please include your platform and ideally a test application that demonstrates the problem (or at least an app.yaml file [without the handlers section])? Also, do you have a copy of markupsafe installed locally or in the application directory? Sorry for the trouble! Cheers, Brian On Thu, Feb 14, 2013 at 8:24 AM, Thomas Marban tho...@marban.com wrote: I'm getting an [ImportError: No module named markupsafe] Error after the 1.7.5 upgrade (markupsafe used to be included via app.yaml) -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: New experimental development server for Python
On Sat, Feb 9, 2013 at 1:31 AM, Jacob G ja...@fareclock.com wrote: Hi Brian, This new development server sounds very exciting. I haven't had a chance yet to try it, but I look forward to checking out all these great features. One question I have is regarding your bullet point Add automatic conversion from the datastore file stub to the sqlite stub. Does this feature address the conversion failure problem discussed here: http://stackoverflow.com/questions/14627143/gae-port-sqlite-fails-on-local-dev-server I don't understand the reason for that failure so I can't say for sure. It looks like the discussion is ongoing here: https://code.google.com/p/googleappengine/issues/detail?id=7717 Cheers, Brian Thanks. On Wednesday, February 6, 2013 2:06:03 AM UTC-5, Brian Quinlan wrote: We have released a new version of the new Python development server. This development server will replace the existing one in the 1.7.6 release so we'd greatly appreciate any feedback. To download: https://code.google.com/p/appengine-devappserver2-experiment/downloads/ Features: - Add a blobstore viewer (thanks Alex!) - Several UI improvements (thanks Alex) - Add protorpc support - Add automatic conversion from the datastore file stub to the sqlite stub. - Add update checking Bug fixes: - Allow starting background threads from backends handling /_ah/start requests. - Set DEFAULT_VERSION_HOSTNAME - Avoid crashing on empty backends - Set environ['SERVER_NAME'] to match environ['HTTP_HOST'] - Make exception logging more robust - Bypass login: required for inbound xmpp requests from the admin UI - Stop raising KeyboardInterrupts in async_quit and for SIGTERM and SIGINT - Gracefully handle the case where 'headers' or 'response_code' are not set - Always filter x-appengine-fake-is-admin and x-appengine-fake-logged-in headers Cheers, Brian On Mon, Jan 21, 2013 at 3:11 PM, Brian Quinlan bqui...@google.com wrote: We have released a new version of the experimental Python development server. We think that it is very usable at this point and has some significant improvements over the existing version. To download: http://code.google.com/p/appengine-devappserver2-experiment/downloads/detail?name=devappserver2-v0.5.zip Changes: - a functionally complete administrative UI (it is also a lot more polished but still needs some work) - more reliable task execution - (experimental) support for local Cloud SQL development - lots of bug fixes! As always, feedback is very welcome! Cheers, Brian On Tue, Nov 27, 2012 at 1:14 PM, Brian Quinlan bqui...@google.com wrote: Hi, I'm happy to announce that the App Engine team is working on a new development server for Python (and eventually Go). The new development server aims to provide higher performance for complex applications and better emulation of the App Engine deployment environment. There are several features that are currently missing: - support for backends - the administrative UI (i.e. _ah/admin) - Go Development is still at an early stage so I'm sure that there are lots of bugs to be found and fixed. Which is why we'd appreciate your help in testing and, if you are feeling keen, fixing these issues. You can download the latest version from: http://code.google.com/p/appengine-devappserver2-experiment/downloads/ Or work directly with the git repository: http://code.google.com/p/appengine-devappserver2-experiment/source/checkout There is also a public discussion group associated with the project: https://groups.google.com/forum/?fromgroups#!forum/appengine-devappserver2-discuss Thanks for your help! Cheers, Brian - App Engine Python Team -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Re: New experimental development server for Python
We have released a new version of the new Python development server. This development server will replace the existing one in the 1.7.6 release so we'd greatly appreciate any feedback. To download: https://code.google.com/p/appengine-devappserver2-experiment/downloads/ Features: - Add a blobstore viewer (thanks Alex!) - Several UI improvements (thanks Alex) - Add protorpc support - Add automatic conversion from the datastore file stub to the sqlite stub. - Add update checking Bug fixes: - Allow starting background threads from backends handling /_ah/start requests. - Set DEFAULT_VERSION_HOSTNAME - Avoid crashing on empty backends - Set environ['SERVER_NAME'] to match environ['HTTP_HOST'] - Make exception logging more robust - Bypass login: required for inbound xmpp requests from the admin UI - Stop raising KeyboardInterrupts in async_quit and for SIGTERM and SIGINT - Gracefully handle the case where 'headers' or 'response_code' are not set - Always filter x-appengine-fake-is-admin and x-appengine-fake-logged-in headers Cheers, Brian On Mon, Jan 21, 2013 at 3:11 PM, Brian Quinlan bquin...@google.com wrote: We have released a new version of the experimental Python development server. We think that it is very usable at this point and has some significant improvements over the existing version. To download: http://code.google.com/p/appengine-devappserver2-experiment/downloads/detail?name=devappserver2-v0.5.zip Changes: - a functionally complete administrative UI (it is also a lot more polished but still needs some work) - more reliable task execution - (experimental) support for local Cloud SQL development - lots of bug fixes! As always, feedback is very welcome! Cheers, Brian On Tue, Nov 27, 2012 at 1:14 PM, Brian Quinlan bquin...@google.com wrote: Hi, I'm happy to announce that the App Engine team is working on a new development server for Python (and eventually Go). The new development server aims to provide higher performance for complex applications and better emulation of the App Engine deployment environment. There are several features that are currently missing: - support for backends - the administrative UI (i.e. _ah/admin) - Go Development is still at an early stage so I'm sure that there are lots of bugs to be found and fixed. Which is why we'd appreciate your help in testing and, if you are feeling keen, fixing these issues. You can download the latest version from: http://code.google.com/p/appengine-devappserver2-experiment/downloads/ Or work directly with the git repository: http://code.google.com/p/appengine-devappserver2-experiment/source/checkout There is also a public discussion group associated with the project: https://groups.google.com/forum/?fromgroups#!forum/appengine-devappserver2-discuss Thanks for your help! Cheers, Brian - App Engine Python Team -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Index is still building for master/slave - HRD migration
Hi Barry, On Fri, Jan 25, 2013 at 9:20 AM, Barry Hunter barrybhun...@gmail.comwrote: The time it takes for the index is not a function of how big your data is, but how indexes are sitting in the queue in front of you. AppEngine processes index builds sequentically, so have to wait your turn. Someone else may well have a really big index that is holding things up. Actually, index builds are now done concurrently. But there is still a lot of complexity in index construction due to the eventually consistent nature of datastore queries. Cheers, Brian On Thu, Jan 24, 2013 at 9:42 PM, Tayeb Karim t...@google.com wrote: Hi, I started the migration a few hours ago and the indexes seem to still be building on the new application (ggc-hrd) Any pointers to see what's up? The data size is pretty small; For the tables that are still being indexed; CacheNode: 2k entries, 323KB Maintenance: 42 entries, 6KB -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/-KyKMVJuAJEJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: New experimental development server for Python
That should work. Are you creating a background thread on a frontend or a backend instance? Cheers, Brian On Thu, Jan 24, 2013 at 6:38 AM, Jeff Schnitzer j...@infohazard.org wrote: BackgroundThreadLimitReachedError -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: New experimental development server for Python
On Thu, Jan 24, 2013 at 1:41 PM, Jeff Schnitzer j...@infohazard.org wrote:
It's a resident backend.
Jeff
Are you sure that this happens on the first attempt to create a
background thread? If so, then could you could try to find this block
in google/appegine/tools/devappserver2/dispatcher.py:
try:
inst.reserve_background_thread()
except instance.CannotAcceptRequests:
+logging.exception('Cannot create background thread.'
raise request_info.BackgroundThreadLimitReachedError()
And see what logging you see?
Cheers,
Brian
On Wed, Jan 23, 2013 at 3:50 PM, Brian Quinlan bquin...@google.com wrote:
That should work.
Are you creating a background thread on a frontend or a backend instance?
Cheers,
Brian
On Thu, Jan 24, 2013 at 6:38 AM, Jeff Schnitzer j...@infohazard.org wrote:
BackgroundThreadLimitReachedError
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: New experimental development server for Python
We have released a new version of the experimental Python development server. We think that it is very usable at this point and has some significant improvements over the existing version. To download: http://code.google.com/p/appengine-devappserver2-experiment/downloads/detail?name=devappserver2-v0.5.zip Changes: - a functionally complete administrative UI (it is also a lot more polished but still needs some work) - more reliable task execution - (experimental) support for local Cloud SQL development - lots of bug fixes! As always, feedback is very welcome! Cheers, Brian On Tue, Nov 27, 2012 at 1:14 PM, Brian Quinlan bquin...@google.com wrote: Hi, I'm happy to announce that the App Engine team is working on a new development server for Python (and eventually Go). The new development server aims to provide higher performance for complex applications and better emulation of the App Engine deployment environment. There are several features that are currently missing: - support for backends - the administrative UI (i.e. _ah/admin) - Go Development is still at an early stage so I'm sure that there are lots of bugs to be found and fixed. Which is why we'd appreciate your help in testing and, if you are feeling keen, fixing these issues. You can download the latest version from: http://code.google.com/p/appengine-devappserver2-experiment/downloads/ Or work directly with the git repository: http://code.google.com/p/appengine-devappserver2-experiment/source/checkout There is also a public discussion group associated with the project: https://groups.google.com/forum/?fromgroups#!forum/appengine-devappserver2-discuss Thanks for your help! Cheers, Brian - App Engine Python Team -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Error in importing python matplotlib in GAE
From the documentation (https://developers.google.com/appengine/docs/python/tools/libraries27): Note: The experimental release of matplotlib is not supported on the development server. You can still add matplotlib to the libraries list, but it will raise an exception when imported. Cheers, Brian On Sat, Dec 29, 2012 at 7:30 AM, Xin Meng mengxin891...@gmail.com wrote: Hi guys, I was testing out the new GAE sdk 1.7.4 python version with the matplotlib support, and I have encountered the following error: WARNING 2012-12-28 20:27:55,009 py_zipimport.py:139] Can't open zipfile D:\Python27\lib\site-packages\python_memcached-1.48-py2.7.egg: IOError: [Errno 13] file not accessible: 'D:\\Python27\\lib\\site-packages\\python_memcached-1.48-py2.7.egg' WARNING 2012-12-28 20:27:55,025 py_zipimport.py:139] Can't open zipfile D:\Python27\lib\site-packages\setuptools-0.6c12dev_r88846-py2.7.egg: IOError: [Errno 13] file not accessible: 'D:\\Python27\\lib\\site-packages\\setuptools-0.6c12dev_r88846-py2.7.egg' WARNING 2012-12-28 20:27:55,976 dev_appserver_import_hook.py:557] Blocking access to skipped file d:\nus\quality control sms system\webapp5\.matplotlibrc WARNING 2012-12-28 20:27:55,976 dev_appserver_import_hook.py:557] Blocking access to skipped file d:\nus\quality control sms system\webapp5\~\.matplotlibrc WARNING 2012-12-28 20:27:55,976 dev_appserver_import_hook.py:557] Blocking access to skipped file d:\nus\quality control sms system\webapp5\~\.matplotlib ERROR2012-12-28 20:27:55,976 wsgi.py:219] Traceback (most recent call last): File E:\Program Files\Google\google_appengine\google\appengine\runtime\wsgi.py, line 196, in Handle handler = _config_handle.add_wsgi_middleware(self._LoadHandler()) File E:\Program Files\Google\google_appengine\google\appengine\runtime\wsgi.py, line 255, in _LoadHandler handler = __import__(path[0]) File E:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py, line 692, in Decorate return func(self, *args, **kwargs) File E:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py, line 1766, in load_module return self.FindAndLoadModule(submodule, fullname, search_path) File E:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py, line 692, in Decorate return func(self, *args, **kwargs) File E:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py, line 1630, in FindAndLoadModule description) File E:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py, line 692, in Decorate return func(self, *args, **kwargs) File E:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py, line 1577, in LoadModuleRestricted description) File D:\NUS\Quality Control SMS System\Webapp5\main.py, line 15, in module import Qualitics_Logic File E:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py, line 692, in Decorate return func(self, *args, **kwargs) File E:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py, line 1766, in load_module return self.FindAndLoadModule(submodule, fullname, search_path) File E:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py, line 692, in Decorate return func(self, *args, **kwargs) File E:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py, line 1630, in FindAndLoadModule description) File E:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py, line 692, in Decorate return func(self, *args, **kwargs) File E:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py, line 1577, in LoadModuleRestricted description) File D:\NUS\Quality Control SMS System\Webapp5\Qualitics_Logic.py, line 4, in module import matplotlib.mlab as mlab File E:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py, line 692, in Decorate return func(self, *args, **kwargs) File E:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py, line 1766, in load_module return self.FindAndLoadModule(submodule, fullname, search_path) File E:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py, line 692, in Decorate return func(self, *args, **kwargs) File E:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py, line 1630, in FindAndLoadModule description) File E:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py, line 692, in Decorate return func(self, *args,
Re: [google-appengine] Re: New experimental development server for Python
Hi PK, Thanks for the bug description! We understand the cause (the lack of the dev~ prefix on app ids) and will fix it soon [1]. Cheers, Brian [1] soon may not be that soon due to the holidays :-) On Wed, Dec 12, 2012 at 5:47 PM, PK p...@gae123.com wrote: Great... Thanks!! More regression tests pass now but one of them fails with the message below. I am on 1.7.3, all my tests pass with the regular dev server. BadRequestError: remote_api supports transactional queries only in the high-replication datastore. Is this something consistent with the bugs you already track? Thanks, PK On Dec 11, 2012, at 8:59 PM, Brian Quinlan bquin...@google.com wrote: We have released a new version of the development server: To download: http://code.google.com/p/appengine-devappserver2-experiment/downloads/detail?name=devappserver2-v0.3.zip Changes: - threads are implicitly joined on request exit - basic taskqueue UI (more later) - fixes for virtualenv - windows unit test fixes - reduced logging noise - correct logout url generation As always, feedback is welcome! Cheers, Brian On Tue, Nov 27, 2012 at 1:14 PM, Brian Quinlan bquin...@google.com wrote: Hi, I'm happy to announce that the App Engine team is working on a new development server for Python (and eventually Go). The new development server aims to provide higher performance for complex applications and better emulation of the App Engine deployment environment. There are several features that are currently missing: - support for backends - the administrative UI (i.e. _ah/admin) - Go Development is still at an early stage so I'm sure that there are lots of bugs to be found and fixed. Which is why we'd appreciate your help in testing and, if you are feeling keen, fixing these issues. You can download the latest version from: http://code.google.com/p/appengine-devappserver2-experiment/downloads/ Or work directly with the git repository: http://code.google.com/p/appengine-devappserver2-experiment/source/checkout There is also a public discussion group associated with the project: https://groups.google.com/forum/?fromgroups#!forum/appengine-devappserver2-discuss Thanks for your help! Cheers, Brian - App Engine Python Team -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] New experimental development server for Python
On Fri, Dec 14, 2012 at 12:30 PM, PK p...@gae123.com wrote: OK Thanks... I filed issue #16 to track it. Thanks! Another test code showstopper, is the following: We have some test code that waits until all tasks have executed and only then continues. The way it does this is by scraping the /_ah/admin/queueues page until all the counters are 0. Without admin console this part also fails. I assume you will provide an admin console at some point so this will take care of it. Could you try adding this to your app.yaml: - url: /admin_console/.* script: $PYTHON_LIB/google/appengine/ext/admin Cheers, Brian (BTW, this is really old ugly code that will also fail next time the UI changes, it is probably time to start using the QueueStatistics for the same purpose, incidentally it just became fully supported with 1.7.4). On Dec 11, 2012, at 10:54 PM, PK p...@gae123.com wrote: I looked in more detail the stack trace and hits an issue I had seen before that was fixed when bug 3643 was fixed. The bug was fixed in 1.7.2, I am wondering whether you could have forked some code before that fix went in? On Dec 11, 2012, at 10:47 PM, PK p...@gae123.com wrote: Great... Thanks!! More regression tests pass now but one of them fails with the message below. I am on 1.7.3, all my tests pass with the regular dev server. BadRequestError: remote_api supports transactional queries only in the high-replication datastore. Is this something consistent with the bugs you already track? Thanks, PK On Dec 11, 2012, at 8:59 PM, Brian Quinlan bquin...@google.com wrote: We have released a new version of the development server: To download: http://code.google.com/p/appengine-devappserver2-experiment/downloads/detail?name=devappserver2-v0.3.zip Changes: - threads are implicitly joined on request exit - basic taskqueue UI (more later) - fixes for virtualenv - windows unit test fixes - reduced logging noise - correct logout url generation As always, feedback is welcome! Cheers, Brian On Tue, Nov 27, 2012 at 1:14 PM, Brian Quinlan bquin...@google.com wrote: Hi, I'm happy to announce that the App Engine team is working on a new development server for Python (and eventually Go). The new development server aims to provide higher performance for complex applications and better emulation of the App Engine deployment environment. There are several features that are currently missing: - support for backends - the administrative UI (i.e. _ah/admin) - Go Development is still at an early stage so I'm sure that there are lots of bugs to be found and fixed. Which is why we'd appreciate your help in testing and, if you are feeling keen, fixing these issues. You can download the latest version from: http://code.google.com/p/appengine-devappserver2-experiment/downloads/ Or work directly with the git repository: http://code.google.com/p/appengine-devappserver2-experiment/source/checkout There is also a public discussion group associated with the project: https://groups.google.com/forum/?fromgroups#!forum/appengine-devappserver2-discuss Thanks for your help! Cheers, Brian - App Engine Python Team -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: New experimental development server for Python
We have released a new version of the development server: To download: http://code.google.com/p/appengine-devappserver2-experiment/downloads/detail?name=devappserver2-v0.3.zip Changes: - threads are implicitly joined on request exit - basic taskqueue UI (more later) - fixes for virtualenv - windows unit test fixes - reduced logging noise - correct logout url generation As always, feedback is welcome! Cheers, Brian On Tue, Nov 27, 2012 at 1:14 PM, Brian Quinlan bquin...@google.com wrote: Hi, I'm happy to announce that the App Engine team is working on a new development server for Python (and eventually Go). The new development server aims to provide higher performance for complex applications and better emulation of the App Engine deployment environment. There are several features that are currently missing: - support for backends - the administrative UI (i.e. _ah/admin) - Go Development is still at an early stage so I'm sure that there are lots of bugs to be found and fixed. Which is why we'd appreciate your help in testing and, if you are feeling keen, fixing these issues. You can download the latest version from: http://code.google.com/p/appengine-devappserver2-experiment/downloads/ Or work directly with the git repository: http://code.google.com/p/appengine-devappserver2-experiment/source/checkout There is also a public discussion group associated with the project: https://groups.google.com/forum/?fromgroups#!forum/appengine-devappserver2-discuss Thanks for your help! Cheers, Brian - App Engine Python Team -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: My app engine wishlist
Hi, On Fri, Nov 30, 2012 at 8:10 PM, Verilocos W verilo...@gmail.com wrote: This is really a good wishlist. I also wish many items on that! However, 10, 11, 12, yeah three years. Santa Claus did not hear us! Is GAE obsoleted? On Thursday, October 22, 2009 6:11:28 PM UTC+8, Daniel Florey wrote: I'm working with app engine (Java) for quite a while now and as Christmas is near, it is time to publish my wishlist: Improvements in the admin interface: - Logs: Right now when scrolling through the pages after a while the formatting is gone and the back link is not working any more Respect the users timezone. It's very annoying to recalculate the timestamps to find log statements Delete log to start a fresh log from scratch Download logs button right from the admin interface - Cron Jobs: Let us change the cron settings right from the console. Would be great for debugging purposes What would happen the next time you run appcfg update? - Task Qeues: Please give us a delete all button to delete all tasks from a specific queue Does the Purge Queue button not do what you want? Better paging through the tasks (ordered by ETA etc.) There is an Order by: Task Name | Task ETA control above the list of tasks in the queue viewer. Is that not what you want? - Indexes: Delete index right from the admin interface would be awesome What would happen the next time you run appcfg update? - Data viewer: Please show the Key(name or id) in the data grid from the parent entity when entity has ancestors Delete all button for the whole datastore Delete all button for a certain kind of entities Improve GQL to query entities by key / parent You can query by ancestor. See: https://developers.google.com/appengine/docs/python/datastore/gqlreference Editing entities often leads to Internal server error Backup/Restore of datastore as .zip right from the admin interface (or store backup on some Google servers) Improvements in the SDK: Don't know if this works with python, but it would be cool to be able to delete entities based on a query without fetching all keys and deleting by keys (e.g. something like a super-fast DELETE * FROM kind) Some more free CPU quota ;-) That's it. Let's see what Santa Claus can do for me... It would be awesome if you could fine issues for these feature requests! Cheers, Brian Cheers, Daniel -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/J-EeTM2r7E0J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] New experimental development server for Python
Hi, I'm happy to announce that the App Engine team is working on a new development server for Python (and eventually Go). The new development server aims to provide higher performance for complex applications and better emulation of the App Engine deployment environment. There are several features that are currently missing: - support for backends - the administrative UI (i.e. _ah/admin) - Go Development is still at an early stage so I'm sure that there are lots of bugs to be found and fixed. Which is why we'd appreciate your help in testing and, if you are feeling keen, fixing these issues. You can download the latest version from: http://code.google.com/p/appengine-devappserver2-experiment/downloads/ Or work directly with the git repository: http://code.google.com/p/appengine-devappserver2-experiment/source/checkout There is also a public discussion group associated with the project: https://groups.google.com/forum/?fromgroups#!forum/appengine-devappserver2-discuss Thanks for your help! Cheers, Brian - App Engine Python Team -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Dev Server on PyPy
Hi Michael, On Tue, Nov 6, 2012 at 3:51 PM, Michael Sander michael.san...@gmail.com wrote: I know it's not an officially supported platform, but has anyone tried playing around with the App Engine dev server on PyPy? Its JIT should make the dev server much faster than when running on cpython. I'm not sure what performance improvement you are looking for. Improvements in your application code seem like they would be deceptive since they disappear in production. Or are you looking for improved API or general serving performance? Cheers, Brian With a bit of tinkering, I can get it to start up, but I quickly get the following exception whenever I try request a page on the dev server: File C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py, line 1606, in FindModuleRestricted raise import_error ImportError: Access to module file denied: c:\PyPy19\lib_pypy\_functools.py The _functools.py file exists on my machine and is readable. Any idea why it's broken? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/fIAcI0X-9DIJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: IE/Opera + Dev Server = slow, problematic
Hi Kaan, On Fri, Oct 12, 2012 at 6:20 AM, Kaan Soral kaanso...@gmail.com wrote: I just had this idea now, If there is any http headers or modifications that will prohibit IE/Opera from making async requests we may try those. (Make responses HTTP 1.0 etc.) It should be a not so hard modification on maybe dev_appserver.py or other files that prepare the response Any ideas? The App Engine team is currently working on a new development server that has a multi-threaded frontend and supports concurrent requests. The announcement is here: https://groups.google.com/d/topic/google-appengine/TCQuJpF44cY/discussion And you can get the code from the Google Code project: http://code.google.com/p/appengine-devappserver2-experiment/ Cheers, Brian On Wednesday, October 10, 2012 9:44:33 PM UTC+3, Adam Lofting wrote: I've been suffering with the same experience testing App Engine in IE running on Windows 7. Weirdly it runs fine in IE Tester (all versions), which has been my solution for now. On Monday, September 24, 2012 2:38:16 AM UTC+2, Kaan Soral wrote: I can't use IE or Opera for testing with development server (local). For some reason some of the scripts/images don't load. For the ones that load, it takes a huge amount of time for them to load compared to Firefox/Chrome. When you make a request, it takes some time for it to show up on the logs. (IE is much much worse than opera) Do you have similar experiences? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/vhwDmScXlKEJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Tracing the source of datastore reads/writes
Hi Phil, Have you looked at Appstats? https://developers.google.com/appengine/docs/java/tools/appstats Cheers, Brians On Mon, Oct 8, 2012 at 12:42 PM, Phil McDonnell phil.a.mcdonn...@gmail.com wrote: Is there a way to trace where the bulk of datastore reads are coming from in appengine? I'm on Java and I've recently noticed that our datastore reads are starting to cost a significant amount of money per day. I'd like to optimize our code, but I'm not sure where to begin since our app is fairly large and complex. Are there any good tools out there associate with AppEngine to help with this? Thanks, Phil -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Unit testing with webtest problem
On Tue, Sep 11, 2012 at 4:00 PM, Richard Arrano rickarr...@gmail.com wrote:
What I'm asking more specifically is what does App Engine do, if anything,
to os.putenv? Because using the App Engine console, I can do things like:
os.environ[x] = ndb.Model()
and access it with os.environ[x] without a problem. However, when I pull
up the interpreter outside of Python and attempt to put anything other than
a string into os.environ, it will balk:
import os
import os
os.environ[abc] = {}
Traceback (most recent call last):
File stdin, line 1, in module
File C:\Python27\lib\os.py, line 420, in __setitem__
putenv(key, item)
TypeError: must be string, not dict
So it seems like App Engine does something to the putenv call where it
serializes model instances into strings first, and if that's the case must
also do a similar operation to os.getenv to deserialize them. What is App
Engine doing, if anything to the getenv/putenv calls?
Modifying os.environ in App Engine doesn't actually result in
getenv/putenv calls. You can see the class used here:
http://googleappengine.googlecode.com/svn/trunk/python/google/appengine/runtime/request_environment.py
Please file a bug if the incompatibility between regular Python and
App Engine os.environ is causing you problems:
http://code.google.com/p/googleappengine/issues/list
Cheers,
Brian
Thanks,
Richard
On Tuesday, September 11, 2012 6:09:16 AM UTC-7, Guido van Rossum wrote:
You'll probably get more help from StackOverflow.com. You'll need to
provide more info, nobody can help you debug this with just that traceback
information unless they're psychic.
On Sunday, September 9, 2012 10:06:33 AM UTC-7, Richard Arrano wrote:
Hello,
I've been using webtest to unit test my application and I've encountered
a strange issue. I wrap many of my get/post handlers in decorators and in
some of those decorators, I put ndb.Model instances in os.environ for later
use in the subsequent handler. This works on my local dev server and in
production. However, when I run nosetests it always gives me an error:
os.environ[user] = user
File C:\Python27\lib\os.py, line 420, in __setitem__
putenv(key, item)
TypeError: must be string, not User
Any ideas on how to mitigate this so my tests won't error out at this
point?
Thanks,
Richard
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/er6JJQK-EJoJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Not able to vacuum indexes for my application
On Tue, Aug 21, 2012 at 8:11 AM, prakhil samar prakhilsa...@gmail.com wrote: Actually i do not get an option to press Y/N/A; As soon as i run the command it directly gives me error, that i listed in the image You are redirecting stdin to the file ...\pwd. Since that file only contains your password, appcfg.py generates an EOFError when trying to read your answer to the confirmation prompts. Try running appcfg.py without --passin. Cheers, Brian On Tuesday, August 21, 2012 5:33:36 PM UTC+5:30, Takashi Matsuo (Google) wrote: Are you sure that you pushed 'y' then 'return' key when prompted 'Are you sure you want to delete this index? N/y/a'? On Tue, Aug 21, 2012 at 7:53 PM, prakhil samar prakhi...@gmail.com wrote: Anyone with a solution??? On Tuesday, August 21, 2012 12:40:38 PM UTC+5:30, prakhil samar wrote: Hi GAE team, I m not able to Vacuum indexes for my application. I ran this command: appcfg.py vacuum_indexes path I have attached the command prompt image too Please suggest how to delete the unused indexes ? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/85PEVKSApkIJ. To post to this group, send email to google-a...@googlegroups.com. To unsubscribe from this group, send email to google-appengi...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Takashi Matsuo | Developers Advocate | tma...@google.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/uCJIlGozQrsJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Problem with ctypes
Hi Yuval, Take a look at this link: https://developers.google.com/appengine/kb/libraries Cheers, Brian On Mon, Jul 23, 2012 at 3:11 AM, Yuval Kalev yuvalka...@gmail.com wrote: I seem to be having problems with importing ctypes to use for an MGRS module I want to load. I keep getting this error: ImportError: No module named _ctypes Reading posts I read that version 2.7.1-2 might have fixed this issue. Still I cannot get it to work. Any ideas? Thanks, - Yuval -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/S7pu8and_AEJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] how to get all entities of one Model for ndb?
db.Model.query() On Thu, Jun 21, 2012 at 2:52 AM, saintthor saintt...@gmail.com wrote: as db.Model.all() does. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/aPHU2eqM_3AJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Simple cron job in google app engine (python)
On Fri, Jun 8, 2012 at 6:35 PM, afroze 08beeabaqap...@seecs.edu.pk wrote:
I need help in implementing a simple cron job in GAE (python).
According to what I understood from appengine documentation, I made a file
cron.yaml in the application root directory with the following content:
cron:
- description: blah blah
url: /crontask
schedule: every 1 minute
And my app.yaml file has the following content:
application: template-123
version: 1
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /.*
script: template-123.app
Do you have a file called template-123.py? template-123 is not a
valid Python module name so that script will not be importable.
Cheers,
Brian
I made all my application code (cron and other parts) into a single file
template-123.py. In the code I implement cron in the following way:
class CronTask(Handler):
def post(self):
i=25
number = Birthday(day = i)
number.put()
And I tell the code to use this class for cron by stating: ('/crontask',
CronTask).
However no new entries are uploaded to the datastore (as I believe they
should be). And I know that it isn't a problem with the way I'm accessing
the data store because when I try to do the same thing manually (upload
entries to the data store in my non-cron part of application) it returns
with appropriate results.
So I need some guidance as to what might I be doing wrong or missing? Do
have to make some more changes to the yaml files or add some other libraries
or files (i think i read somewhere about cron.xml file but it wasn't
mentioned in the documentation..) etc?
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/VFd3h57Uk6gJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Is it possible to use multiprocessing Pool in Python 2.7 on App Engine?
Have you considered using concurrent.futures: http://pypi.python.org/pypi/futures ? Cheers, Brian On Sun, May 27, 2012 at 5:47 PM, Luca de Alfaro luca.de.alf...@gmail.com wrote: I have some processing that would benefit from parallelism -- the multiprocessing Pool class would be ideal. Is it available in App Engine? The Mapping API does not work for my purpose, since it is in-memory processing that I am trying to do. Thanks! Luca -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/yzxvmQQyWAIJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Is it possible to use multiprocessing Pool in Python 2.7 on App Engine?
On Mon, May 28, 2012 at 2:08 AM, Luca de Alfaro luca.de.alf...@gmail.com wrote: No, and that would be perfect... is that supported on AppEngine? The ThreadPool functionality will work fine but you might have to remove the multiprocessing imports. And btw, how do I find out about which packages are supported? There is no list of packages supported on App Engine - you have to look into the ones that you want to use. I did not know that some futures packages from pypi could by supported. I ask because there are a few more packages (scipy, for instance) that I really wish would be supported... Many (possibly most) packages from pypi are supported. But packages won't work if: - they have C/C++/FORTRAN components (scipy has these) - they require features that App Engine doesn't support such as a writeable filesystem Cheers, Brian Luca On Sunday, May 27, 2012 2:39:18 AM UTC-7, Brian Quinlan wrote: Have you considered using concurrent.futures: http://pypi.python.org/pypi/futures ? Cheers, Brian On Sun, May 27, 2012 at 5:47 PM, Luca de Alfaro luca.de.alf...@gmail.com wrote: I have some processing that would benefit from parallelism -- the multiprocessing Pool class would be ideal. Is it available in App Engine? The Mapping API does not work for my purpose, since it is in-memory processing that I am trying to do. Thanks! Luca -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/yzxvmQQyWAIJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. On Sunday, May 27, 2012 2:39:18 AM UTC-7, Brian Quinlan wrote: Have you considered using concurrent.futures: http://pypi.python.org/pypi/futures ? Cheers, Brian On Sun, May 27, 2012 at 5:47 PM, Luca de Alfaro luca.de.alf...@gmail.com wrote: I have some processing that would benefit from parallelism -- the multiprocessing Pool class would be ideal. Is it available in App Engine? The Mapping API does not work for my purpose, since it is in-memory processing that I am trying to do. Thanks! Luca -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/yzxvmQQyWAIJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/HNyRzPrDYQUJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Feedback on experimental Mac Launcher for Python
Hi Alex, Thanks very much for the feedback! On Thu, May 17, 2012 at 4:54 PM, alex a...@cloudware.it wrote: Hey Brian, I did try the Mac OS version a couple times. The idea is nice. Not only because of to reduce the number of deps. I think it's good to have dev appserver running in a clean, separate environment, closer to how it works in production (e.g. I sometimes mess up my python 2.7 and site packages, making them do weird stuff, experimenting). The reason I'm not using it daily (both with and w/o bundled python) is that I'd have it frozen (the Launcher app) so I had to forse quit it from Activity Monitor. And it happens too often so, at the end of the day I find myself using SDK *.py scripts directly from the Terminal (which I don't mind actually). Unfortunately, I can't really provide some deterministic steps to reproduce freezing right now. But, I noticed it almost always happend when I click stop (to stop the dev appserver). Did this problem occur with the regular launcher or just the experimental one? Cheers, Brian -- alex On Thursday, May 17, 2012 6:59:19 AM UTC+2, Brian Quinlan wrote: Hi, Along with the usual components of the App Engine 1.6.5 release, we also released an experimental version of the Mac OS X Launcher that includes a siloed version of Python 2.7 (see https://developers.google.com/appengine/downloads). Our reason for providing this was to reduce the number of dependencies that Mac users needed to download before they have a usable SDK. This experimental launcher was downloaded 500 times but we haven't received any feedback about it. Could anyone who downloaded and try it please respond to this thread saying whether they had any problems or not? The more feedback we get, the better our release will be and your help is really appreciated! Cheers, Brian -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/bG6kFyDQ_cIJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Feedback on experimental Mac Launcher for Python
Hi, Along with the usual components of the App Engine 1.6.5 release, we also released an experimental version of the Mac OS X Launcher that includes a siloed version of Python 2.7 (see https://developers.google.com/appengine/downloads). Our reason for providing this was to reduce the number of dependencies that Mac users needed to download before they have a usable SDK. This experimental launcher was downloaded 500 times but we haven't received any feedback about it. Could anyone who downloaded and try it please respond to this thread saying whether they had any problems or not? The more feedback we get, the better our release will be and your help is really appreciated! Cheers, Brian -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: tree = lxml.html.parse(url) Server Error
On Mon, May 14, 2012 at 12:59 PM, Bassam Aoun bassam.a...@gmail.com wrote: thanks for the reply. the app.yaml already has the following: libraries: - name: jinja2 version: latest - name: lxml version: 2.3 The imports are working fine, my the line tree = lxml.html.parse(url) is causing a Server Error when the application is deployed on GAE although it works on the SDK. The webpage prints: Error: Server Error The server encountered an error and could not complete your request. any idea? Have you looked at the logs for your application? Cheers, Brian On Sunday, May 13, 2012 3:42:03 PM UTC-6, Dmitry Chusovitin wrote: lxml only availailable on Python 2.7. You must change your app.yml file. See https://developers.google.com/appengine/docs/python/python27/using27?hl=ru-RU#Configuring_Libraries add this code: libraries: - name: lxml version: 2.3 воскресенье, 13 мая 2012 г., 22:49:09 UTC+4 пользователь Bassam Aoun написал: Hello, My application work fine on the GAE and the SDK. However, when I include the line tree = lxml.html.parse(url) I get SERVER ERROR when I deploy my application to GAE. It works fine however in the SDK on my desktop. All those imports work fine on both the GAW and the SDK. import lxml from lxml import html import lxml.html but, tree = lxml.html.parse(url) gives me SERVER ERROR on GAE. I appreciate any help or suggestion. Thanks in advance. application address is http://hps109.appspot.com/?ticker=BIDUmanager=Blue -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/aYc1qZR0_IEJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Acquire Thread Lock takes 50% of response time
Hi Austin,
Could you include all of the profile results?
Cheers,
Brian
On Mon, Apr 23, 2012 at 3:45 AM, Austin austin.h@gmail.com wrote:
Hi,
I'm using python27 on GAE, and with threadsafe either false or true,
{method 'acquire' of 'thread.lock' objects} takes up to 50% of my
response time. Obviously 40s is too long for a web app so I'm trying
to cut it down, and this acquiring of thread locks is the biggest
culprit according to cProfile. (The 1.6M function calls also seems
high but I can't find where they're coming from either...)
Any thoughts you can provide on what I might be doing wrong is
appreciated. I also wonder if it is related to some of the Python 2.7
performance issues at the end of last year (that I thought were
resolved):
* https://code.google.com/p/googleappengine/issues/detail?id=6323
*
http://stackoverflow.com/questions/8341112/appengine-performance-degradation-with-python27
*
https://groups.google.com/forum/?fromgroups#!topic/google-appengine/jMah6aWfZWg
cProfile Output:
Profile data:
1662549 function calls (1652247 primitive calls) in 39.545
seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall
filename:lineno(function)
[lines removed]
10816 19.245 0.002 19.245 0.002 {method 'acquire' of
'thread.lock' objects}
[lines removed]
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: How to filter images on GAE?
Hi, On Fri, Apr 20, 2012 at 12:30 AM, Computer_Engineer groupd...@gmail.com wrote: Please i want to ask how can i test the application locally before uploading it to GAE,how can i solve the PIL problem on localhost?and if i want to filter images using PIL i will open it firstly,so my code initially let the user to upload his photo then i store this image in blobstorem and get the url of this image top open it,but this didn't solve the problem;that i can't open it? You might be better to ask search for the answer to these questions at http://stackoverflow.com/ and, if you can't find one, to ask a new one. The tag that you should use is google-app-engine. Cheers, Brian On Tuesday, April 17, 2012 5:11:44 PM UTC+3, Computer_Engineer wrote: I'm working on python google app engine application,i'm using python2.5, i want to deal with images on GAE(filter images), initially i tried wit PIL library and i installed it successfully and i tested it and it works correctly on my computer,but when i test it on localhost(run on GAE), i have an error thatNotImplementedError: Unable to find the Python PIL library,then i tried to deal with images using Images Python API , as in this tutorial:https://developers.google.com/appengine/docs/python/images/overview, and i tested resizing images service,and it works correctly,but the main application that i want is to filter the images,the question is :if this service(image filtering) supported in Images Python API in GAE?and if not how can i filter images on GAE?i saw that this service available in PIL,but this library is not supported by GAE I tried as a second choice to use python2.7 such that it supports PIL library and i edited the app.yaml configuration file,but i don't know how to use this library in the python page(main.py)? app.yaml: application: app_id version: 1 runtime: python27 api_version: 1 threadsafe: false handlers: - url: /.* script: main.py libraries: - name: PIL version: 1.1.7 ...any suggestions are welcome...Thanks -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/W3-vzVSeq94J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: How to filter images on GAE?
Hi, On Thu, Apr 19, 2012 at 2:53 AM, Computer_Engineer groupd...@gmail.com wrote: Robert Kluin: I firstly run my code on localhost to check errors.then i upload it to GAE,i use eclipse for working on python with GAE in windows, i installed python2.7 and PIL version 1.1.7,then i edited the app.yaml,but the problem still,please i want a solution for this problem as soon as possible. Thank you.. In this case, could you try what Robert suggested and just upload your application without running it locally to see if the problem is with your code or with your local development environment? If it works on the production system but not locally then you try starting a Python shell and typing import Image. If that doesn't work then your PIL installation is not correct. Cheers, Brian On Tuesday, April 17, 2012 5:11:44 PM UTC+3, Computer_Engineer wrote: I'm working on python google app engine application,i'm using python2.5, i want to deal with images on GAE(filter images), initially i tried wit PIL library and i installed it successfully and i tested it and it works correctly on my computer,but when i test it on localhost(run on GAE), i have an error thatNotImplementedError: Unable to find the Python PIL library,then i tried to deal with images using Images Python API , as in this tutorial:https://developers.google.com/appengine/docs/python/images/overview, and i tested resizing images service,and it works correctly,but the main application that i want is to filter the images,the question is :if this service(image filtering) supported in Images Python API in GAE?and if not how can i filter images on GAE?i saw that this service available in PIL,but this library is not supported by GAE I tried as a second choice to use python2.7 such that it supports PIL library and i edited the app.yaml configuration file,but i don't know how to use this library in the python page(main.py)? app.yaml: application: app_id version: 1 runtime: python27 api_version: 1 threadsafe: false handlers: - url: /.* script: main.py libraries: - name: PIL version: 1.1.7 ...any suggestions are welcome...Thanks -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/tmkdHRDGqAYJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: How to filter images on GAE?
Could you send the complete traceback? Cheers, Brian On Thu, Apr 19, 2012 at 5:56 AM, Computer_Engineer groupd...@gmail.com wrote: Brian Quinlan: i tried to upload it into my application on GAE, the error still???what can i do? On Tuesday, April 17, 2012 5:11:44 PM UTC+3, Computer_Engineer wrote: I'm working on python google app engine application,i'm using python2.5, i want to deal with images on GAE(filter images), initially i tried wit PIL library and i installed it successfully and i tested it and it works correctly on my computer,but when i test it on localhost(run on GAE), i have an error thatNotImplementedError: Unable to find the Python PIL library,then i tried to deal with images using Images Python API , as in this tutorial:https://developers.google.com/appengine/docs/python/images/overview, and i tested resizing images service,and it works correctly,but the main application that i want is to filter the images,the question is :if this service(image filtering) supported in Images Python API in GAE?and if not how can i filter images on GAE?i saw that this service available in PIL,but this library is not supported by GAE I tried as a second choice to use python2.7 such that it supports PIL library and i edited the app.yaml configuration file,but i don't know how to use this library in the python page(main.py)? app.yaml: application: app_id version: 1 runtime: python27 api_version: 1 threadsafe: false handlers: - url: /.* script: main.py libraries: - name: PIL version: 1.1.7 ...any suggestions are welcome...Thanks -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Gy6_kxeqsY0J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: How to filter images on GAE?
Hi, On Thu, Apr 19, 2012 at 7:12 AM, Computer_Engineer groupd...@gmail.com wrote: I provided my app.yaml in the question,and this is the trace: INFO 2012-04-18 21:08:25,615 appengine_rpc.py:159] Server: appengine.google.com INFO 2012-04-18 21:08:25,904 rdbms_sqlite.py:58] Connecting to SQLite database '' with file 'c:\\users\\me\\appdata\\local\\temp\\dev_appserver.rdbms' INFO 2012-04-18 21:08:26,200 dev_appserver_multiprocess.py:637] Running application compressanddecompress on port 8080: http://localhost:8080 ERROR 2012-04-18 21:09:16,936 dev_appserver.py:4096] Exception encountered handling request Traceback (most recent call last): File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver.py, line 4039, in _HandleRequest self._Dispatch(dispatcher, self.rfile, outfile, env_dict) File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver.py, line 3945, in _Dispatch base_env_dict=env_dict) File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver.py, line 605, in Dispatch base_env_dict=base_env_dict) File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver.py, line 3015, in Dispatch self._module_dict) File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver.py, line 2925, in ExecuteCGI reset_modules = exec_script(handler_path, cgi_path, hook) File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver.py, line 2785, in ExecuteOrImportScript exec module_code in script_module.__dict__ File C:\Users\me\workspace\img_filter\helloworld.py, line 1, in module from PIL import Image ImportError: No module named PIL Could you send the stack trace from production, not from your development server? Cheers, Brian INFO 2012-04-18 21:09:16,938 dev_appserver.py:4143] GET / HTTP/1.1 500 - ERROR 2012-04-18 21:09:17,328 dev_appserver.py:4096] Exception encountered handling request Traceback (most recent call last): File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver.py, line 4039, in _HandleRequest self._Dispatch(dispatcher, self.rfile, outfile, env_dict) File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver.py, line 3945, in _Dispatch base_env_dict=env_dict) File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver.py, line 605, in Dispatch base_env_dict=base_env_dict) File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver.py, line 3015, in Dispatch self._module_dict) File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver.py, line 2925, in ExecuteCGI reset_modules = exec_script(handler_path, cgi_path, hook) File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver.py, line 2785, in ExecuteOrImportScript exec module_code in script_module.__dict__ File C:\Users\me\workspace\img_filter\helloworld.py, line 1, in module from PIL import Image ImportError: No module named PIL INFO 2012-04-18 21:09:17,328 dev_appserver.py:4143] GET /favicon.ico HTTP/1.1 500 - ERROR 2012-04-18 21:09:37,575 dev_appserver.py:4096] Exception encountered handling request Traceback (most recent call last): File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver.py, line 4039, in _HandleRequest self._Dispatch(dispatcher, self.rfile, outfile, env_dict) File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver.py, line 3945, in _Dispatch base_env_dict=env_dict) File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver.py, line 605, in Dispatch base_env_dict=base_env_dict) File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver.py, line 3015, in Dispatch self._module_dict) File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver.py, line 2925, in ExecuteCGI reset_modules = exec_script(handler_path, cgi_path, hook) File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver.py, line 2785, in ExecuteOrImportScript exec module_code in script_module.__dict__ File C:\Users\me\workspace\img_filter\helloworld.py, line 1, in module from PIL import Image ImportError: No module named PIL INFO 2012-04-18 21:09:37,576 dev_appserver.py:4143] GET / HTTP/1.1 500 - ERROR 2012-04-18 21:09:37,904 dev_appserver.py:4096] Exception encountered handling request Traceback (most recent call last): File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver.py, line 4039, in _HandleRequest self._Dispatch(dispatcher, self.rfile, outfile, env_dict) File C:\Program
Re: [google-appengine] Re: How to filter images on GAE?
On Thu, Apr 19, 2012 at 7:27 AM, Computer_Engineer groupd...@gmail.com wrote: Sorry,but how can i get the log file from the product? Go to the administration console (http://appengine.google.com), select your application and then click Logs. Cheers, Brian On Tuesday, April 17, 2012 5:11:44 PM UTC+3, Computer_Engineer wrote: I'm working on python google app engine application,i'm using python2.5, i want to deal with images on GAE(filter images), initially i tried wit PIL library and i installed it successfully and i tested it and it works correctly on my computer,but when i test it on localhost(run on GAE), i have an error thatNotImplementedError: Unable to find the Python PIL library,then i tried to deal with images using Images Python API , as in this tutorial:https://developers.google.com/appengine/docs/python/images/overview, and i tested resizing images service,and it works correctly,but the main application that i want is to filter the images,the question is :if this service(image filtering) supported in Images Python API in GAE?and if not how can i filter images on GAE?i saw that this service available in PIL,but this library is not supported by GAE I tried as a second choice to use python2.7 such that it supports PIL library and i edited the app.yaml configuration file,but i don't know how to use this library in the python page(main.py)? app.yaml: application: app_id version: 1 runtime: python27 api_version: 1 threadsafe: false handlers: - url: /.* script: main.py libraries: - name: PIL version: 1.1.7 ...any suggestions are welcome...Thanks -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/CcIwSJCCDXEJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: How to filter images on GAE?
Hi Andreas, On Thu, Apr 19, 2012 at 7:41 AM, Andreas a.schmi...@gmail.com wrote: you are trying to open a file. you can't do that. you dont have any access to the filesystem on gae. That is not true - you have access to a read-only file system containing all of the files in the directory that you uploaded with appcfg.py update (and don't appear in a static handler in your app.yaml, doesn't match skip_files, etc.). The reason that this fails is because the file is being opened for *write*, which is not supported. Cheers, Brian On Apr 18, 2012, at 5:38 PM, Computer_Engineer wrote: This is the log from the admin console log: 2012-04-18 14:37:10.162 /favicon.ico 500 666ms 0kb Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.597.67 Safari/534.13 86.108.69.231 - - [18/Apr/2012:14:37:10 -0700] GET /favicon.ico HTTP/1.1 500 0 - Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.597.67 Safari/534.13 filterimages2012.appspot.com ms=667 cpu_ms=350 api_cpu_ms=0 cpm_usd=0.009764 instance=00c61b117c8fe6c4727b8c723b40c98b47e0df7f E2012-04-18 14:37:10.033 Traceback (most recent call last): File /base/data/home/apps/s~filterimages2012/1.358303610596814412/helloworld.py, line 7, in module out.save(2.jpg) File /base/python27_runtime/python27_lib/versions/third_party/PIL-1.1.7/PIL/Image.py, line 1433, in save fp = __builtin__.open(fp, wb) IOError: [Errno 13] Permission denied: '2.jpg' 2012-04-18 14:37:09.075 / 500 957ms 0kb Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.597.67 Safari/534.13 86.108.69.231 - - [18/Apr/2012:14:37:09 -0700] GET / HTTP/1.1 500 0 - Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.597.67 Safari/534.13 filterimages2012.appspot.com ms=958 cpu_ms=467 api_cpu_ms=0 cpm_usd=0.013012 loading_request=1 instance=00c61b117c8fe6c4727b8c723b40c98b47e0df7f E2012-04-18 14:37:09.071 Traceback (most recent call last): File /base/data/home/apps/s~filterimages2012/1.358303610596814412/helloworld.py, line 7, in module out.save(2.jpg) File /base/python27_runtime/python27_lib/versions/third_party/PIL-1.1.7/PIL/Image.py, line 1433, in save fp = __builtin__.open(fp, wb) IOError: [Errno 13] Permission denied: '2.jpg' I2012-04-18 14:37:09.074 This request caused a new process to be started for your application, and thus caused your application code to be loaded for the first time. This request may thus take longer and use more CPU than a typical request for your application. On Tuesday, April 17, 2012 5:11:44 PM UTC+3, Computer_Engineer wrote: I'm working on python google app engine application,i'm using python2.5, i want to deal with images on GAE(filter images), initially i tried wit PIL library and i installed it successfully and i tested it and it works correctly on my computer,but when i test it on localhost(run on GAE), i have an error thatNotImplementedError: Unable to find the Python PIL library,then i tried to deal with images using Images Python API , as in this tutorial:https://developers.google.com/appengine/docs/python/images/overview, and i tested resizing images service,and it works correctly,but the main application that i want is to filter the images,the question is :if this service(image filtering) supported in Images Python API in GAE?and if not how can i filter images on GAE?i saw that this service available in PIL,but this library is not supported by GAE I tried as a second choice to use python2.7 such that it supports PIL library and i edited the app.yaml configuration file,but i don't know how to use this library in the python page(main.py)? app.yaml: application: app_id version: 1 runtime: python27 api_version: 1 threadsafe: false handlers: - url: /.* script: main.py libraries: - name: PIL version: 1.1.7 ...any suggestions are welcome...Thanks -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Rk9EAnJyPIEJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post
Re: [google-appengine] How to filter images on GAE?
Hi, I'm going to address the Python 2.7 case since that is the best choice for this problem. On Wed, Apr 18, 2012 at 12:11 AM, Computer_Engineer groupd...@gmail.com wrote: I'm working on python google app engine application,i'm using python2.5, i want to deal with images on GAE(filter images), initially i tried wit PIL library and i installed it successfully and i tested it and it works correctly on my computer,but when i test it on localhost(run on GAE), i have an error thatNotImplementedError: Unable to find the Python PIL library,then i tried to deal with images using Images Python API , as in this tutorial:https://developers.google.com/appengine/docs/python/images/overview, and i tested resizing images service,and it works correctly,but the main application that i want is to filter the images,the question is :if this service(image filtering) supported in Images Python API in GAE?and if not how can i filter images on GAE?i saw that this service available in PIL,but this library is not supported by GAE I tried as a second choice to use python2.7 such that it supports PIL library and i edited the app.yaml configuration file,but i don't know how to use this library in the python page(main.py)? app.yaml: application: app_id version: 1 runtime: python27 api_version: 1 threadsafe: false handlers: - url: /.* script: main.py libraries: - name: PIL version: 1.1.7 ...any suggestions are welcome...Thanks You use PIL from main.py just like you would use PIL from any Python script e.g. import Image i = Image.open(...) In order to use Python 2.7 with the dev_appserver, you'll have to install Python 2.7 and PIL on your computer. Cheers, Brian -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/g48i1VaP0IUJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Datastore Indexes - Building
Hi SM Bhaskar, On Sun, Apr 15, 2012 at 6:05 AM, SM Bhaskar sm.bhas...@gmail.com wrote: Hi, My Datastore Indexes shows a status of Building for a few hours now...Can you error it out as mentioned in http://groups.google.com/group/google-appengine-python/browse_thread/thread/ee8d1eb862bb0e63?pli=1 Also, how do i run vacuum_indexes command .. My application id is: smbaskam Sorry about this - I'm not sure how your index got into this state. I restarted the index build and it is complete now. In the future, would you mind filing a bug there so this kind of issue doesn't slip through the cracks? http://code.google.com/p/googleappengine/issues Sorry again for the trouble! Cheers, Brian rgds, SM Bhaskar -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] How can I disable load balancing function in google app engine
Hi, On Sun, Apr 15, 2012 at 6:59 PM, 石奇偲 shiqi...@gmail.com wrote: I find that even when I request a same url(using HttpURLConnection class in java) in a short time by google app engine, my google app engine's ip is not the same. It doesn't matter in most times,but my app simulates a logging user to a third-party web.I use cookie to maintain the log-in status of the third web.But when my IP is changing ,the cookies are not work again . The app runs well when I debug from local,because my local ip don't change. So,is there any mathod that fixed the IP address google app engine used? No, using a fixed IP for urlfetch is not supported. But I'm not sure why a changing IP address would cause problems with cookies. Cheers, Brian -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/29GHxd9f_cUJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Happy birthday?
We did cake :-) https://plus.google.com/111039891542127786416/posts/hN4PMZ4QqmU Cheers, Brian On Sun, Apr 15, 2012 at 3:48 AM, alex a...@cloudware.it wrote: I meant to post this a few days back. Correct me if I'm wrong but April 8th or 11th is the day App Engine was announced as a limited preview. If it is correct, well here it is: happy birthday app engine! alex. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/DZmq8ZLjg1YJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] help migrating from python2.5 to python2.7
Hi, On Fri, Apr 6, 2012 at 11:28 PM, Kwame iweg...@gmail.com wrote: I'm trying to migrate my app to python2.7 and so far it's been a nightmare! My latest issue is that I'm getting this error: Traceback (most recent call last): File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2/webapp2.py, line 1536, in __call__ rv = self.handle_exception(request, response, e) File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2/webapp2.py, line 1530, in __call__ rv = self.router.dispatch(request, response) File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2/webapp2.py, line 1278, in default_dispatcher return route.handler_adapter(request, response) File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2/webapp2.py, line 1101, in __call__ handler = self.handler(request, response) TypeError: __init__() takes exactly 1 argument (3 given) I'm not sure where to start with debugging this. is this a bug? Please help. Is that the complete traceback? It doesn't seem to include the entry point into your code. Cheers, Brian -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/vARS7lQZe9oJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Mispelling in
On Fri, Apr 6, 2012 at 8:52 PM, abdelkrim boujraf abdelkrim.bouj...@gmail.com wrote: how can we give a feedback to the team dealing with the website developers.google.com? You can file a bug at: http://code.google.com/p/googleappengine/issues Ideally with a label of Component-Docs Cheers, Brian what if we found mistakes like this one: ... its number value is the same as for the the previously created... URL: https://developers.google.com/appengine/docs/python/tools/localunittesting Best regards, Abdelkrim http://blog.i14y.net -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/tNzncdJdSXIJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] location of Django admin media files
Hi Josvic, On Fri, Mar 23, 2012 at 2:21 AM, Josvic Zammit jvzam...@gmail.com wrote: I am running GoogleAppEngine (GAE) 1.6.3 with Python 2.7 and Django 1.3 by having: libraries: - name: django version: 1.3 in my app.yaml. The following should serve the admin media files at url /static/admin: - url: /static/admin static_dir: django/contrib/admin/media expiration: '0' But I get 404s for such admin media (css, etc). Am I using the correct location for the Django admin's media file? No, that isn't the correct path and we don't expose the correct path. Have you considered the approach Serving the admin files described at: https://docs.djangoproject.com/en/1.3/howto/deployment/modwsgi/ Cheers, Brian -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Syw-aH-RnskJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Py2.7
Hi, On Tue, Feb 28, 2012 at 6:06 AM, pdknsk pdk...@googlemail.com wrote: To be honest, it seems a bit early to declare it stable. It still has many minor bugs, and performance problems. Not to be flip but all software has bugs and the Python 2.7 runtime (and the Python 2.5 runtime and the Java runtime and the Go runtime...) are no exceptions. We aren't aware of any unfixed bugs that would prevent someone from using the runtime but, if you encounter a bug, please file it on the issue tracker and we will fix it as soon as possible. Now onto performance :-) There are definitely instances where the Python 2.5 runtime outperforms the Python 2.7 runtime. The reverse is also true. It is not feasible for us to outperform the Python 2.5 runtime in every case. To pick a trivial example, Python 2.7 supports user-created threads and concurrent requests so some APIs now use locks when mutating global state and copy some data structures when previously that wasn't necessary. There is obviously some cost associated with doing this but it is hard to avoid paying it. We think that the benefits outweigh the costs. Brandon mentioned a 50% reduction in billed instances when moving to the Python 2.7 runtime and enabling threadsafe. We have seen a similar reduction in other applications. But, as Cayden said in another message, applications will benefit from concurrent requests to varying degrees. This is not only an App Engine property - any Python application modified to use threads may become more or (usually slightly) less performant. Given the way that App Engine is normally used, we expect concurrent requests to provide a cost benefit for most (but not all) non-trivial-volume [1] applications. Some of our larger customers have already migrated to reduce their instance costs, to use features not available in Python 2.5, or to gain access to higher-performance modules like cPickle and json. But, if you think that the Python 2.7 runtime is too risky to use right now due to bugs or performance problems, you can always wait a few months and re-evaluate. Now I'm going to reactivate my reality-distortion-field and eat some more cake :-) Oh, and thanks for starting the discussion! Cheers, Brian [1] An application serving one request per minute doesn't have any concurrent requests so it can't benefit. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Py2.7
On Tue, Feb 28, 2012 at 3:02 PM, pdknsk pdk...@googlemail.com wrote: What! Google has App Engine plushies! But of course: http://www.googlestore.com/Fun/App+Engine+Squishable.axd Cheers, Brian -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Memcache.get error: TypeError: expected string or Unicode object, int found
Hi Rishi, On Thu, Feb 23, 2012 at 10:50 AM, Rishi Arora rishi.ar...@ship-rack.com wrote: I'm not sure what memcache.get() is expecting that should be a string or unicode but is an int instead: example: return memcache.get(key='PackageStatusMap', namespace=NS_Generic) The key is a string, and the namespace is an integer. Should the namespace be a string? Yes, the namespace should be a string or Unicode object. Cheers, Brian Interestingly, when I executed the same command from a remote API session, all went well, and no errors were reported. On Wed, Feb 22, 2012 at 5:44 PM, Rishi Arora rishi.ar...@ship-rack.com wrote: For the last 30 minutes or so, my application (app id shiprack-test1) is getting these errors sourced to a memcache.get() - see stack trace below. My app has M/S datastore and a python2.5 runtime. My app is essentially out-of-service now. Anybody seen this? File /base/python_runtime/python_lib/versions/1/google/appengine/api/memcache/__init__.py, line 562, in get rpc = self.get_multi_async([key], namespace=namespace, for_cas=for_cas) File /base/python_runtime/python_lib/versions/1/google/appengine/api/memcache/__init__.py, line 615, in get_multi_async self.__get_hook, user_key) File /base/python_runtime/python_lib/versions/1/google/appengine/api/memcache/__init__.py, line 385, in _make_async_call rpc.make_call(method, request, response, get_result_hook, user_data) File /base/python_runtime/python_lib/versions/1/google/appengine/api/apiproxy_stub_map.py, line 509, in make_call self.__rpc.MakeCall(self.__service, method, request, response) File /base/python_runtime/python_lib/versions/1/google/appengine/api/apiproxy_rpc.py, line 115, in MakeCall self._MakeCallImpl() File /base/python_runtime/python_lib/versions/1/google/appengine/runtime/apiproxy.py, line 161, in _MakeCallImpl request_data = self.request.SerializeToString() File /base/python_runtime/python_lib/versions/1/google/net/proto/ProtocolBuffer.py, line 60, in SerializeToString return self.Encode() File /base/python_runtime/python_lib/versions/1/google/net/proto/ProtocolBuffer.py, line 53, in Encode return self._CEncode() File /base/python_runtime/python_lib/versions/1/google/appengine/api/memcache/memcache_service_pb.py, line 430, in _CEncode return _net_proto___parse__python.Encode(self, 'apphosting.MemcacheGetRequest') TypeError: expected string or Unicode object, int found -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Python 2.7: Instance memory limitations with concurrent requests
Hi Mike, On Wed, Feb 22, 2012 at 7:22 AM, Mike Wesner mike.wes...@webfilings.com wrote: I would love to learn more from google on python memory handling. As Robert mentioned, we have observed that memory is not released/garbage collected on python 2.5 instances. It seems to just hold on to it. This works because the instances don't live forever and eventually get shutdown and new ones take their place. We haven't changed Python's built-in memory management in either of our Python runtimes. Note that, even excluding cyclic garbage collection, Python and the C allocator do not always release memory eagerly - they maintain their own memory pools for performance reasons. So, for example, you can't assume that an instance consuming 100MB of memory only has 28MB remaining to handle requests. Cheers, Brian The memory management used in the App Engine Python 2.5 and 2.7 runti -Mike On Feb 21, 10:33 am, alex a...@cloudware.it wrote: One of the biggest issues is that the Python runtime leaks memory like mad could you elaborate on this? or give some references that support your claims (provided an app code is written according to best practices, etc) On Tuesday, February 21, 2012 5:10:16 PM UTC+1, Robert Kluin wrote: To be clear, the scheduler can still dispatch multiple requests for that url to the same instance. Only one request will execute at a time though. One of the biggest issues is that the Python runtime leaks memory like mad, so in this case you may wind up with 1) increased latency and 2) still blowing mem limits fast. Robert On Tue, Feb 21, 2012 at 09:36, Jeff Schnitzer wrote: On Mon, Feb 20, 2012 at 5:33 AM, Johan Euphrosine wrote: On Mon, Feb 20, 2012 at 11:25 AM, Andrin von Rechenberg wrote: I guess that's the same solution as just deploying two different versions. A threadsafe one and a non threadsafe one. Or did I misunderstand you? appcfg.py provide commands to help you manage your backends deployment and configuration, with `backends.yaml` and `appcfg backends update`. That make the solution more convenient than using multiple versions on frontend instances. This adds the significant downside of no auto-scaling. I don't see how that's more convenient :-) Andrin: In javaland I would simply synchronize the people-search function so that at most one thread can execute that routine in a single instance at once. It means all people-search requests in that instance will queue up in serial, which could cause undesirable waits if the function takes significant time, but if the requests are spread out among enough instances it probably won't be an issue. I don't know what the python equivalent of 'synchronized' is. Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] [JDO] Getting an Object by Key will count as Small Datastore Operations
On Fri, Feb 10, 2012 at 7:53 PM, poke buidinhngoc.a...@gmail.com wrote: Few days ago , i change my code to access by key (not by query) . But in Billing report Small Datastore Operations still equal zero and Datastore Reads same as using query - billing not go to down. JDO pm.getObjectById() method still count as Datastore Reads Gets still cost a read operation. See: http://code.google.com/appengine/docs/billing.html Entity Get (per entity): 1 Read Query: 1 Read + 1 Read per entity returned Cheers, Brian -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: [JDO] Getting an Object by Key will count as Small Datastore Operations
On Fri, Feb 10, 2012 at 9:03 PM, poke buidinhngoc.a...@gmail.com wrote: thanks . finally pm.getObjectById() is Entity Get ? Yes, you are loading the contents of a datastore entity. Cheers, Brian On Feb 10, 4:05 pm, Brian Quinlan bquin...@google.com wrote: On Fri, Feb 10, 2012 at 7:53 PM, poke buidinhngoc.a...@gmail.com wrote: Few days ago , i change my code to access by key (not by query) . But in Billing report Small Datastore Operations still equal zero and Datastore Reads same as using query - billing not go to down. JDO pm.getObjectById() method still count as Datastore Reads Gets still cost a read operation. See:http://code.google.com/appengine/docs/billing.html Entity Get (per entity): 1 Read Query: 1 Read + 1 Read per entity returned Cheers, Brian -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] APP ABUSE
Hi Angelo, On Fri, Feb 3, 2012 at 9:19 PM, Angelo idol...@gmail.com wrote: The owner of this url http://www.asburyisgay.appspot.com hacked my site http://www.trafalgarsqaure.info/, copied it all and redirected it to the url: http://www.asburyisgay.appspot.com/TrafalgarSquare.info. I don't think that any hacking or copying was involved. It looks like this is a simple proxy application. Note, for example: http://www.asburyisgay.appspot.com/google.com How to contact google and to tell them to remove this url? The contact form for APP Abuse doesn't work for me, because I don't have app ID and I need it to submit a report. The app id is asburyisgay but this does not look like abuse to me. Cheers, Brian -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: App Engine SDK 1.6.2 is out!
Hi Ron, On Thu, Feb 2, 2012 at 8:04 AM, Ron rc...@bonline.com wrote: why is Django 1.3 not available with Python 2.7 ? Before Python 2.7's official release we want to maximize the amount of testing that its existing libraries get instead of adding new ones. and will Django non-rel be available in a future release? We generally don't commit to packaging new libraries but you can use Django nonrel right now by downloading it yourself and putting it in your application's directory. Cheers, Brian On Jan 31, 11:17 pm, Ikai Lan (Google) ika...@google.com wrote: Hey everyone, Just wanted to announce that App Engine 1.6.2 is out! You can check out the blog post here: http://googleappengine.blogspot.com/2012/01/app-engine-162-released.html Note that if you're getting an import error for task queues, you're probably still using the old labs import. You'll want to get this cleared up (you'll also want to pay attention to deprecation warnings). Release notes are below for your reading pleasure: *Java 1.6.2* -- - The Admin Console Datastore Admin has added experimental backup and restore functionality. The job occurs within your application and counts against your application quota, including Instance Hours, Datastore Ops and Datastore Storage. - Developers can now specify how long a channel token will last until it expires, with the default remaining two hours. Channel API quota is now measured both in calls to create a channel and the number of hours of channel time requested. The maximum hours of quota is the maximum number of channel creation calls * 2, so free apps get 200 hours of requested channel token time. - Task Queue API requests now include a X-Appengine-TaskETA header, that can be used to measure task delivery latency. - We have removed the deprecated labs version of the TaskQueue API. - The default API deadlines for Blobstore API calls have been raised to 15s for online and 30s for offline requests, up from 5s. - The Images API now allows you to stretch an image without maintaining the aspect ratio. -http://code.google.com/p/googleappengine/issues/detail?id=2220 - Mail Quota for App Engine apps that have signed up for billing will only be increased after the first payment for the app is processed. - Fixed an issue where the SDK did not resize images down to 512 pixels by default, as it does in production. - Fixed an issue with the Images API where valid images were returning a NotImageError. -http://code.google.com/p/googleappengine/issues/detail?id=5545 *Python 1.6.2* - The Admin Console Datastore Admin has added experimental backup and restore functionality. The job occurs within your application and counts against your application quota, including Instance Hours, Datastore Ops and Datastore Storage. - Developers can now specify how long a channel token will last until it expires, with the default remaining two hours. Channel API quota is now measured both in calls to create a channel and the number of hours of channel time requested. The maximum hours of quota is the maximum number of channel creation calls * 2, so free apps get 200 hours of requested channel token time. - Python Django now work with Cloud SQL without additional configuration. For further information see https://developers.google.com/cloud-sql/. - Task Queue API requests now include a X-Appengine-TaskETA header, that can be used to measure task delivery latency. - The default API deadlines for Blobstore API calls have been raised to 15s for online and 30s for offline requests, up from 5s. - The Images API now allows you to stretch an image without maintaining the aspect ratio. -http://code.google.com/p/googleappengine/issues/detail?id=2220 - The Blobstore API now includes the asynchronous function calls create_upload_url_async, delete_async, and fetch_data_async. - Django version 1.3 is now available in the Python 2.5 runtime. - We've added a django_wsgi builtin to allow easier bootstrapping of Django applications. - Mail Quota for App Engine apps that have signed up for billing will only be increased after the first payment for the app is processed. - As announced in 1.6.1, in this release for the experimental Python 2.7 runtime, the mapreduce and datastore_admin builtins are not available to apps deployed to Python 2.7. - Fixed an issue with remote_api where calling fetch_page() with a page size of 301 and chaining the calls through the returned cursor was skipping half of the results. - Fixed an issue where the PIL _imagingmath module was not available in Python 2.7. - Fixed an issue where the SDK did not resize images down to 512 pixels by default, as it does in production. - Fixed an issue with the
Re: [google-appengine] webapp2_extras sessions_ndb problem (appengine 1.6.1)
On Sat, Jan 21, 2012 at 3:37 AM, Mariatta mwij...@vendasta.com wrote: I'm using webapp2 framework that is shipped together with Google AppEngine 1.6.1 In webapp2_extras/appengine/sessions_ndb.py line 20 it reads: from ndb import model I wonder if this should have been: from google.appengine.ext.ndb import model Because as I am trying to use the datastore as the sessions_backend in webapp2, it complains about ' no module named ndb found' webapp2 has not been updated since ndb was packaged as part of the App Engine runtime and SDK. For now you can download ndb yourself and put it in the root directory of your application. The next release of webapp2 should behave as you expect. Cheers, Brian Here's the full stack trace. I bold the part where I think the problem is 500 Internal Server Error import_string() failed for 'webapp2_extras.appengine.sessions_ndb.DatastoreSessionFactory'. Possible reasons are: - missing __init__.py in a package; - package or module path not included in sys.path; - duplicated package or module name taking precedence in sys.path; - missing module, class, function or variable; Original exception: ImportError: No module named ndb Debugged import: - 'webapp2_extras' found in '/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2/webapp2_extras/__init__.pyc'. - 'webapp2_extras.appengine' found in '/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2/webapp2_extras/appengine/__init__.py'. - 'webapp2_extras.appengine.sessions_ndb' not found. Is it possible to have this updated properly? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/-XbP9mGoycQJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Random DeadlineExceed Errors with python27 and HRD
Hi Sergio, On Thu, Jan 19, 2012 at 4:46 AM, sergio.jar...@gmail.com sergio.jar...@gmail.com wrote: Hi, I'm seeing random DeadlineExceeded errors with python27 and HRD since about a week ago, I do have threadsafe False so it's not the threadsafe latency issue. I can provide the app id off-list. Without knowing what your application is doing, it is hard to say with the problem is. Are these exceptions triggered by any particular query pattern? How many datastore calls are you making? What limit are you using? In a successful request, what does AppStats say about the API-call timing for the request? Cheers, Brian Here is a (partial) trace File /base/python27_runtime/python27_lib/versions/1/google/appengine/ ext/db/__init__.py, line 2084, in fetch return list(self.run(limit=limit, offset=offset, **kwargs)) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/ext/db/__init__.py, line 2237, in next return self.__model_class.from_entity(self.__iterator.next()) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/datastore/datastore_query.py, line 2655, in next next_batch = self.__batcher.next() File /base/python27_runtime/python27_lib/versions/1/google/ appengine/datastore/datastore_query.py, line 2525, in next return self.next_batch(self.AT_LEAST_ONE) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/datastore/datastore_query.py, line 2562, in next_batch batch = self.__next_batch.get_result() File /base/python27_runtime/python27_lib/versions/1/google/ appengine/api/apiproxy_stub_map.py, line 592, in get_result return self.__get_result_hook(self) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/datastore/datastore_query.py, line 2317, in __query_result_hook self._conn.check_rpc_success(rpc) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/datastore/datastore_rpc.py, line 1176, in check_rpc_success rpc.wait() File /base/python27_runtime/python27_lib/versions/1/google/ appengine/api/apiproxy_stub_map.py, line 535, in wait assert self.__rpc.state == apiproxy_rpc.RPC.FINISHING, repr(self.state) DeadlineExceededError -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Do you Serialize your Data?
Hi Brandon, On Thu, Jan 19, 2012 at 11:25 AM, Brandon Wirtz drak...@digerat.com wrote: Round one of my testing says that Serializing via JSON, or Pickle is SLOWER than just doing the reads from the datastore. This might be different in the Python 2.7 runtime, where cPickle and C-accelerated JSON are available. Cheers, Brian I'm reading up on model_to_protobuf Which is supposed to be faster. I'm a little sad. It would be awesome if there was a .put(key,serialize(stuff_i_would_write) -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of bFlood Sent: Tuesday, January 17, 2012 6:52 AM To: Google App Engine Subject: [google-appengine] Re: Do you Serialize your Data? hi brandon compression - not really, at least not with HRD. I remember it being useful in M/S but only for larger strings. I'm using this python property now, it only compresses over a certain size http://atastypixel.com/blog/wp-content/uploads/2011/01/compressible_text_pro perty.py_.txt great article jeff, I know I've read of few of your older posts about this in the past cheers brian On Jan 17, 9:21 am, Brandon Wirtz drak...@digerat.com wrote: You wrote How to Use GAE to Get Laid and didn't tell anyone? Evil. -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Jeff Schnitzer Sent: Tuesday, January 17, 2012 6:10 AM To: google-appengine@googlegroups.com Subject: Re: [google-appengine] Re: Do you Serialize your Data? In fact, I wrote up an explanation of this 6 months ago: http://blog.similarity.com/post/7541938593/how-to-build-an-online-dat... e-nosql-edition Jeff On Tue, Jan 17, 2012 at 2:06 PM, Jeff Schnitzer j...@infohazard.org wrote: I tend to do this a lot to minimize the cost of fetching data; if I need all the information about a Person then I get all that info by loading a single entity, usually pulled from memcache. Most of my analytic processes look like map/reduce so indexing these complex structures wouldn't really help. If you're in Javaland you can use Objectify4 and annotate a field with @Serialize(zip=true) to get it automatically compressed. I haven't compared resulting entity sizes, however. Jeff On Tue, Jan 17, 2012 at 1:27 PM, Brandon Wirtz drak...@digerat.com wrote: Have you weighed Compression vs uncompressed in terms of speed, and cost? I'll end up testing to see, but I like to know the answers beforehand :-) -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of bFlood Sent: Tuesday, January 17, 2012 4:54 AM To: Google App Engine Subject: [google-appengine] Re: Do you Serialize your Data? hi brandon I do this with a bunch of models, keep a few properties for search and then dump the bulk of data into a compressed, json text property. Obviously the json part might be overkill but its a lot more flexible going forward if (err, when) you need to add more data (especially nested data) reads and writes are faster, YMMV YMMV b On Jan 17, 4:44 am, Brandon Wirtz drak...@digerat.com wrote: We are looking at restructuring some of our calls, optimizing again for speed, we are thinking that rather than having a datastore call for a single cell, we'd serialize things. This makes searching suck, and we have had a few issues where serialized and unserialized data didn't quite match when we were done (because of weird encoded characters from sites) Serializing also locks you in to a single data structure for the most part. We are also just considering: data as a zipped, delimited array uncompressed, delimited Double storing, so that we pay twice for writes, and can run analytics, but so that reads are against the serialized data, and faster and cost less. What are anyone's thoughts? Have you had these discussions with your dev team? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe
Re: [google-appengine] Re: function save from numpy works, but not savez
Hi, savez(), unlike save(), writes its intermediate results to a temporary file. Since App Engine doesn't support temporary files, this doesn't work. Working around this should be fairly easy but you may have to careful that your intermediate data isn't bigger than the available instance memory. Cheers, Brian On Mon, Jan 9, 2012 at 7:36 AM, redjava redj...@gmail.com wrote: savez(StringIO,numpy array) it works for save(StringIO,numpy array), but will raise that error if change save() to savez() Thanks, redjava On Jan 7, 9:37 pm, Brian Quinlan bquin...@google.com wrote: On Sun, Jan 8, 2012 at 4:19 PM, redjava redj...@gmail.com wrote: The rest is just a function call to savez With what arguments? Cheers, Brian On Jan 7, 4:41 pm, Brian Quinlan bquin...@google.com wrote: Are you sure that is the full traceback? On Sun, Jan 8, 2012 at 10:52 AM, redjava redj...@gmail.com wrote: Hi, The traceback for using savez is: File C:\Python27\lib\site-packages\numpy\lib\npyio.py, line 491, in savez File C:\Python27\lib\site-packages\numpy\lib\npyio.py, line 543, in _savez File C:\Program Files (x86)\Google\google_appengine\google\appengine \dist\tempfile.py, line 61, in PlaceHolder raise NotImplementedError(Only tempfile.TemporaryFile is available for use) NotImplementedError: Only tempfile.TemporaryFile is available for use Thanks! redjava On Jan 6, 5:51 pm, Brian Quinlan bquin...@google.com wrote: Hi, On Sat, Jan 7, 2012 at 12:43 PM, redjava redj...@gmail.com wrote: To save large numpy array, one way is to create a StringIO, call save() to compress the array and save it as Blob This method works fine with save(), but not with savez(). Instead it will raise NotImplementedError: Only tempfile.TemporaryFile is available for use Could you provide the entire traceback? Cheers, Brian -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com . To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com . To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com . To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: function save from numpy works, but not savez
On Mon, Jan 9, 2012 at 2:04 PM, Brandon Wirtz drak...@digerat.com wrote: I don’t use numpy, so I have no clue, but doesn’t Savez just compress save? No, it uses a fairly different code path. Can the same thing be achieved using Zip compress on the Save? I'm not 100% sure - save and savez end up in different C code and I didn't spend enough time examining the code to see how they are different. Also could the GAE version of Numpy not be modified easily to not use a temp file and just doing the compression in memory? Even if modifying it were easy (and I'm not sure about that), we'd prefer to not have a set of patches lying around that we have to apply to various third-party libraries every time that we upgrade them. Cheers, Brian From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Brian Quinlan Sent: Sunday, January 08, 2012 6:43 PM To: google-appengine@googlegroups.com Subject: Re: [google-appengine] Re: function save from numpy works, but not savez Hi, savez(), unlike save(), writes its intermediate results to a temporary file. Since App Engine doesn't support temporary files, this doesn't work. Working around this should be fairly easy but you may have to careful that your intermediate data isn't bigger than the available instance memory. Cheers, Brian On Mon, Jan 9, 2012 at 7:36 AM, redjava redj...@gmail.com wrote: savez(StringIO,numpy array) it works for save(StringIO,numpy array), but will raise that error if change save() to savez() Thanks, redjava On Jan 7, 9:37 pm, Brian Quinlan bquin...@google.com wrote: On Sun, Jan 8, 2012 at 4:19 PM, redjava redj...@gmail.com wrote: The rest is just a function call to savez With what arguments? Cheers, Brian On Jan 7, 4:41 pm, Brian Quinlan bquin...@google.com wrote: Are you sure that is the full traceback? On Sun, Jan 8, 2012 at 10:52 AM, redjava redj...@gmail.com wrote: Hi, The traceback for using savez is: File C:\Python27\lib\site-packages\numpy\lib\npyio.py, line 491, in savez File C:\Python27\lib\site-packages\numpy\lib\npyio.py, line 543, in _savez File C:\Program Files (x86)\Google\google_appengine\google\appengine \dist\tempfile.py, line 61, in PlaceHolder raise NotImplementedError(Only tempfile.TemporaryFile is available for use) NotImplementedError: Only tempfile.TemporaryFile is available for use Thanks! redjava On Jan 6, 5:51 pm, Brian Quinlan bquin...@google.com wrote: Hi, On Sat, Jan 7, 2012 at 12:43 PM, redjava redj...@gmail.com wrote: To save large numpy array, one way is to create a StringIO, call save() to compress the array and save it as Blob This method works fine with save(), but not with savez(). Instead it will raise NotImplementedError: Only tempfile.TemporaryFile is available for use Could you provide the entire traceback? Cheers, Brian -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com . To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com . To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received
Re: [google-appengine] Re: function save from numpy works, but not savez
Are you sure that is the full traceback? On Sun, Jan 8, 2012 at 10:52 AM, redjava redj...@gmail.com wrote: Hi, The traceback for using savez is: File C:\Python27\lib\site-packages\numpy\lib\npyio.py, line 491, in savez File C:\Python27\lib\site-packages\numpy\lib\npyio.py, line 543, in _savez File C:\Program Files (x86)\Google\google_appengine\google\appengine \dist\tempfile.py, line 61, in PlaceHolder raise NotImplementedError(Only tempfile.TemporaryFile is available for use) NotImplementedError: Only tempfile.TemporaryFile is available for use Thanks! redjava On Jan 6, 5:51 pm, Brian Quinlan bquin...@google.com wrote: Hi, On Sat, Jan 7, 2012 at 12:43 PM, redjava redj...@gmail.com wrote: To save large numpy array, one way is to create a StringIO, call save() to compress the array and save it as Blob This method works fine with save(), but not with savez(). Instead it will raise NotImplementedError: Only tempfile.TemporaryFile is available for use Could you provide the entire traceback? Cheers, Brian -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com . To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Datastore Small Operations
Hi James, In the future could you address your questions to the group rather than me personally? There may be other people interested in this information. Some examples of various operation sizes are here: http://code.google.com/appengine/docs/billing.html#Billable_Resource_Unit_Cost On Sun, Jan 8, 2012 at 12:16 PM, James Gilliam jimgill...@gmail.com wrote: Is get_by_key_name( aKey ) a small datastore operation ? No. If not, what is? Allocating a key or returning a key as a query result. Cheers, Brian On Nov 7 2011, 2:27 pm, Brian Quinlan bquin...@google.com wrote: Hi Sargis, Please see:http://code.google.com/appengine/docs/quotas.html#Resources Key allocations and keys-only queries consumesmalloperations. Let me know if I can do anything else to help. Cheers, Brian On Tue, Nov 8, 2011 at 9:18 AM, Sargis Dallakyan dallaky...@gmail.com wrote: Greetings, Does anyone know whatDatastoreSmallOperations means and where I can find more info about it. I'm currently getting over the quota on this after Nov 7 update: 100% - 0.05 of 0.05 Million Ops. I've searched the web and documentations, but can't find anything about this in the release notes or in the documentations. Thanks, Sargis -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com . To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group athttp:// groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: function save from numpy works, but not savez
On Sun, Jan 8, 2012 at 4:19 PM, redjava redj...@gmail.com wrote: The rest is just a function call to savez With what arguments? Cheers, Brian On Jan 7, 4:41 pm, Brian Quinlan bquin...@google.com wrote: Are you sure that is the full traceback? On Sun, Jan 8, 2012 at 10:52 AM, redjava redj...@gmail.com wrote: Hi, The traceback for using savez is: File C:\Python27\lib\site-packages\numpy\lib\npyio.py, line 491, in savez File C:\Python27\lib\site-packages\numpy\lib\npyio.py, line 543, in _savez File C:\Program Files (x86)\Google\google_appengine\google\appengine \dist\tempfile.py, line 61, in PlaceHolder raise NotImplementedError(Only tempfile.TemporaryFile is available for use) NotImplementedError: Only tempfile.TemporaryFile is available for use Thanks! redjava On Jan 6, 5:51 pm, Brian Quinlan bquin...@google.com wrote: Hi, On Sat, Jan 7, 2012 at 12:43 PM, redjava redj...@gmail.com wrote: To save large numpy array, one way is to create a StringIO, call save() to compress the array and save it as Blob This method works fine with save(), but not with savez(). Instead it will raise NotImplementedError: Only tempfile.TemporaryFile is available for use Could you provide the entire traceback? Cheers, Brian -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com . To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com . To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] [Python] Threadsafe or not?
On Sat, Jan 7, 2012 at 7:19 AM, Kaan Soral kaanso...@gmail.com wrote: Like I mentioned in my post, I've enabled threadsafe, so you can deduct that I am on 2.7 Actually, you can set threadsafe: yes in Python 2.5 as well so that would not be a safe assumption. There are know issues with using concurrent requests with Python 2.7: http://code.google.com/p/googleappengine/issues/detail?id=6323 Cheers, Brian I have also tried F4 instances today, but although I had 12 instances when I checked, QPS values was around 3-5 and memory was around 90's on all instances (max 256 but all instances were using 80-90) So I don't think concurrency is used much -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/2mXQgZP8MBkJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] function save from numpy works, but not savez
Hi, On Sat, Jan 7, 2012 at 12:43 PM, redjava redj...@gmail.com wrote: To save large numpy array, one way is to create a StringIO, call save() to compress the array and save it as Blob This method works fine with save(), but not with savez(). Instead it will raise NotImplementedError: Only tempfile.TemporaryFile is available for use Could you provide the entire traceback? Cheers, Brian -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] [Python] Threadsafe or not?
The Python 2.5 runtimes does not support concurrent requests. The Python 2.7 runtime (currently experimental) does. Cheers, Brian On Fri, Jan 6, 2012 at 1:06 PM, Kaan Soral kaanso...@gmail.com wrote: I think my requests are lightweight. I have enabled threadsafe a long time ago but my QPS values for Instances are 0.1 0.3 1.4 etc Very low I also have a lot of instances like these, 10s of them ( min_idle = 1 ) Is there anyone who can confirm concurrency in Python? (I think my regular requests should use 0-3mb of ram, but I have no way of testing it, how can I test it? ) -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/fWBjhEcToosJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Datastore Admin broken on python27
On Sat, Dec 31, 2011 at 11:38 AM, Maximillian Dornseif m.dorns...@hudora.de wrote: On Dec 31, 1:22 am, Maximillian Dornseif m.dorns...@hudora.de wrote: On Dec 27, 7:43 pm, Anand Mistry amis...@google.com wrote: The recommended way to use datastore admin with Python 2.7 is by enabling it through the administration console (http://code.google.com/appengine/docs/adminconsole/datastoreadmin.html). Thanks for the hint! Hmm. Now I only get: The requested URL /_ah/login_required?continue=https://ah-builtin- python-bundle-dot-latest-dot-hdwawi.appspot.com/_ah/datastore_admin/? app_id=s~hdwawi was not found on this server. seems to be broken for apps using federated login :-( Yes, this is a bug: http://code.google.com/p/googleappengine/issues/detail?id=4111 Cheers, Brian --md -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] PIL Import Error
Hi Richard, This looks like a bug. Sorry that it bit you but thanks for helping us by testing Python 2.7! Cheers, Brian On Mon, Jan 2, 2012 at 4:49 PM, Richard Arrano rickarr...@gmail.com wrote: Hello, I've been attempting to work with the Python Imaging Library in my app, and for the most part I can do so. In my app.yaml I have: libraries: - name: PIL version: latest However, I wanted to use ImageMath.eval and I found this error: line 11, in module import ImageMath File /base/python27_runtime/python27_lib/versions/third_party/ PIL-1.1.7/PIL/ImageMath.py, line 19, in module import _imagingmath It seems like I should be able to use ImageMath since it is in the directory on App Engine's side, but there's a problem with finding the SO file. Is this something on purpose that's disabled for us? Thanks, Rick -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: DeadlineExceededError on HRD and sky-high latency (python27+threadsafe)
Hi Felippe, We have been treating http://code.google.com/p/googleappengine/issues/detail?id=6323 as a high-priority issue and I've marked it as such. That being said, the Python 2.7 runtime is experimental and we can't offer any support for it or provide any guarantees on when this issue will be fixed. Cheers, Brian On Wed, Dec 28, 2011 at 7:06 PM, Felippe Bueno felippe.bu...@gmail.com wrote: Hello All, Someone has any work around for this, or aren't you using python27+threadsafe ? Thank you very much. Hello Googlers, Any progress or prevision on this ? It is very frustrating having python 2.7 without thread safe. I'm also trying to contact premier account's email but without success. I'd like to get real support. About 6323, is it really medium priority ? I think this is a high priority issue. Any way. Thanks On Wed, Dec 7, 2011 at 7:11 PM, sergio.jar...@gmail.com sergio.jar...@gmail.com wrote: Hi, Brian, Gregory Thanks for your valuable insights! I did know about the issue 6323 and had stared it. However I was not sure it was the same as the issue mentioned CPU bound requests. My app is very much RPC bound. I've added a comment to this thread to the issue and I've set threadsafe: no and will monitor. So far latency seems Ok and the number of instances is 5x-6x of when I had threadsafe: yes. I do agree with Felippe, we should be able to maintain the 50% discount until python27 and threadsafe are fully supported. This is also making things worse - it seems the minimum latency is not being enforced correctly, I also have this. http://code.google.com/p/googleappengine/issues/detail?id=5765can=5sort=-statuscolspec=ID%20Type%20Component%20Status%20Stars%20Summary%20Language%20Priority%20Owner%20Log Sérgio On Dec 7, 5:12 am, Gregory D'alesandre gr...@google.com wrote: Hi Felippe, As you noted Python 2.7 is released. While Python 2.7 is still experimental it is working well for some but indeed there are still issues before it will be fully GA. There was a concern that we were abandoning Python because there was no solution for concurrent requests. Python 2.7, even in its current state, shows that we are not abandoning Python. We tried our best to make appropriate trade-offs with the new pricing model but at some point we needed to roll it out for the reasons we've discussed at length in the past. For instance, when this was written we were also going to launch the new pricing 1.5 months earlier than we actually released it, which likely saved you quite a bit of money, as opposed to this which is costing you more. While my guess is you'd still rather that the 50% discount persisted I hope that answers your question, Greg On Tue, Dec 6, 2011 at 5:29 PM, Felippe Bueno felippe.bu...@gmail.comwrote: On Tue, Dec 6, 2011 at 8:56 PM, Brian Quinlan bquin...@google.com wrote: There is a known bug where the combination of Python 2.7 and enabling concurrent requests causes large latency increases. The Python 2.7 runtime is still experimental and bugs like this are expected. Ok, I understand that. Greg, could you please let us know why we have the new full-sized instance hour for python and still not have python concurrent requests ? I'm asking because you wrotehttp://groups.google.com/group/google-appengine/browse_thread/thread/... : Q: Will there be a solution for Python concurrency? Will this require any code changes? Python concurrency will be handled by our release of Python 2.7 on App Engine. We’ve heard a lot of feedback from our Python users who are worried that the incentive is to move to Java because of its support for concurrent requests, so we’ve made a change to the new pricing to account for that. *While Python 2.7 support is currently in progress it is not yet done *so we will be ** providing a half-sized instance for Python (at half the price) until Python 2.7 is released.* * Ok, I know. It was already released (as experimental), but I thought that the purpose to holding full-sized instances was to give to python users the concurrent requests. Thanks -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com
Re: [google-appengine] Least Expensive Way to Delete Entities
I think that Ikai made an off-by-one-error: it requires 22 writes assuming that every property is indexed :-) Check out Entity Delete (per entity) at: http://code.google.com/appengine/docs/billing.html#Billable_Resource_Unit_Cost Cheers, Brian On Tue, Dec 20, 2011 at 11:47 PM, supercobra superco...@gmail.com wrote: Could you explain (again?) how a deletion of an entity w/ 10- property is 21 writes? On Mon, Dec 19, 2011 at 3:18 PM, Ikai Lan (Google) ika...@google.com wrote: One thing you can do is making this really slow. The free quota actually gives you quite a bit of quota (if your app is relatively small) so if you can stretch out your job you can purge a relatively small datastore at low cost. I suspect what makes this expensive is datastore ops, not instance hours. Deleting an entity with 10 properties is 21 writes! There's not a lot we can do there ... -- Ikai Lan Developer Programs Engineer, Google App Engine plus.ikailan.com | twitter.com/ikai On Mon, Dec 19, 2011 at 9:50 AM, Patrick Twohig patr...@namazustudios.com wrote: Hi, I have a bit of old data that I want to clean out. I tried to write a fanout operation to clean out some of the old data, but it's actually manged to run out my budget in a few hours. I'm curious if there would be a way to delete huge amounts of old data that isn't needed anymore. Is there a tool or something that i can use to efficiently purge entity groups from my application? Any help would be greatly appreciated. Thanks, Patrick. -- Patrick H. Twohig. Namazu Studios P.O. Box 34161 San Diego, CA 92163-4161 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Isn't .08/hr 1.92/day $59.52/month for a 600 MHZ CPU instance with 128 MB memory a LITTLE EXPENSIVE
HI John, On Thu, Dec 15, 2011 at 7:58 AM, John sc...@peoplepedia.org wrote: When it comes to writes I call it the times two phenomenon. I have NO IDEA why you can't do anything that is not 2 writes??? I'm not sure if this is what you are looking for but the translation of high-level datastore operations into billable operations is described here: http://code.google.com/appengine/docs/billing.html#Billable_Resource_Unit_Cost Cheers, Brian e.g. If you have a very simple Entity with 5 properties (none set to the NON default status of Unindexed) and you save it, it is 12 writes. Key http://localhost:8080/_ah/admin/datastore?kind=Testorder=__key__Write Ops ID/Namehttp://localhost:8080/_ah/admin/datastore?kind=Testorder=__key__ firstNamehttp://localhost:8080/_ah/admin/datastore?kind=Testorder=firstName four http://localhost:8080/_ah/admin/datastore?kind=Testorder=four lastNamehttp://localhost:8080/_ah/admin/datastore?kind=Testorder=lastName six http://localhost:8080/_ah/admin/datastore?kind=Testorder=six threehttp://localhost:8080/_ah/admin/datastore?kind=Testorder=three agtwaXhvdG8tbGl2ZXILCxIEVGVzdBiNAQw12141JoedoorBobsticksfree So you get slammed with 12 writes. Each property is 2 writes. Here is a PropertyLess Entity Keyhttp://localhost:8080/_ah/admin/datastore?kind=PropertyLessorder=-__key__Write Ops ID/Namehttp://localhost:8080/_ah/admin/datastore?kind=PropertyLessorder=-__key__ agtwaXhvdG8tbGl2ZXITCxIMUHJvcGVydHlMZXNzGI4BDA 2142 2 writes. Who needs properties anyhow? That would mean you could query on them. Queries return results, results are reads. Reads cost money. Oh wait, that is what memcache is for... wait a sec, memcache took down my whole site Monday from MemcacheServiceExceptions http://code.google.com/status/appengine/detail/memcache/2011/12/12#ae-trust-detail-memcache-get-latency Also in my app app engine often spins up idle instances (that I cannot get rid of no matter what I configure) and send them exactly 1 request every 15 minutes so that I'm being charged the whole time for this instance I don't want. This happens to me also... Why is it if you have 6 instances, 2 of them get most of the requests 3 of them get none and occasionally App Engine will start up a 7th instance while the idle 3 still get nothing? Don't get me wrong. I LOVE what App Engine stands for and I have all the respect in the world for the App Engine team. BUT, I have been through SO much grief ranging from random app engine problems to having to migrate to an HR datastore to dramatic increases in pricing. When I signed up for this (old pricing), I thought the pricing would eventually get better (almost like gmail and disk space), but instead it went the opposite. Had my experience been perfect here and my app had run flawlessly all this time, I would have had no gripes and shut up and spent the extra cash without blinking. But, instead I have experienced hair loosing problems, massive variations in performance and got stuck with a much larger bill. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/DElz4NPSc8oJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] 40%+ increase of frontend instance hours over previous day
Hi Alan, On Wed, Dec 14, 2011 at 7:15 PM, Alan Xing alanx...@gmail.com wrote: Dear there, Our app 'snsanalytics' experienced more than 40% front end CPU hours usages on Dec 13, comparing to Dec 12. Our traffic level on Dec 13 is actually flat to less comparing to that of Dec 12. Except for the abnormal increase of front end CPU hours, there is no any noticeable change on other resource consumption like db write/read. There is absolutely no change from our side that can explain this increase. We didn't deploy any code in between. We didn't have any heavy lifting operations. Nothing we did is unusual comparing to previous day. Could any one from GAE team help explain/investigate what happened? I took a very quick look at your application. It looks like your datastore reads and writes did increase by about 10% between those two days. Your latency also increased for about 12 hours (without an increase in CPU usage). It is possible that a small increase in datastore latency slowed down your application enough that more instances were needed to service requests. It could also be that you had some particularly long running IO-bound tasks. But I don't have any strong evidence of either case (the latency increase does correlate with the increase in billed instances but I can't easily say why your latency increased). Cheers, Brian Thanks, Alan -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] 40%+ increase of frontend instance hours over previous day
On Thu, Dec 15, 2011 at 4:27 AM, Alan Xing alanx...@gmail.com wrote: Yes, we are still using the M/S datastore. We feel that we are not offering mission critical services. These services don't require the HRD level availability. HRD db read/write/store all costs more. What do you mean? The dollar cost for HRD is the same as MS. I know we could save some CPU hours by using Python 2.7 concurrency feature if we move over to HRD. There is loss and there is gain. Overall, we don't see our cost will reduce by moving M/S to HRD. That is why we are reluctant to make the move. I have always wondered why GAE doesn't extend Python 2.7 support to M/S. It doesn't seem there is any particular technical blocker. Maybe I'm wrong. In this random latency case, I again wonder why GAE doesn't plan to fix for M/S servers. We do have a fix - the HRD :-) Seriously, to make MS more consistent and reliable, you'd need to synchronously replicate the data across machines and data centers and that is exactly what MRD. Cheers, Brian Is the plan to completely phase out M/S servers in some near future? On Wed, Dec 14, 2011 at 1:27 AM, Kenneth kennet...@aladdinschools.com wrote: Are you using the old MS datastore or the HR datastore? If you're using MS then pretty much anything to do with the datastore is totally random, so expect random latency increases which result in higher instance counts and thus higher cost to you, randomly of course. Google will not be fixing these so move to the hr datastore when you can. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/PsCn4-PDjvUJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] 40%+ increase of frontend instance hours over previous day
On Thu, Dec 15, 2011 at 8:20 AM, Alan Xing alanx...@gmail.com wrote: The dollar cost of HRD and MS are the same? It was a surprise for me. I always had the impression HRD costed way more. Now I could not find that document except from Google search engine snapshot. As of Dec 10, 2011, GAE doc still mentioned HRD uses approximately three times the storage and CPU cost of the master/slave option. Please see attached snapshot. Regardless, I'm very happy to know that HRD is not costing more than MS any more. I will seriously think about to migrate to HRD soon. When HRD was launched it did cost 3x more than MS (since it costs Google at least 3x more to do the replication). But the pricing has later adjusted to be the same as MS. Cheers, Brian As of this moment today, we are still seeing way much higher front end instance hours than I would have expected before yesterday's spike. I'm not convinced by the explanations I have received so far. I'd think it is good to be transparent about pricing. Choosing a platform is a long term relationship, transparency can help stabilize the relationship. On Wed, Dec 14, 2011 at 12:14 PM, Brian Quinlan bquin...@google.com wrote: On Thu, Dec 15, 2011 at 4:27 AM, Alan Xing alanx...@gmail.com wrote: Yes, we are still using the M/S datastore. We feel that we are not offering mission critical services. These services don't require the HRD level availability. HRD db read/write/store all costs more. What do you mean? The dollar cost for HRD is the same as MS. I know we could save some CPU hours by using Python 2.7 concurrency feature if we move over to HRD. There is loss and there is gain. Overall, we don't see our cost will reduce by moving M/S to HRD. That is why we are reluctant to make the move. I have always wondered why GAE doesn't extend Python 2.7 support to M/S. It doesn't seem there is any particular technical blocker. Maybe I'm wrong. In this random latency case, I again wonder why GAE doesn't plan to fix for M/S servers. We do have a fix - the HRD :-) Seriously, to make MS more consistent and reliable, you'd need to synchronously replicate the data across machines and data centers and that is exactly what MRD. Cheers, Brian Is the plan to completely phase out M/S servers in some near future? On Wed, Dec 14, 2011 at 1:27 AM, Kenneth kennet...@aladdinschools.com wrote: Are you using the old MS datastore or the HR datastore? If you're using MS then pretty much anything to do with the datastore is totally random, so expect random latency increases which result in higher instance counts and thus higher cost to you, randomly of course. Google will not be fixing these so move to the hr datastore when you can. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/PsCn4-PDjvUJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: App Engine 1.6.1 Released!
Hi Waleed, On Wed, Dec 14, 2011 at 7:04 AM, Waleed Abdulla wal...@ninua.com wrote: A couple of questions: 1. Since not all my requests are the same, I wouldn't want to use bigger front-end instances across the board. Are there plans to allow setting instance type based on app.yaml routes? 2. Can you clarify this point? Will mapreduce not be supported on Python 2.7? WARNING: Starting with the 1.6.2 release of the experimental Python 2.7 runtime, the mapreduce and datastore_admin builtins will not be supported. mapreduce will be supported before release (meaning before both Python 2.7 a mapreduce are released non-experimentally) But it doesn't work in Python 2.7 right now. Cheers, Brian Regards, Waleed On Tue, Dec 13, 2011 at 11:46 AM, Vivek Puri v...@vivekpuri.com wrote: Ikai, Looking at my Application setting, i see that the Frontend Instance Class is set to F2 by default. I didnt change it to F2. However the docs(http://code.google.com/appengine/docs/adminconsole/ performancesettings.html#Setting_the_Frontend_Instance_Class) say that F1 is default . Let me know which statement is correct. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] DeadlineExceededError on HRD and sky-high latency (python27+threadsafe)
On Tue, Dec 6, 2011 at 6:14 AM, sergio.jar...@gmail.com sergio.jar...@gmail.com wrote: Hi! I've successfully migrated from master-slave to hdd with python 27 and threadsafe however I've been getting sporadically a few deadline exceeded errors (bellow stack trace excerpt). I also have configured min idle requests to automatic and maximum to 1 and pending latency to 800ms, I've observed on occasion latencies of up to 63000(!!!)ms and without app engine spinning extra instances to serve requests. Any thoughts? I can provide app-id off-list if someone from google wants to have a look. You are likely encountering this issue: http://code.google.com/p/googleappengine/issues/detail?id=6323 Could you try setting threadsafe to no and seeing if that fixes the problem? Cheers, Brian Thanks! File /base/python27_runtime/python27_lib/versions/1/google/appengine/ ext/db/__init__.py, line 3588, in __get__ instance = get(reference_id) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/ext/db/__init__.py, line 1515, in get return get_async(keys, **kwargs).get_result() File /base/python27_runtime/python27_lib/versions/1/google/ appengine/api/apiproxy_stub_map.py, line 592, in get_result return self.__get_result_hook(self) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/datastore/datastore_rpc.py, line 1437, in __get_hook entities = rpc.user_data(entities) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/api/datastore.py, line 623, in local_extra_hook return extra_hook(result) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/ext/db/__init__.py, line 1485, in extra_hook model = cls1.from_entity(entity) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/ext/db/__init__.py, line 1419, in from_entity return cls(None, _from_entity=entity, **entity_values) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/ext/db/__init__.py, line 1800, in __init__ super(Expando, self).__init__(parent, key_name, _app, **kwds) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/ext/db/__init__.py, line 945, in __init__ prop.__set__(self, value) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/ext/db/__init__.py, line 600, in __set__ setattr(model_instance, self._attr_name(), value) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/ext/db/__init__.py, line 1826, in __setattr__ check_reserved_word(key) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/ext/db/__init__.py, line 300, in check_reserved_word if datastore_types.RESERVED_PROPERTY_NAME.match(attr_name): DeadlineExceededError -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: DeadlineExceededError on HRD and sky-high latency (python27+threadsafe)
Hey Andrius, On Wed, Dec 7, 2011 at 9:14 AM, Andrius A andriu...@gmail.com wrote: You are not the only one, I don't like the fact that google puts a blame on MS, there is something wrong with GAE constantly going on and they should admit that and explain us in more details what's happening. There is a known bug where the combination of Python 2.7 and enabling concurrent requests causes large latency increases. The Python 2.7 runtime is still experimental and bugs like this are expected. There are also known issues where accessing the MS datastore can result in highly variable latencies and possibly errors. This is the result of the design of the MS datastore and is why we promote migrating to the HR datastore. The App Engine Administration Console uses the MS datastore (for a variety of esoteric reasons) so its latency spikes will often correlate with MS latency spikes. We have a form for issue like this to be noticed, please fill in to get more attention: https://docs.google.com/spreadsheet/viewform?hl=en_USformkey=dC1DMmpPSU1WZnk0d1FMa3JmNXIwaGc6MQ#gid=0 Can I also suggest that people file bugs using the public issue tracker at: http://code.google.com/p/googleappengine/issues ? Also, if you are using an experimental feature, it would be great if you do a quick search to see if the issue has already been filed and make a note or star it yourself it is has. The more information we have the easier it is to prioritize and fix bugs. I really believe into GAE and still think this is a great a platform, but it has issues which need be admitted and taken care of. No one is more aware of App Engine's imperfections than the App Engine team and we have a public issue tracker so that people can make us aware of their particular issues and see what problems other people are having. Cheers, Brian On 6 December 2011 17:41, Sarang sar...@mycontactid.net wrote: Glad to see this post as I am also suffering at the hands of GAE for the last month now. Every now and then DeadlineExceeded errors shoot up giving response times of over a minute! See my thread: https://groups.google.com/forum/?hl=en#!topic/google-appengine/NpqadsOUyZ4 Sarang -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/-JsYHq24EyMJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: DeadlineExceeded errors haunting every now and then
On Wed, Dec 7, 2011 at 10:44 AM, Andrius A andriu...@gmail.com wrote: In docs it says that HRD is more costly The High Replication Datastore costs more due to the additional replication (see billing page for pricing details). Due to the higher cost, we recommend High Replication Datastore primarily for those developers building critical App Engine applications that require the highest availability. I've filed a bug against this documentation. It also says queries on a single guestbook to be strongly consistent, but also limits changes to the guestbook to 1 write per second (the supported limit for entity groups). Therefore, writing to a single entity group per guestbook is not ideal when high usage is expected. If your app is likely to encounter heavy write usage, consider using another means. For example, you can put recent posts in memcache with an expiration, and then display a mix of recent posts from memcache and posts retrieved from the datastore. How could we trust by putting data to memcache if we know it can be evicted any time? You wouldn't as definitive storage but it is reasonable to cache recent posts there. To use HRD is not viable for applications which need strong consistency for high rate of puts. You may have to think about how your organize your data in order to satisfy your consistency and throughput requirements but I am very skeptical when you say that it is simply not viable. Google, for example, has been successful using the same technology for many of its own high-volume properties. At the moment MS is perfect what it does but bloody thing keeps dying.. That is an architectural problem that is very hard to fix with making the same kinds of trade-offs that we made when implementing HRD . Cheers, Brian I can't believe there is no way to improve MS and you are switching to HRD. In long term HRD can cause more problems for applications parts were it wasn't aticipating to receive higher rate of inserts, and having a limit of 1 write per second is a disastrous. On 6 December 2011 23:29, Gregory D'alesandre gr...@google.com wrote: Hi Kenneth, We are planning to add blobstore migration at some point but it won't likely be until the middle of Q1 at the earliest. Greg On Tue, Dec 6, 2011 at 3:27 PM, Kenneth kennet...@aladdinschools.com wrote: Hi Greg, Can you give us some indication if the migration tool is ever going to include blobstore migration? That's the only thing holding me back. I know I can do it myself but I'd rather not. I see now that it has gone ga, have you stuck a fork in it and called it done? Thanks -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/f7KzNP-pCuwJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: DeadlineExceededError on HRD and sky-high latency (python27+threadsafe)
Hi Andrius, On Wed, Dec 7, 2011 at 11:15 AM, Andrius A andriu...@gmail.com wrote: Hey Brian, thank you for your reply. I am still on 2.5 since 2.7 is still experimental, and I don't really believe that MS is everything to blame, there are issues with requests latencies, which I have monitored and there was nothing wrong with datastore. I have been posting about that here and opened a ticket but have not received an answer unfortunately. I couldn't see your name attached to an issue in the public issue tracker. Could you provide a link? I am moaning here all the time because we shouldn't be charged so much as you confirmed already many times that there are issues and 2.7 is still faulty. The main point here is that we should not pay more just because GAE is now called production ready. I will be repeating that until you go back to preview release or admit that by dropping the prices or improve the reliability. You can send pricing-related issues to Peter at p...@google.com. Cheers, Brian Thanks, Andrius On 6 December 2011 22:56, Brian Quinlan bquin...@google.com wrote: Hey Andrius, On Wed, Dec 7, 2011 at 9:14 AM, Andrius A andriu...@gmail.com wrote: You are not the only one, I don't like the fact that google puts a blame on MS, there is something wrong with GAE constantly going on and they should admit that and explain us in more details what's happening. There is a known bug where the combination of Python 2.7 and enabling concurrent requests causes large latency increases. The Python 2.7 runtime is still experimental and bugs like this are expected. There are also known issues where accessing the MS datastore can result in highly variable latencies and possibly errors. This is the result of the design of the MS datastore and is why we promote migrating to the HR datastore. The App Engine Administration Console uses the MS datastore (for a variety of esoteric reasons) so its latency spikes will often correlate with MS latency spikes. We have a form for issue like this to be noticed, please fill in to get more attention: https://docs.google.com/spreadsheet/viewform?hl=en_USformkey=dC1DMmpPSU1WZnk0d1FMa3JmNXIwaGc6MQ#gid=0 Can I also suggest that people file bugs using the public issue tracker at: http://code.google.com/p/googleappengine/issues ? Also, if you are using an experimental feature, it would be great if you do a quick search to see if the issue has already been filed and make a note or star it yourself it is has. The more information we have the easier it is to prioritize and fix bugs. I really believe into GAE and still think this is a great a platform, but it has issues which need be admitted and taken care of. No one is more aware of App Engine's imperfections than the App Engine team and we have a public issue tracker so that people can make us aware of their particular issues and see what problems other people are having. Cheers, Brian On 6 December 2011 17:41, Sarang sar...@mycontactid.net wrote: Glad to see this post as I am also suffering at the hands of GAE for the last month now. Every now and then DeadlineExceeded errors shoot up giving response times of over a minute! See my thread: https://groups.google.com/forum/?hl=en#!topic/google-appengine/NpqadsOUyZ4 Sarang -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/-JsYHq24EyMJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received
Re: [google-appengine] Re: DeadlineExceeded errors haunting every now and then
On Wed, Dec 7, 2011 at 11:34 AM, Andrius A andriu...@gmail.com wrote: Hi Brian, just an idea, do you think you could build highly reliable simple but fast hash table similar to memcache but that could guarantee limited number of single entities permanent existence per application? Probably not. Highly reliable suggests that you synchronously replicate your data across several machines and data centers (assuming that you want consistency) but it is hard to make that fast. Cheers, Brian Having that and HRD would make things work brilliantly! On 7 December 2011 00:13, Brian Quinlan bquin...@google.com wrote: On Wed, Dec 7, 2011 at 10:44 AM, Andrius A andriu...@gmail.com wrote: In docs it says that HRD is more costly The High Replication Datastore costs more due to the additional replication (see billing page for pricing details). Due to the higher cost, we recommend High Replication Datastore primarily for those developers building critical App Engine applications that require the highest availability. I've filed a bug against this documentation. It also says queries on a single guestbook to be strongly consistent, but also limits changes to the guestbook to 1 write per second (the supported limit for entity groups). Therefore, writing to a single entity group per guestbook is not ideal when high usage is expected. If your app is likely to encounter heavy write usage, consider using another means. For example, you can put recent posts in memcache with an expiration, and then display a mix of recent posts from memcache and posts retrieved from the datastore. How could we trust by putting data to memcache if we know it can be evicted any time? You wouldn't as definitive storage but it is reasonable to cache recent posts there. To use HRD is not viable for applications which need strong consistency for high rate of puts. You may have to think about how your organize your data in order to satisfy your consistency and throughput requirements but I am very skeptical when you say that it is simply not viable. Google, for example, has been successful using the same technology for many of its own high-volume properties. At the moment MS is perfect what it does but bloody thing keeps dying.. That is an architectural problem that is very hard to fix with making the same kinds of trade-offs that we made when implementing HRD . Cheers, Brian I can't believe there is no way to improve MS and you are switching to HRD. In long term HRD can cause more problems for applications parts were it wasn't aticipating to receive higher rate of inserts, and having a limit of 1 write per second is a disastrous. On 6 December 2011 23:29, Gregory D'alesandre gr...@google.com wrote: Hi Kenneth, We are planning to add blobstore migration at some point but it won't likely be until the middle of Q1 at the earliest. Greg On Tue, Dec 6, 2011 at 3:27 PM, Kenneth kennet...@aladdinschools.com wrote: Hi Greg, Can you give us some indication if the migration tool is ever going to include blobstore migration? That's the only thing holding me back. I know I can do it myself but I'd rather not. I see now that it has gone ga, have you stuck a fork in it and called it done? Thanks -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/f7KzNP-pCuwJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received
Re: [google-appengine] Re: DeadlineExceededError on HRD and sky-high latency (python27+threadsafe)
On Wed, Dec 7, 2011 at 11:48 AM, Andrius A andriu...@gmail.com wrote: Thanks Brian for sending this info, as it is really tricky to contact you guys. If you need technical help from the App Engine team, you should consider a Premier Account: http://code.google.com/appengine/docs/premier/index.html This is the issue I am mostly concerned: http://code.google.com/p/googleappengine/issues/detail?id=6309can=1q=175mscolspec=ID%20Type%20Component%20Status%20Stars%20Summary%20Language%20Priority%20Owner%20Log I had a look at your issue and it isn't something that I can explain without some significant digging. Cheers, Brian On 7 December 2011 00:24, Brian Quinlan bquin...@google.com wrote: Hi Andrius, On Wed, Dec 7, 2011 at 11:15 AM, Andrius A andriu...@gmail.com wrote: Hey Brian, thank you for your reply. I am still on 2.5 since 2.7 is still experimental, and I don't really believe that MS is everything to blame, there are issues with requests latencies, which I have monitored and there was nothing wrong with datastore. I have been posting about that here and opened a ticket but have not received an answer unfortunately. I couldn't see your name attached to an issue in the public issue tracker. Could you provide a link? I am moaning here all the time because we shouldn't be charged so much as you confirmed already many times that there are issues and 2.7 is still faulty. The main point here is that we should not pay more just because GAE is now called production ready. I will be repeating that until you go back to preview release or admit that by dropping the prices or improve the reliability. You can send pricing-related issues to Peter at p...@google.com. Cheers, Brian Thanks, Andrius On 6 December 2011 22:56, Brian Quinlan bquin...@google.com wrote: Hey Andrius, On Wed, Dec 7, 2011 at 9:14 AM, Andrius A andriu...@gmail.com wrote: You are not the only one, I don't like the fact that google puts a blame on MS, there is something wrong with GAE constantly going on and they should admit that and explain us in more details what's happening. There is a known bug where the combination of Python 2.7 and enabling concurrent requests causes large latency increases. The Python 2.7 runtime is still experimental and bugs like this are expected. There are also known issues where accessing the MS datastore can result in highly variable latencies and possibly errors. This is the result of the design of the MS datastore and is why we promote migrating to the HR datastore. The App Engine Administration Console uses the MS datastore (for a variety of esoteric reasons) so its latency spikes will often correlate with MS latency spikes. We have a form for issue like this to be noticed, please fill in to get more attention: https://docs.google.com/spreadsheet/viewform?hl=en_USformkey=dC1DMmpPSU1WZnk0d1FMa3JmNXIwaGc6MQ#gid=0 Can I also suggest that people file bugs using the public issue tracker at: http://code.google.com/p/googleappengine/issues ? Also, if you are using an experimental feature, it would be great if you do a quick search to see if the issue has already been filed and make a note or star it yourself it is has. The more information we have the easier it is to prioritize and fix bugs. I really believe into GAE and still think this is a great a platform, but it has issues which need be admitted and taken care of. No one is more aware of App Engine's imperfections than the App Engine team and we have a public issue tracker so that people can make us aware of their particular issues and see what problems other people are having. Cheers, Brian On 6 December 2011 17:41, Sarang sar...@mycontactid.net wrote: Glad to see this post as I am also suffering at the hands of GAE for the last month now. Every now and then DeadlineExceeded errors shoot up giving response times of over a minute! See my thread: https://groups.google.com/forum/?hl=en#!topic/google-appengine/NpqadsOUyZ4 Sarang -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/-JsYHq24EyMJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group
Re: [google-appengine] Re: GAE 1.6 can not import module jinja2
On Sat, Dec 3, 2011 at 2:08 PM, speg stevehiems...@gmail.com wrote: How are we supposed to know this? I jumped into GAE with the 2.7 tutorial and there was no mention of installing packages locally. I thought it was all bundled in GAE. The fact that required pure-Python modules are not packaged with the SDK is simply a bug. Installing it yourself is the work-around. Cheers, Brian On Nov 8, 6:39 pm, Brian Quinlan bquin...@google.com wrote: As Matt says, you need to install jinja2 on your local Python instance. This will be fixed in a future release. Cheers, Brian On Wed, Nov 9, 2011 at 7:45 AM, NG nikolas.goe...@googlemail.com wrote: Same here -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/BDGGzz0ahFMJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: 1.6.1 Pre-release SDKs are live
On Tue, Dec 6, 2011 at 6:05 PM, Westmark fredrik.westm...@gmail.com wrote: I would be happy if you could specify the versions of included third- party libraries, e.g. ndb and webapp2. ndb is being integrated into the google namespace so it isn't really third-party anymore. You can specify the version of other third-party libraries. See: http://code.google.com/appengine/docs/python/python27/using27.html#Configuring_Libraries Cheers, Brian Cheers / Fredrik -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Doom day
Hi Brandon, On Sat, Dec 3, 2011 at 8:11 PM, Brandon Wirtz drak...@digerat.com wrote: I run on 2.7. I have been rock solid and my costs are 1/10th what they are on 2.5 Wow, that's awesome! I had very few code changes other than removing CGI handlers. Also good news. I would recommend 2.7 without hesitation. Pointing out 2 bugs that aren't well documented, and (one of which I can't Repo) Is more of a I'm too lazy to do the migration than a real excuse. If you aren't testing on 2.7 you are weeks from being deployed on it anyway, and are just griping. Get a test version of your app on it, find out where it fails and if you have bugs file them. Don't be a wuss unless you have the same code snippet that is listed in a bug in your source code. Then I might say you should test on 2.7 but not invest time in adapting the code beyond removing 2.7 incompatible code. Both of the bugs that Alexis mentioned are significant and we can reproduce them - though the probability of being affected by http://code.google.com/p/googleappengine/issues/detail?id=6401 should be pretty low. Python 2.7 is still experimental and your good experiences aren't a guarantee that others won't encounter serious problems. Cheers, Brian -Brandon -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Alexis Sent: Saturday, December 03, 2011 1:02 AM To: Google App Engine Subject: [google-appengine] Re: Doom day We are still using Python25 too and don't feel confident moving our production apps to 2.7 when seeing issues like this: http://code.google.com/p/googleappengine/issues/detail?id=6401 or this: http://code.google.com/p/googleappengine/issues/detail?id=6323 or the one Kaan linked to... -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Unhappy GAE Developers Petition
Hi Andrius, On Thu, Nov 24, 2011 at 11:40 PM, Andrius A andriu...@gmail.com wrote: Yes, we all do mistakes and GAE platform is great. But this thread is not about this discussion. So if you have any outstanding issues fill in the form! Personally I can't understand how you can be happy with price increase and production issues with python 2.7 As an experimental release, we were expecting significant issues with the Python 2.7 runtime (i.e. instability, missing features, performance problems) and I've actually been pleasantly surprised at how few have been found (thanks to everyone for helping us test!). Are you unhappy with Python 2.7 production issues because we didn't adequately explain that you might experience problems when using it? Or is there some other reason? Cheers, Brian I totally believe that for hosting just html in app engine is great but if you try using more apis such as backends, channels, task queues you will come up to lots of problems. On Nov 24, 2011 12:10 PM, Simon Knott knott.si...@gmail.com wrote: Hi, In my experience on the GAE forums, a lot of unhappy developers I see are actually the people who don't understand the GAE architecture and have poorly designed applications for the environment. Their applications are subsequently slow and/or expensive to run, for example because they haven't denormalised their data structure or they've indexed every property under the sun, and they then lay the blame on Google. I personally applaud the development team who provide adhoc support on these forums, as they have far more patience than I have! I'm not saying that there couldn't be some improvements to the support channels for genuine issues with the service and I would love to have a more tiered structure, as someone has suggested in the last day, so that smaller companies could get support for less money, within a larger SLA. It will be interesting to see who responds to your questionnaire and the complaints that are aired. Cheers, Simon -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/WRUFpsUp6IcJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Multithreading question
Hi Richard, On Wed, Nov 23, 2011 at 7:18 AM, Richard Arrano rickarr...@gmail.com wrote: Hello, Quick question regarding multithreading in Python 2.7: I have some requests that call 2-3 functions that call the memcache in each function. It would be possible but quite complicated to just use get_multi, and I was wondering if I could simply put each function into a thread and run the 2-3 threads to achieve some parallelism. Would this work or am I misunderstood about what we can and cannot do with regards to multithreading in 2.7? This will certainly work put I'm not sure that it would be worth the complexity. Fetching a value from memcache usually takes 5ms so parallelizing 3 memcache gets is going to save you ~10ms. Cheers, Brian Thanks, Richard -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Querying App Engine Datastore with Date WHERE clause
Hi Jamil, On Wed, Nov 23, 2011 at 8:20 AM, Jamil Rzayev jamil.rza...@gmail.com wrote: I need to query App Engine Datastore with date parameter in where clause. When I use App Engine Datastore Viewer and type the query like: Select * From Kind where date='2011-11-22' it does give me nothing. Could you please suggest me how to query data entities with specific date value in where clause? Take a look at: http://code.google.com/appengine/docs/python/datastore/gqlreference.html Cheers, Brian -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Multithreading question
On Wed, Nov 23, 2011 at 8:48 AM, Richard Arrano rickarr...@gmail.com wrote: @Brandon: This is true but it just would take a lot of rewriting that may or may not be worth it. @Brian Thanks for the tip, I didn't even realize that(I haven't been using AppStats, shame on me). Would the savings be worth it, in your opinion, when they're not present in the cache and have to resort to 3 gets of varying size? Its hard to give advice on this kind of complexity vs. performance trade-off without really understanding the application. Datastore gets are slower than memcache gets but are still pretty quick. Cheers, Brian On Nov 22, 12:37 pm, Brian Quinlan bquin...@google.com wrote: Hi Richard, On Wed, Nov 23, 2011 at 7:18 AM, Richard Arrano rickarr...@gmail.com wrote: Hello, Quick question regarding multithreading in Python 2.7: I have some requests that call 2-3 functions that call the memcache in each function. It would be possible but quite complicated to just use get_multi, and I was wondering if I could simply put each function into a thread and run the 2-3 threads to achieve some parallelism. Would this work or am I misunderstood about what we can and cannot do with regards to multithreading in 2.7? This will certainly work put I'm not sure that it would be worth the complexity. Fetching a value from memcache usually takes 5ms so parallelizing 3 memcache gets is going to save you ~10ms. Cheers, Brian Thanks, Richard -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Multithreading question
On Wed, Nov 23, 2011 at 9:11 AM, Richard Arrano rickarr...@gmail.com wrote: I see, I'm guessing it probably isn't worth it to optimize this particular area but it's good to know that the multithreading ability would work in a more complex instance where I truly needed the parallelism. One last question on the topic, having to do with threadsafe: the function that I was referring to was actually a decorator that checks certain permissions that I insert before a large amount of handlers. It also stores the returned objects via self.permissions for example. Is there a possibility of a race condition on self.permissions or does it function in such a manner that this is impossible? If you are using webapp2 then a new handler instance is created for every request. Cheers, Brian Thanks, Richard On Nov 22, 1:56 pm, Brian Quinlan bquin...@google.com wrote: On Wed, Nov 23, 2011 at 8:48 AM, Richard Arrano rickarr...@gmail.com wrote: @Brandon: This is true but it just would take a lot of rewriting that may or may not be worth it. @Brian Thanks for the tip, I didn't even realize that(I haven't been using AppStats, shame on me). Would the savings be worth it, in your opinion, when they're not present in the cache and have to resort to 3 gets of varying size? Its hard to give advice on this kind of complexity vs. performance trade-off without really understanding the application. Datastore gets are slower than memcache gets but are still pretty quick. Cheers, Brian On Nov 22, 12:37 pm, Brian Quinlan bquin...@google.com wrote: Hi Richard, On Wed, Nov 23, 2011 at 7:18 AM, Richard Arrano rickarr...@gmail.com wrote: Hello, Quick question regarding multithreading in Python 2.7: I have some requests that call 2-3 functions that call the memcache in each function. It would be possible but quite complicated to just use get_multi, and I was wondering if I could simply put each function into a thread and run the 2-3 threads to achieve some parallelism. Would this work or am I misunderstood about what we can and cannot do with regards to multithreading in 2.7? This will certainly work put I'm not sure that it would be worth the complexity. Fetching a value from memcache usually takes 5ms so parallelizing 3 memcache gets is going to save you ~10ms. Cheers, Brian Thanks, Richard -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
