[google-appengine] Re: How to Backup Datastore
Another solution, that I just released over the holidays, is Google App Engine Backup and Restore (Gaebar). The screencast, links to repositories, etc., are in the blog post: http://aralbalkan.com/1784 Bug reports, comments, suggestions, and patches are welcome! :) Aral On Nov 25, 12:35 pm, kaspars...@gmail.com kaspars...@gmail.com wrote: You may also want to check out AppRocket which replicates AppEngine datastore to MySQL: http://code.google.com/p/approcket/ On Nov 18, 6:00 am, Edward 061...@gmail.com wrote: Is there a best practice for backing up one's datastore? Is there a way to do this on the Google site to avoid having to do this individually? If not, is there something in the pipeline? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Google Accounts down?
Is anyone else getting: Sorry, there seems to be a problem. The service you're looking for is temporarily unavailable. We're working hard to restore your access as soon as possible. Please try again in a few hours. Thanks for your patience. While trying to sign in to their App Engine app? Thanks, Aral --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Can i use google app engines ssl with level 2 integration?
Hi Marzia, I've got a question about this: On my app, I know that I was losing the session when I had PayPal return to www.myapp.com when the user had originally signed in from myapp.com (naked domain). I've since fixed that issue when by forwarding everything to www. in my front controller. However, how will the user's session be affected if they sign in from www.myapp.com and then return to myapp.appspot.com? Does this mean that the user will have to sign in twice? Thanks, Aral On Nov 18, 9:44 pm, Marzia Niccolai [EMAIL PROTECTED] wrote: Hi, I'm guessing you mean 'level 2' integration with Google Checkout. And yes, this is possible, simply enter the appspot URL where you have written your secure callback handler in to your Google Checkout merchant account settings. -Marzia On Tue, Nov 18, 2008 at 1:42 PM, Barry Hunter [EMAIL PROTECTED]wrote: level 2 integration of what? On Tue, Nov 18, 2008 at 8:28 PM, Will F. [EMAIL PROTECTED] wrote: Can i use google app engines ssl with level 2 integration? -- Barry -www.nearby.org.uk-www.geograph.org.uk- --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Can i use google app engines ssl with level 2 integration?
And here's a useful link of Google Checkout Level 1 and 2 checkout implemented in web2py: http://mail.python.org/pipermail/python-announce-list/2008-October/006988.html Aral --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: urlfetch ResponseTooLargeError - what is the limit?
Hi Barry, python variables are limited to 1Mb anyway, so the urlfetch wouldnt be able to return bigger files. I'm assuming you're saying that in App Engine, variables are limited to 1MB (any data structure appears to have this hard limit). Just to clarify, in Python, variables do not have a size limit (separate from your OS's memory limit). Aral --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Google App Engine Roadmap - Now Published
The roadmap is a great step forward, thank you. I am, however, disappointed at seeing Support for a new runtime language on the roadmap for the immediate feature. Quite bluntly, I would have expected better triage. There are showstopping issues in Google App Engine at the moment. Applications that require reporting, or any sort of maintenance features/long-running processes have to implement quite ridiculous client-side hacks. My app has webbasedcron hitting a URL every minute to run its email queue. If I want to email everyone, I run my admin routine which uses Ajax calls to iterate over the several thousand members. When I send an email from my app I have no idea what happens to it (and we had such a hard time because said email was ending up in users' spam folders. Google App Engine is missing an entire mode of operation that is required for web apps. It handles lots of tiny request/response transactions well but you also need to be able to run reporting and maintenance. Long-running processes that *do not need to scale* but are nonetheless essential. I don't care that I will soon be able to create such hacks in another language. I'd like those issues addressed first before time and effort is expanded on a feature that is purely one of preference. As I mentioned in a recent post, adding support for other languages to Google App Engine today is like sewing a new set of drapes for a house that doesn't have any walls yet. I'd rather have the walls instead of a nice to have vanity feature. Thanks, Aral On Oct 24, 2:11 am, Bill [EMAIL PROTECTED] wrote: * Support for a new runtime language Would you be able to say how different languages will be partitioned? Could you access a single datastore with more than one language or have different languages running on the same VM? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Google App Engine Roadmap - Now Published
To illustrate my point further with an analogy: Google App Engine is here right now: http://icanhaz.com/rightscaleweb It needs to be here: http://icanhaz.com/rightscalepremium Aral On Nov 5, 3:54 pm, Aral Balkan [EMAIL PROTECTED] wrote: The roadmap is a great step forward, thank you. I am, however, disappointed at seeing Support for a new runtime language on the roadmap for the immediate feature. Quite bluntly, I would have expected better triage. snip --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Can Google please stop marking Google App Engine mail as spam?
No, I'm saying it shouldn't blacklist them by default. Aral On Oct 14, 7:18 pm, Barry Hunter [EMAIL PROTECTED] wrote: I'm not quite sure I understand, are you suggesting that Postini should whitelist all AppEngine emails? That sounds bad, would make appengine an attractive target for spammers. It's true at the moment the technical hurdles make it unattractive, but a default whitelist would make it very much less so. snip --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] exceptions.SystemError?
Is anyone else seeing errors similar to the following. It seems to
happen randomly -- possibly following a datastore write.
The only thing that fixes the app after it happens is to re-deploy.
Thoughts?
Issue link: http://tinyurl.com/4c2z2v
Thanks,
Aral
500 Error at /admin/ (type 'exceptions.SystemError')
type 'exceptions.SystemError': Parent module 'django.template' not
loaded
Traceback (most recent call last):
File /base/data/home/apps/si/4.18/common/appenginepatch/main.py,
line 24, in real_main
util.run_wsgi_app(application)
File /base/python_lib/versions/1/google/appengine/ext/webapp/
util.py, line 76, in
run_wsgi_app
result = application(env, _start_response)
File /base/data/home/apps/si/4.18/common/django/core/handlers/
wsgi.py, line 239, in
__call__
response = self.get_response(request)
File /base/data/home/apps/si/4.18/common/django/core/handlers/
base.py, line 128, in
get_response
return self.handle_uncaught_exception(request, resolver, exc_info)
File /base/data/home/apps/si/4.18/common/django/core/handlers/
base.py, line 160, in
handle_uncaught_exception
return callback(request, **param_dict)
File /base/data/home/apps/si/4.18/core/decorators.py, line 189, in
autorization_decorator
return target_method(request, first_level, second_level, context,
*args, **kwargs)
File /base/data/home/apps/si/4.18/core/views.py, line 2643, in
error_500
return helpers.render_response('500', first_level, second_level,
context,
'HttpResponseServerError')
File /base/data/home/apps/si/4.18/core/helpers.py, line 214, in
render_response
return eval(response_type)(loader.render_to_string(templateFile,
context))
File /base/data/home/apps/si/4.18/common/django/template/
loader.py, line 102, in
render_to_string
t = get_template(template_name)
File /base/data/home/apps/si/4.18/common/django/template/
loader.py, line 80, in
get_template
source, origin = find_template_source(template_name)
File /base/data/home/apps/si/4.18/common/django/template/
loader.py, line 54, in
find_template_source
mod = __import__(module, globals(), locals(), [attr])
type 'exceptions.SystemError': Parent module 'django.template' not
loaded
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: Why Google App Engine is broken and what Google must do to fix it.
Hi Jorge, You clearly don't get must of the stuff you are talking about. ALL the quotas are there for you to be efficient they the whole purpose, they are not hard coded to make your live bad they are in there to make the system better. I feel that you have missed the point of my post. Rest assured that I get the stuff I'm talking about. Please re-read the post and you will see that it is not an attempt to bash Google App Engine. Quite the contrary, it is a simple summary of the issues encountered by one developer while developing a real-world application. The reason for the post is to highlight these issues so that they can be fixed and thus, ultimately, to improve Google App Engine. As you can see from the replies here, and on the post, I am not the only one who has encountered these issues either. We provide constructive criticism for the things that we care about so that they can evolve. I invite you to add value to the conversation and help make Google App Engine better. Aral --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Why Google App Engine is broken and what Google must do to fix it.
@davew: As Sylvain points out, below, the simplest things can cause this. I've had straight-to-template calls cause high CPU errors and, to tell you the truth, even if I could meaningfully profile Django templates, I don't think there's much I could do about it. Especially, as Sylvain states, it appears to be random. Aral On Oct 4, 9:30 am, Sylvain [EMAIL PROTECTED] wrote: @davew : If you don't know what causes the problems, then find out? Is it in your code? Is it a datastore put timeout? Profile it? I've spent a lot of time to understand it too. And I realize that you can raise this Quota Warning in 2 lines : myModel.all(), myModel.fetch(100). So here, there is nothing to optimize or to profile, particularly when it is random. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Why Google App Engine is broken and what Google must do to fix it.
Hi David, The point of my post is that the scalable and distributed system should be just one mode of operation of Google App Engine as it ignores the other 50% use case which is admin and long-running processes. Those do _not_ have to scale. But they are essential. Aral I fail to see how either of these could be implemented in a truly scalable and distributed system such as App Engine. The data is potentially spread over thousands of machines, so any kind of global co-ordination (needed for both entity counts and transactions) would require a single machine to be in charge, and hence the scaling would be limited to that machine's capacity. snip --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Why Google App Engine is broken and what Google must do to fix it.
I just wrote up a blog post summarizing the biggest issues I have with App Engine: http://aralbalkan.com/1504 If you are developing real-world/commercial apps with App Engine, please add your thoughts to the discussion. Thanks, Aral --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Retrieving More than 1000 entities
There is now a 1,000 limit on offsets (which was introduced without any notice). Effectively, your only option is to do what José recommends. The docs must be updated to reflect this limit and the fact that everyone is basically forced to keep manual numeric keys until either sortable keys are implemented or the offset limit lifted (the former is the long-term solution). Aral On Oct 3, 8:20 am, José Oliver Segura [EMAIL PROTECTED] wrote: On Fri, Oct 3, 2008 at 5:36 AM, Venkatesh Rangarajan [EMAIL PROTECTED] wrote: Theo, that will not work. It will aways get the same 1000..and the only thing offset does is get 1 to 100 or 100 to 200. Tony : Well, i have a simple search http://payrate.appspot.com/infosys_salary This search returns more than 1000 records..who do i show them all ? My page explodes after page 10 Wihtout knowing exactly what explodes means in this context: if you need to page more than 1000 results, you need to add a condition to your query to perform manual pagination (that is: filter by the query current conditions plus a condition telling that whatever field needs to be bigger than the item at position n [where n is elements/page]) That is: you have to do it by hand... I thinkhttp://sites.google.com/site/io/building-scalable-web-applications-wi... has some slides showing how to paginate (you can take them as example) best, Jose --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Too Many Versions (403)
How many versions did you have? (I'm assuming you've hit some sort of upper limit on _minor_ versions?) Aral On Sep 30, 7:34 pm, Venkatesh Rangarajan [EMAIL PROTECTED] wrote: Hi, I am getting the following error when I try to upload my app. Been having this since last night and cannot update my APP at all ?snip --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] BadRequestError: offset may not be above 1000
Hi, I'm not sure when this changed but it seems that you cannot specify offsets above 1,000 which kind of kills my backup app. How are we supposed to iterate over large datasets? Thanks, Aral --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: High Amount CPU Quota should be removed
Starred it. I agree that this is a big problem. The quota system in general is the thorn in Google App Engine's side and, from a public relations perspective, quite a nightmare. Given that most people will check out an application when it's getting a lot of publicity (and hits), the first experience many people have with a Google App Engine app is the Over Quota message. Not exactly the first impression you want to make for your shiny new Cloud solution. Aral On Sep 21, 12:27 pm, mitnickcbc [EMAIL PROTECTED] wrote: I have filed a feature request for this issue, if you feel so please star it. http://code.google.com/p/googleappengine/issues/detail?id=720 Here is the description: Well, I don't get the purpose for having a High Amount CPU Quota since there is already a general CPU Quota. And there are quite a lot of problems caused by this quota which restrict my application from affording more load. And my application is far away from a 5 million month PV app. 1. Requests can become a high amount CPU one very easily. A little complex request will consume more than 3K mcycles. And what do I mean a little complex? If you do more than 2 put, or you do a url fetch, or you do a query based on an order to select more than 50 model, or you do a put on a model more than 20 fields, it is that complex. In my case, 80% of the requests are that complex and I don't think it can be optimized anymore since I do need to do put and url fetch. 2. Bad visibility to this quota. There is no way we can know when we will run out of quota for this. All we got is the warning in admin console that says our requests are using high amount CPU quota and will soon run out of it. 3. Doesn't back to normal quickly. I'm using a script to continue fetching my data from production and do some offline process. Unfortunately I fetch too much in one call that I try to get 100 models at a time and which makes the request a high CPU one. Soon, I meet quota deny. Then I stop my script, but after 1.5 hours, I still see quota deny for requests from my users. The Many of the requests to your application are taking a very long time. Please optimize these requests. warning keeps there even I have stopped the script for so long time. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Google app engine django helper, manage.py update, and (1,0,'final')
I was thinking of updating to the latest versions of Django and the helper but will be holding off based on these issues. Can someone respond as to the roadmap for Django 1.0 compatibility? Thanks, Aral On Sep 21, 9:46 pm, Josh [EMAIL PROTECTED] wrote: I believe I was able to solve this problem by syncing my django install to revision 8250'ish. Using that version these obvious errors go away... I just don't get to use the newest (best?) release of django. snip --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Datastore backup and bulk loading
There is a bulk loader but currently no recommended backup solution. I've got one that I need to clean-up and release as open source but I've been sidetracked with other things. I hope to get around to it soon. Aral On Sep 3, 2:48 pm, Ed [EMAIL PROTECTED] wrote: What is the recomended procedure for backing up our data and bulk loading it? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
