[google-appengine] This webpage has a redirect loop
Hello. I'm new to GAE. And somewhat old with web development. I was trying to do a redirect - to a jsp page - from my servlet class but keep getting the error message "The webpage at http://localhost:/my_clock.jsp has resulted in too many redirects". My servlet method simply calls resp.sendRedirect("/my_clock.jsp"). And the jsp file is saved directly under the war/ directory. Spitting out html using PrintWriter's out.println() method however works fine. Please what am I doing wrong? My web.xml looks like this: http://java.sun.com/xml/ns/javaee"; version="2.5"> clock clock.ClockServlet clock /* -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: how to develop blog in the site using python
Bloog is MIT license and cpedialog is open source as well. You can check out all the open source software for App Engine over here: http://groups.google.com/group/google-appengine/web/google-app-engine-open-source-projects I didn't incorporate a forum or rating into Bloog, but it has Tags with a memcached, sharded counter. Have fun, Bill On Aug 26, 9:41 pm, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > Hi Brian, > I really appreciate your time and help.I will take your advice and > look for the Bloog and/or cpedialog.But do i need to pay anything for > using the mentioned services in my site or I can use it as it > is.Thanks in advance for your suggestion > > On Aug 27, 2:15 am, Brian Clapper <[EMAIL PROTECTED]> wrote: > > > On 8/26/08 1:26 PM, [EMAIL PROTECTED] wrote: > > > > I have a requirement of developing blog in my site that includes > > > tagging,discussion forum,rating and voting.Has anybody done something > > > like this using python and google app engine.Thanks in advance for the > > > help > > > See Bloog and/or cpedialog, both of which run on App Engine. > > > http://bloog.billkatz.com/ > > http://code.google.com/p/cpedialog/ > > > Also, I run my blog on GAE, using custom software I wrote myself; I've > > written several articles on how to write blogging software for GAE. See: > > > http://brizzled.clapper.org/id/77 > > http://brizzled.clapper.org/id/78 > > http://brizzled.clapper.org/id/80 > > > I've posted the software I use in those articles on my web site. See the > > articles for links. My software doesn't support voting, rating or comments > > (since I don't want those things right now), but you could certainly add > > them. > > > In your shoes, I'd probably start with Bloog or cpedialog. (I'm partial to > > Bloog's approach, myself. YMMV.) > > > Brian Clapper,http://www.clapper.org/bmc/ > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Accessing a datastore across apps (was "cross app queries")
Being able to access any app datastore from other apps (with some sort of authentication/authorization in place) would be a very powerful feature. It's currently an advantage of Amazon SimpleDB and non- siloed datastores. This would be particularly useful when different languages are supported and a component in one language needs access to an app's datastore. Technically, it seems doable because keys have app and key name components, and all App Engine datastores are in one bigtable anyway. Please star the feature request if you'd like to see this: http://code.google.com/p/googleappengine/issues/detail?id=106 This is a continuation of the thread: http://groups.google.com/group/google-appengine/browse_thread/thread/12eb676e98a25293/f5cfaad4e0d79ac8 (Apparently this thread is closed and only allows replies to authors.) -Bill --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Having doubts about AppEngine
I understand and also worry a little about the long-term roadmap for App Engine as a significant Google offering. AWS has a much larger community and with the addition of persistent storage and sorting of Query results, Amazon is rapidly shoring up their deficiencies. That said, the two services target different audiences since you still have to do a lot of infrastructure management to get something like App Engine on top of AWS. (Or you can pay RightScale or others to do it.) Also, App Engine is significantly younger than AWS. Forum traffic and articles on Amazon EC2 was light at the beginning as well. IMHO, I think a lot of programmers are waiting for pay-as-you-go and possibly https. Beta AWS services start with pricing and charging at launch. EC2/SimpleDB programmers will be more self-selected than Google App Engine developers because you have to know the full stack (from OS on AMI on up). I can also see a lot of small development shops going the App Engine route because (1) free resources up to a certain point and (2) little infrastructure issues. I've been impressed with the Google App Engine team but also wonder how much support they can draw from Google. Jeff Bezos attends AWS events and provides a clear signal that they view AWS as a core offering. I think Yahoo management will do the same if Yahoo launches their version of App Engine. Having stars like GvR on the App Engine team sends a signal as well, but it's different than having top executives talk up the service. On Aug 27, 3:09 pm, javaDinosaur <[EMAIL PROTECTED]> wrote: > I am starting to have doubts about continuing to develop my > applications for GAE. My concerns are not technical although I have a > some anxieties about transaction data propagation performance. > > My concerns center around Google’s commitment to the App Engine > project. Compared to Amazon’s Web Service forums this place feels like > a technical backwater. Developers hosting on Amazon AWS post > interesting questions and get deep-dive replies promptly from Amazon > staff. Amazon is releasing new Cloud development services monthly yet > all we get is minor patches. > > Here on the GAE forum elementary questions about how GAE ticks go > unanswered for months. Basic roadmap type info such as will we get SSL > or scheduled tasks is missing. > > I just feel that the GAE Team is not building up any development > stream in what should be the last 4 month run up to the year-end > release. Communication with the developer community here is abysmal > compared to the investment in developer relations made by companies > such as Microsoft, Redhat or Amazon. > > What’s happened to the early buzz Google? Has the top bass pinched > half the team to firefight problems on another project? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Queries on polymorphic relationships
App Engine handles what Rails calls "polymorphic association". An example: class Rating(db.Model): rating = db.RatingProperty() rated_obj = db.ReferenceProperty(required=True) Since I didn't specify a particular model for the ReferenceProperty, I can use any kind of model. App Engine uses keys. In Rails this would be done by two fields: rated_obj_id --> integer rated_obj_type --> string describing the type or class so Rails knows which table to use In some of my App Engine models, I find myself adding a "type" string property whenever I have a general reference property. I think this is needed to do queries on just certain kinds of references. So I would do this: class Rating(db.Model): rating = db.RatingProperty() rated_obj = db.ReferenceProperty(required=True) rated_obj_kind = db.StringProperty() Now I can easily query for all ratings on a particular kind of class by using a filter("rated_obj_kind =", "Foo"). It seems a bit wasteful since the kind information is already in the reference key. Am I missing a query to get at kind from the keys? Or should I just auto- initialize rated_obj_kind when rated_obj is set? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Having doubts about AppEngine
> Instead of comparing their free preview service to S3, try > comparing it to any other free hosting service out there. While that's a reasonable comparison for hobbyists, there's a lot of us using App Engine with grander aspirations. We want to build easily scaled web applications (real businesses) on top of App Engine. I wouldn't have spent much time with App Engine if it were going to remain just a free hosting service with quotas. If you look at the front page of App Engine, it's clear that Google wants to handle real businesses as well. So for developers looking to build scalable apps that could potentially grow "to millions of users," the best comparison right now is to Amazon's cloud computing effort and the App Engine-like businesses that are being built on it (RightScale, Heroku, AppJet, etc). As a preview user, it's useful to voice your concerns and issues in a reasonable manner. Part of it is through the issue tracker, but this board is a valuable sounding board for developers. I *want* Google to succeed since I'm putting my time and ventures into it. The more effectively they address these concerns, the better they'll look compared to AWS, and that fosters a strong developer community. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Full Feeds for App Engine Cookbook
The cookbook is a great idea. A suggestion: Let users rate an item without entering a comment. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Unable to make comments on http://appengine-cookbook.appspot.com
It's a bit ironic since the cause of that error might be mitigated by one of the submitted recipes: http://appengine-cookbook.appspot.com/recipe/mapping-keys-to-urls/?id=ahJhcHBlbmdpbmUtY29va2Jvb2tylgELEgtSZWNpcGVJbmRleCI8YWhKaGNIQmxibWRwYm1VdFkyOXZhMkp2YjJ0eUZ3c1NDRU5oZEdWbmIzSjVJZ2xFWVhSaGMzUnZjbVVNDAsSBlJlY2lwZSI9YWhKaGNIQmxibWRwYm1VdFkyOXZhMkp2YjJ0eUZ3c1NDRU5oZEdWbmIzSjVJZ2xFWVhSaGMzUnZjbVVNMgw "Might" because using numeric IDs only work if you didn't store the entities using a key name. On Aug 30, 3:22 pm, "noah.gift" <[EMAIL PROTECTED]> wrote: > Traceback (most recent call last): > File "/base/python_lib/versions/1/google/appengine/ext/webapp/ > __init__.py", line 501, in __call__ > handler.post(*groups) > File "/base/data/home/apps/appengine-cookbook/2.4/cookbook.py", line > 373, in post > template_values = memcache.get(key) > File "/base/python_lib/versions/1/google/appengine/api/memcache/ > __init__.py", line 384, in get > request.add_key(_key_string(key)) > File "/base/python_lib/versions/1/google/appengine/api/memcache/ > __init__.py", line 115, in _key_string > 'received %d bytes' % (MAX_KEY_SIZE, len(server_key))) > ValueError: Keys may not be more than 250 bytes in length, received > 282 bytes --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Having doubts about AppEngine
> If I was a member of the marketing staff, I would have pursued a > strategy that ensured that defaults for appengine didn't lock you into > an ultra orthodox view of web development like Django Templates take. > By ultra orthodox I mean handcuffing the templates so you cannot > insert Python code in them. While I agree that claims shouldn't be made for not fully working frameworks, I'd rather the core AppEngine team implement features the rest of the community would have great difficulty implementing. I don't consider templates a difficult piece of the pie. I'm a relative python newbie and if I had a little more time and motivation, I'd create a version of Haml (template system out of Ruby world that uses pythonic indentation for clarity) that allows embedded python. As far as I can tell, there's nothing preventing any of us from creating such a framework because evals aren't sandboxed. I haven't created that template system (yet) because I find Django's templates sufficient. Yes, I run into the python embedding issue sometimes, but there are ways to work around it and Django does provide a nice number of filters and a tailorable template system. There's a difference between things the community can build on the existing framework and things Google probably has to do themselves, like https support, cross-app datastore access, and relaxing the sandbox or securing important packages like full PIL support. Stuff we can't do as a community should take priority. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Having doubts about AppEngine
> > ... if I had a little more time and motivation, I'd > > create a version of Haml (template system out of Ruby world that uses > > pythonic indentation for clarity) that allows embedded python. > > Hmm, the docs look interesting. I would give it a try if you ported > it to Python. It looks pretty cool, plus it is used as the default by > Merb, which seems to be stealing a bit of the Rails thunder I hear. Didn't realize it was the Merb default. There's already a python port (http://lucumr.pocoo.org/cogitations/2008/05/31/the-new-ghrml-haml-for- genshi/) you can try out on App Engine. It requires Genshi. If Django bothers me enough, I might try it out. Still, it feels like it's a port and not fully exploiting the natural fit with the python language. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Datastore write performance question
The small amount of benchmarks I've run show puts to be very expensive, so minimizing them in general will help. Your question, though, is a bit more complex. First, you are increasing the size of your puts. I've not done any benchmarks checking that tradeoff (# of puts vs size of puts). If you increase the size of your models, depending no your application, you might also increase the chance of requiring transactions, which are much more expensive than vanilla puts. I would suggest running a benchmark experiment in the cloud. Use two app versions, one with chunky models, the other with properties distributed. Then time the puts. That would give you a more definitive answer. If I get the time, I might test it out myself. -Bill On Sep 4, 7:46 am, "I.K." <[EMAIL PROTECTED]> wrote: > Nobody? > > Can somebody point me to any relevant blog entries/ posts / documents, > so I can figure it out myself > > On Sep 1, 1:46 pm, "I.K." <[EMAIL PROTECTED]> wrote: > > > Hi all, > > > In the belief that I will improve systems performance, I have been > > reducing the number of Datastore writes by creating a few larger > > models with lots of data in therm, rather than a larger number of > > smaller models. Am I correct in my assumption? > > > This obviously affects my design and code, so I just want to be sure > > I'm making extra work for myself. > > > Thanks --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Parsing clean URLs?
You'll need to handle the regex in main() to route the URL and also the proper parameters of the regex results in your handler. This is exactly how I handle tags in Bloog where URLs are http://.../tag/someTag. Here's the code that pertains to your question: 1) The regex for URL routing: Line 54 on http://github.com/DocSavage/bloog/tree/master/main.py 2) Handling the matched tag name in your handler: Line 477 on http://github.com/DocSavage/bloog/tree/master/handlers/bloog/blog.py Note in TagHandler's get() method how there's the second parameter that captures whatever matched the ...tag/(.*) regex. For every captured group (parenthesized expression) in your regex, you'll need another parameter in your handler methods. -Bill On Sep 6, 7:24 pm, Kevin <[EMAIL PROTECTED]> wrote: > All, > > I thought this would be easy but am having a bit of difficulty. > > I'm trying to do a site where users will have profiles. So I have my > app.yaml pointing to a profile.py like so: > > handlers: > - url: /profiles > script: profiles.py > > The way I want to have the URL for profiles look is as follows: > > http://www.my-app.com/profiles/username > > All I can get to work iswww.my-app.com/profiles > > In profiles.py I have the following code: > > def main(): > application = webapp.WSGIApplication( > > [ > > ('/profile', ProfileHandler), > > ], > debug=True) > wsgiref.handlers.CGIHandler().run(application) > > I've tried changing /profile to /profile(.*) but that doesn't work. > Can anyone help me out? I would prefer not to use POST data for this. > > Thanks!! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Content-type can alter output?
Just noticed this and wondered if it's a security feature. If you do: self.response.headers['Content-Type'] = 'text/javascript' self.response.out.write("foo") On my machine you get: foo This caused a snafu when passing json back and forth between my client and App Engine. Anyone know if this is standard behavior, maybe in the underlying python framework? Did a simple search on the appengine code and didn't find anything where a gets injected. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Content-type can alter output?
Thanks. Looks like a workaround for one browser issue requires another :/ I should've used curl to test the raw output from App Engine. -Bill On Sep 8, 9:21 am, Ross Ridge <[EMAIL PROTECTED]> wrote: > Bill wrote: > > self.response.headers['Content-Type'] = 'text/javascript' > > self.response.out.write("foo") > > > On my machine you get: > > > foo > > It could just be a transformation your browser is doing to display the > JavaScript code. > > Ross Ridge --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Future: App Engine datacenters in the ocean?
This is a fairly cool Google idea: http://blogs.zdnet.com/BTL/?p=9937 Personally, I think my apps would like to be hosted just offshore of Maui. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: A Very Specific AJAX RPC App Engine Question
I've been building some AJAX clients that call App Engine backends. I'm not sure what documentation your citing, but App Engine is just like any server out there. The standard webapp framework nicely handles GET, POST, and other HTTP actions/verbs. You can set the content header in your handler by doing this: self.response.headers['Content-Type'] = 'text/javascript' # or 'application/json' if you think IE won't choke on it self.response.out.write(...some json...) That's it. The hard part on the backend is the code that autoconverts datastore structures into json. In particular, you have to convert datetime and recurse down into lists and dictionaries and convert them to simple json types. On Sep 8, 8:42 pm, Downtown <[EMAIL PROTECTED]> wrote: > Hi, > > I am attempting to write a GAE application using AJAX. It states: > > "In its most generic sense, an AJAX Request will have a function name > followed by any optional parameters, the last of which, if it is a > function, is an optional OnSuccess callback. So, lets write a function > that handles this: > > . > (resulting in ..) > > GEThttp://localhost:8080/rpc?action=Add&arg0=%221%22&arg1=%222%22&time=1... > " > > Where is the "OnSuccess" callback?? It is missing from the GET URL. > Can I assume the following is correct: > > GEThttp://localhost:8080/rpc?action=Add&arg0=%221%22&arg1=%222%22&time=1... > > Also, can I assume on the server side: > > self.response.out.write(simplejson.dumps(result))...will send the > response to myjavascriptfunction?? > > If I am missing something fundamental, please explain. If not, please > update the example to include better documentation on the callback > function. > > P.S. If you help me fine tune your example I will post a Dojo > equivalent that would substantially reduce coding the AJAX GET. In > Dojo it would be as simple as: > > dojo.io.script.get({ > url: "http://localhost:8080/rpc?";, > content: { > action: "Add", > arg0: "1", > arg2: "2", > time: Date(), > OnSuccess: "myjavascriptfunction" > }, > handleAs: "text/json", > preventCache: true, > error: function(text){ > alert("An error has occurred - " + text); > return text; > } > }); > } > > Thanks! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Compilation of unanswered datastore questions
There are some unanswered questions about datastore that I still have and occasionally crop up on the message board. This thread is an attempt to brain dump and outline areas we could use more info. If you've run into questions about datastore and don't see it on this list, please add to the thread. Please don't put questions on this thread that are easily answered by the docs. Instead we want questions that only Google can answer or that we, as a community, can step up and answer ourselves (like benchmarking). Here's my initial brain dump: 1) Will timeout issues on put/transactions be removed when we go pay- as-you-go or should we develop production apps with these limits in mind? Exact # of puts or transactions you can reasonably expect to work within one request before quota issue. 2) Best practices for robustly handling datastore exceptions due to App Engine maintenance/issues with least impact on users, e.g. DeadlineExceededError (increased latency) and CapabilityDisabledError (maintenance shut down of certain datastore functions). 3) Best practices for (de)normalization and entity sizes. A gut reaction some developers might take when approaching datastore is to denormalize and put stuff in fewer tables. What are the costs of keeping many small entities and using reference properties instead? For example, in a many-to-many relationship, we could have 3 Kinds: A, B, and join(A,B). This is just like a traditional relational DB with a join model. What are the costs of traversing implicit collection sets defined by the reference properties in the join Kind? If you have a limited relationship between two entities, when does using a ListProperty (of keys, for example) make sense, especially in light of the cap on indexed properties per entity? 4) Benchmarks! I've been meaning to run tests on costs for different datastore operations: - Direct get using key or id - Direct get using list of key/id - Fetches using filters - Iterative get from a query - How the above 3 (direct w/ key, bulk fetch, iterative get) scale with request size. - Delete/Put - The big hit using transactions Is it a big win to come up with a good key naming scheme or does that bite you in other ways? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Missing Entities in Admin Console Data Viewer
Custom key name shouldn't matter. Does the missing Kind have any entities stored in it? I believe empty Kinds, even if they are defined in index.yaml, won't show up in the console until there are stored entities. On Sep 11, 12:56 pm, Tony Arkles <[EMAIL PROTECTED]> wrote: > Hi! > > Right now my app has 4 different Entity kinds that get stored in the > Datastore. In the Admin Console Data Viewer, I can see 3 of them, but > not the 4th. The only thing that's different (other than the fact > that they have different fields) is that the missing one uses a custom > key_name. > > If that's "expected behaviour", is there any easy way to get at that > data without writing code to do it? > > Cheers > Tony --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Missing Entities in Admin Console Data Viewer
Is this locally or the admin console up in App Engine? I would check your index.yaml and make sure the missing Kind is listed. If local, I'd restart the server. On Sep 11, 1:07 pm, Tony Arkles <[EMAIL PROTECTED]> wrote: > It definitely has Entities in it. If I do a Feed.all(), I get a bunch > of objects back. (I can dump the content back and get what I expect). > It's just a bit concerning that it's missing from the Admin Console. > > On Sep 11, 2:01 pm, Bill <[EMAIL PROTECTED]> wrote: > > > Custom key name shouldn't matter. Does the missing Kind have any > > entities stored in it? I believe empty Kinds, even if they are > > defined in index.yaml, won't show up in the console until there are > > stored entities. > > > On Sep 11, 12:56 pm, Tony Arkles <[EMAIL PROTECTED]> wrote: > > > > Hi! > > > > Right now my app has 4 different Entity kinds that get stored in the > > > Datastore. In the Admin Console Data Viewer, I can see 3 of them, but > > > not the 4th. The only thing that's different (other than the fact > > > that they have different fields) is that the missing one uses a custom > > > key_name. > > > > If that's "expected behaviour", is there any easy way to get at that > > > data without writing code to do it? > > > > Cheers > > > Tony --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Compilation of unanswered datastore questions
> I do not know how this compares to the live version of appengine, but > on the development sdk it was huge. It would probably be magnified on the local SDK which seems to use an inefficient storage scheme. If you load up your local datastore with data, you'll notice your puts get longer and longer. Obviously, the live datastore doesn't have this characteristic. And as bad as puts() are, local transactions are killer. Try doing your 122 puts under transactions :) Enough is enough. I've decided to make and release via open-source a benchmark app. Will start working on it after a late lunch. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: A Very Specific AJAX RPC App Engine Question
You might want to use Crockford's JSON utility instead of the naked eval of the response. See the section "A word of caution against using eval" on the YUI JSON page: http://developer.yahoo.com/yui/json/ The original non-YUI version can be used as well: http://json.org/json2.js On Sep 12, 6:36 am, Downtown <[EMAIL PROTECTED]> wrote: > Thanks for all the posts. I have GET working very well. I had one > additional issue converting string to JSON object which was solved as > follows: > > dojo.xhrGet({ > url: myurl, > handleAs: "json", > load: function(response, ioArgs){ > var myObject = eval('(' + response + > ')'); > myjsfunction(myObject); > }, > error: function(response, ioArgs){ > alert("An error occurred > retrieving the current feed from the > server cache."); > //dojo.byId("toBeReplaced").innerHTML = > "An error occurred, > with response: " + response; > return response; > } > > }); > > On Sep 10, 12:52 am, "Peter Svensson" <[EMAIL PROTECTED]> wrote: > > > That's a good example of how jsonp can look like. Also, I wrote the last > > post with my head screwed on backwards, I think :) To get jsonp, you can > > just use response.out.write('/*'+foo+'*/') where foo is the json string. > > > Cheers, > > PS > > > On Tue, Sep 9, 2008 at 10:18 PM, Davide Rognoni <[EMAIL PROTECTED]>wrote: > > > > JSON-P example: > > > > > > > function foo(json) { > > > alert( json["responseData"]["translatedText"] ); > > > } > > > > > > >
[google-appengine] Re: Question on search.SearchableModel
You could use my modified version of SearchableModel. I blogged about it here: http://billkatz.com/2008/8/A-SearchableModel-for-App-Engine It allows you to declare a class variable unsearchable_properties. -Bill On Sep 17, 7:28 am, Arun Shanker Prasad <[EMAIL PROTECTED]> wrote: > I have a search.SearchableModel, all the String properties are indexed > and can be queried using .search(). Is there any way to make a > particular property in the model not to be searched or indexed. I need > to hide a particular property when the user performs a search on it. > > Thanks, > Arun Shanker Prasad. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: High Amount CPU Quota should be removed
> Imagine the potential consequences if any app could consume as much > resources as it sees fit at any time from that daily quota it has. > Peak load could (and would) go up unpredictably, and that would > potentially cause problems even to apps that are more optimized than > the CPU hogs. Remeber that this is a shared system, so these barriers > are necessary. A main point of App Engine is to let any app potentially consume as much resources as it needs at any time whether it's from a slashdotting or a lower volume of high CPU requests. I look at the High Amount CPU quota warnings as an information service to help tune your app. After pay-as-you-go is released, these warnings should be purely informational IMHO. High CPU requests will cause slow responses, which is a reason to tune those requests, but you shouldn't get an error. I'm hitting the High Amount CPU warnings on a site that's not even close to daily quotas. It's been helpful to see the flags for specific URLs so I can tune them. For example, I've got an AJAX client that loads in JSON data for a specific "workshop" model. My current model can produce, in one request, the JSON for all submissions in all meetings for that workshop, so it goes three models deep in queries. Now that I see the bottleneck, I'm making the data transfer less chunky. So my AJAX client will issue separate GETs for the models, which is one way around High CPU requests -- make the transactions more granular and use AJAX on the client-side to piece together the supplied data. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Compilation of unanswered datastore questions
Here's another question: What's the best practice for handling timeout (and other) db errors? This is partly prompted by ryan's recent comment that "regardless of how your data is shaped, you'll at least see a small timeout rate. .01% is in the ballpark." [http:// groups.google.com/group/google-appengine/browse_thread/thread/ dd90847d0e7c1ceb#] I admit to not checking for timeouts for a single small put. It means that there should be some thought behind all uses of datastore with respect to db.Error. I looked at rietveld source code for some error handling examples but didn't find any interesting patterns. There are some general db.Error catches but no Timeout or CapabilityDisabled exception handling. (By the way, a couple of the last exceptions in datastore_errors.py aren't documented at http://code.google.com/appengine/docs/datastore/exceptions.html. Should documentation issues be logged in the SDK area?) If you receive a timeout, do you need to launch another request from the client because your time is already used up for the current request? I assume that many errors should re-initiate a request (or ask user to resubmit) because the time quota on each request. Is this correct? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: multiple or single trips to the data store (speed and cpu cycles)
Try using fetch instead of treating the GqlQuery object as an iterable. If you had included a LIMIT or OFFSET clause, it would automatically be retrieved by fetch. So it would be something like: users = db.GqlQuery(toquery).fetch(limit=1000) See http://code.google.com/appengine/docs/datastore/queryclass.html You can also cache the query like rietveld. See http://code.google.com/p/rietveld/source/browse/trunk/codereview/models.py and look at the gql method. On Sep 30, 7:17 pm, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > my app is producing significantly high mcycles used results. i have > debugged it and determined that the problem is in my for loop. So I > tried two versions and both are too slow. > > Version 1: > > somestuff = ['cat', 'dog', 'cow']; > > toquery = "SELECT * FROM Animals Where id IN (" + somestuff + ")" > users = db.GqlQuery(toquery) > > now i get a list of users that match that key and I do a for loop like > this: > > for user in users: > self.response.out.write(user.id) > > Gives me a lot of mcycles with a red hazard warning. If I remove the > for loop the hazard goes away. The datastore query takes less then a > second. > > VERSION 2 > > somestuff = ['cat', 'dog', 'cow']; > > for animal in somestuff: > toquery = "SELECT * FROM Moodster Where id = '" + animal + '\'' > for user in users: > self.response.out.write(user.id) > > in the latter version I am doing many queries (in this case 3) but > because there is only one user in users for each query (i make sure of > this), it takes less mcycles and gets a faster response > > AM I DOING SOMETHING WRONG? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Any Site in Production Use ?
I know buddypoke.com is using AppEngine. It's a fairly successful app under heavy load. (I want to say hundreds of requests per second but can't say my memory is correct there.) On Sep 30, 2:21 am, "Feris Thia" <[EMAIL PROTECTED]> wrote: > Hi All, > > I understand that GAE is still in preview used only, but I just wonder if > anyone has used it in any semi or full production ? And with how many page > hits / visitors per day ? > > -- > Thanks & Best Regards, > > Feris Thia --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Any Site in Production Use ?
You're talking about wordle.net, right? Those are pretty successful metrics. A lot of the issues people have been having are on datastore timeouts, particularly on puts. It looks like the main processing for Wordle is handled by a downloaded Java applet, and the datastore really gets used when storing a wordle or returning a gallery page. Do you run into any timeouts or elevated CPU for wordle puts? I'm using Google Chrome and didn't have the Java plugin, so I noticed even your images are created via the java applet so you're not storing image blobs (?). On Oct 1, 5:33 pm, Jonathan Feinberg <[EMAIL PROTECTED]> wrote: > On Sep 30, 5:21 am, "Feris Thia" <[EMAIL PROTECTED]> wrote: > > > I understand that GAE is still in preview used only, but I just wonder if > > anyone has used it in any semi or full production ? And with how many page > > hits / visitors per day ? > > Yesterday was a good day; my app got 89,537 pageviews, according to > Google Analytics. I typically hum along at 10-13 requests per second > during the work day, according to the dashboard. > > The great thing about GAE is that I don't have to worry about being > "slashdotted". If you do what you're supposed to do, App Engine just > shrugs off a few hours of 50 requests per second. It has been a joy. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Any Site in Production Use ?
Feris, Wordle.net is Jonathan's site so I don't have any info other than looking at the web pages. It looks the image thumbnails are image blobs stored in datastore and the java applet displays graphics on individual gallery pages. So he'd have at least one intensive request when doing the thumbnail processing and image put. There's no tagging or join models from what I can see, so his gallery page GET should be reasonably fast. I wonder what the most datastore-intensive and successful app out there might be? "Compare the Candidates" (http:// comparecandidates.appspot.com) looks reasonably complex with the ability to select issues, source, etc, but I think that page might be broken up into many requests via AJAX and the pieces reassembled. Using a rich client (AJAX, Flex, etc.) that can handle server errors/ timeouts and multiple requests per page might be a good way to use App Engine. On Oct 1, 9:26 pm, "Feris Thia" <[EMAIL PROTECTED]> wrote: > Hi Bill, > Thanks for the sharing, for the puts I have tested myself using single > thread and it get cpu over quota easily. So I think it is not ready for > production yet. > > Indeed, wordle is very popular (page rank 7). I'd take a look at how they > handle datastore, or do you have any further info for that ? > > Regards, > > Feris > > On Thu, Oct 2, 2008 at 11:00 AM, Bill <[EMAIL PROTECTED]> wrote: > > > You're talking about wordle.net, right? Those are pretty successful > > metrics. > > > A lot of the issues people have been having are on datastore timeouts, > > particularly on puts. It looks like the main processing for Wordle is > > handled by a downloaded Java applet, and the datastore really gets > > used when storing a wordle or returning a gallery page. Do you run > > into any timeouts or elevated CPU for wordle puts? I'm using Google > > Chrome and didn't have the Java plugin, so I noticed even your images > > are created via the java applet so you're not storing image blobs (?). --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Any Site in Production Use ?
> I get on the order of 100 bogus high CPU warnings per day in my error > log. They tend to come in clusters of 4 or 5 within a few seconds, so > it's clearly some issue on Google's end (these are simple datastore > GETs by key, and they timeout after 10,000 megacycles or so). They've > never been a threat to my quota, so I don't worry about them. How do you handle the datastore get failures? I'm concerned about the .01% failure rate mentioned in another thread, because I'm thinking this is behavior that's not just preview mode but something that should be expected for datastore access. If so, can we issue temporary redirects back to the same URL in the case of failure for idempotent actions? Or are folks just assuming some very small percentage of requests fail? (I'm thinking of handling it on the client side with AJAX or Flash, where the client either informs the user or reissues a request.) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Long key_name design question?
> Then I think your only solution is to do it yourself (the > "tinyurl" service), assuring uniqueness. Basically, you "only" need to > mix datastore simple Gets with hashtable behaviour (taking care and > assuming that two different urls, despite rare, can result in the same > hash key). You need to query by hash key and then, among the results, > look for the matching url. Most of the time (depending on the url->key > algorithm) you should get few results, so overhead won't be a problem. The above assumes that there'll be collisions when you map from urls to the key name. I might naively assume that with a large enough key space (500 characters), you should be able to find a hashing function that'll pretty much guarantee unique keys to be generated. My first question, though, would be if the urls that are greater than 500 characters can be truncated or simply not permitted. When I've seen really long urls on previous apps, they've been attacks by bots or urls with all kinds of state information. You could impose a restriction on submitted URLs and I think a very small % of URLs would trip it. Anyone know stats of valid URLs by length? Since I also plan on implementing a Digg-like function on one of my websites, I'm interested in good hashing functions that let you produce a reasonably sized hash (less than 500 char) efficiently. Seems like the SHA hashes are overkill. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Any Site in Production Use ?
> nope ... it seems is using AWS now! I met BuddyPoke's creator a week or so ago. He was very happy with App Engine and said how great it was to handle his significant load. If you are seeing AWS, it's probably static flash files being served off Amazon S3. Let me know if that's incorrect. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Why Google App Engine is broken and what Google must do to fix it.
Aral, your blog entry clearly outlines some major issues, so thanks for posting it. Here are my thoughts on the issues: 1) 1 MB limit It's a problem if python variables also have this limit. I was not aware of that. I have been using external resources (Scribd and Amazon S3) to store larger items like documents. This is similar to the way Amazon wants you to use their SimpleDB but at least App Engine datastore has more headroom. 2) The 1000 limit offset I typically have created_at and updated_at time fields for each entity. Couldn't you page through all your items by sorting on the created_at property, fetching the first n+1 records, then fetching additional records using >= the timestamp for the n+1th record? Of course, you'd have to put in some code to check for duplicates or use a secondary property as part of the sort. 3) Short-term high CPU quotas Yeah, I'm not sure how these are computed or what might trip it. Hopefully this gets resolved and/or clarified. 4) Quotas in general. I generally agree but think we should cut Google some slack. This is a preview mode. You're a pioneer and you're taking some arrows with your app :) When pay-as-you-go gets initiated, hopefully soon, the major quotas go away with your credit card. As a counter-point, you can read Dave's commentary on how he's scaled BuddyPoke (http:// groups.google.com/group/google-appengine/msg/2449df5e53b5f026). 5) Long-running processes. I agree and the App Engine developers know this is a need. I believe they are working on it. 6) 25% ready for primetime Depends on your app and need for processing across a large number of records. I absolutely, completely, emphatically agree with you that shoring up the deficiencies should be prioritized over adding new languages. Hopefully the development team agrees and isn't swayed by the relative number of stars in the App Engine issues. (Can I remove my star from the Ruby request?) -Bill --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Deploy app into two environments: Staging and Production.
Hello, I have a Laravel app with app.yaml and cloud_build.yaml files. I need to deploy the master branch to production and dev branch to staging. Such that each environment has its own URL. Do I need a separate Cloud Project for each environment? Or I can use the same App Engine to host both the Production and Staging? If I need two separate projects, how can I have two app.yaml and two cloud_build.yaml files? Regards Bilal Haidar -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/8faa970b-1e32-4513-88e1-91d018247e02n%40googlegroups.com.
[google-appengine] Environment Variables inside cloud_build.yaml + PHP
Hi, The environment variables set inside the cloud_build.yaml file, are they read by PHP? I mean, are they treated as system environment variables? I am considering using environment variables inside the cloud_build.yaml file (via substitution) this way I can drop the use of app.yaml and not expose my environment variables inside a file on source control. Thanks Bil -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/0478d804-f7af-4305-8a35-aa9fe8fb1a4fn%40googlegroups.com.
[google-appengine] "The import com.google.apphosting.api.ApiProxy cannot be resolved"
I followed the instructions here: http://code.google.com/appengine/docs/java/howto/unittesting.html and added appengine-api-stubs.jar and appengine-local-runtime.jar to my classpath. Eclipse was able to resolve these imports fine: import com.google.appengine.api.datastore.dev.LocalDatastoreService; import com.google.appengine.tools.development.ApiProxyLocalImpl; It is not able to resolve this import: import com.google.apphosting.api.ApiProxy; Is there another JAR file I need to add to my classpath? - Bill -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: ListProperty substring matching and filtering
I think the answer is in the documentation for ListProperty: http://code.google.com/appengine/docs/python/datastore/typesandpropertyclasses.html#ListProperty In particular: "list_property < value tests if any of the members of the list are less than the given value,..." So your inequality filters will succeed for that entity because one member of the list will be >= searchWord and another member of the list will be < searchWord. As to your question on doing both substring matching + ordering, I'll think about it and post if I come up with something. -Bill On Jul 3, 12:43 pm, cerea wrote: > Hi, > > I'm new to app engine & coding in general, so any help or advice would > be greatly appreciated. > > Background: > I am making a blog and each page has a list of keywords stored in a > db.StringListProperty. Those keywords will be used when someone > searches the site. I also want each keyword to have an integer > associated with it which hold the number of occurrences of that > keyword in the page. > > My Attempt: > I have code which populates a StringListProperty named 'keywords' when > a page is created. It looks like this: ["books_20", "lotto_100", > "wallet_2"], where "books" is the keyword and "20" is the number of > occurrences. My goal was to be able to perform a query to filter out > all pages that starts with a certain keyword, then order it by the > number of occurrences. Here is an example of my query: > > query = BlogPage.all() > # my attempt at substring matching > # based on the tip > here:http://code.google.com/appengine/docs/python/datastore/queriesandinde... > query = query.filter('keywords >= ', searchWord) > query = query.filter('keywords < ', searchWord + u"\ufffd") > query.order('keywords') > > Problem: > The query doesn't work correctly. If I search for the word "cat" even > though it's not in the list, the blog page containing ["books_20", > "lotto_100", "wallet_2"] still gets returned. I'm sure there is a > logic behind the query and what is returned, but I don't understand. > > Questions: > Is there a way to perform substring matching on a StringListProperty > and also have it ordered? And Is there a better approach to what I'm > trying to accomplish? > > Thanks so much for taking the time to read my post. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: ListProperty substring matching and filtering
You could use your approach if each keyword were in a separate child entity where the parent is BlogPage. The child entity would be like the Relation index described by Slatkin in his Google I/O talk, but in your case, you'd have a single StringProperty instead of a StringListProperty so you could do a prefix matching. class BlogKeyword: # Parent is some BlogPage keyword = StringProperty() Your proposed query would be modified to return keys then the parents: query = BlogKeyword.all(keys_only=True) \ .filter('keyword >= ', searchWord) \ .filter('keyword < ', searchWord + u"\ufffd") \ .order('keyword') ordered_post_keys = [key.parent() for key in query.fetch(100)] I don't think the way you've appended the counts to the keywords will work, because ordering will be via string comparison and '3' will evaluate as larger than '20'. So you should pad the counts based on max # of digits for your counts, e.g., book_00020 not book_20. Having a separate entity per keyword seems wasteful, so I'll think about it more. -Bill On Jul 4, 12:50 am, Bill wrote: > I think the answer is in the documentation for > ListProperty:http://code.google.com/appengine/docs/python/datastore/typesandproper... > > In particular: "list_property < value tests if any of the members of > the list are less than the given value,..." > > So your inequality filters will succeed for that entity because one > member of the list will be >= searchWord and another member of the > list will be < searchWord. > > As to your question on doing both substring matching + ordering, I'll > think about it and post if I come up with something. > -Bill > > On Jul 3, 12:43 pm, cerea wrote: > > > > > Hi, > > > I'm new to app engine & coding in general, so any help or advice would > > be greatly appreciated. > > > Background: > > I am making a blog and each page has a list of keywords stored in a > > db.StringListProperty. Those keywords will be used when someone > > searches the site. I also want each keyword to have an integer > > associated with it which hold the number of occurrences of that > > keyword in the page. > > > My Attempt: > > I have code which populates a StringListProperty named 'keywords' when > > a page is created. It looks like this: ["books_20", "lotto_100", > > "wallet_2"], where "books" is the keyword and "20" is the number of > > occurrences. My goal was to be able to perform a query to filter out > > all pages that starts with a certain keyword, then order it by the > > number of occurrences. Here is an example of my query: > > > query = BlogPage.all() > > # my attempt at substring matching > > # based on the tip > > here:http://code.google.com/appengine/docs/python/datastore/queriesandinde... > > query = query.filter('keywords >= ', searchWord) > > query = query.filter('keywords < ', searchWord + u"\ufffd") > > query.order('keywords') > > > Problem: > > The query doesn't work correctly. If I search for the word "cat" even > > though it's not in the list, the blog page containing ["books_20", > > "lotto_100", "wallet_2"] still gets returned. I'm sure there is a > > logic behind the query and what is returned, but I don't understand. > > > Questions: > > Is there a way to perform substring matching on a StringListProperty > > and also have it ordered? And Is there a better approach to what I'm > > trying to accomplish? > > > Thanks so much for taking the time to read my post. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Transactions and sharded counter example code
The reason sharding works to reduce contention is that the each shard is in a separate entity group. Within the transaction, one GeneralCounterShard entity is either retrieved or written. Each shard forms its own entity group, which is what you want so there's no contention on writes. On Jul 13, 4:36 pm, Nathan Gray wrote: > I'm confused. I thought that you could only get/put entities from one > entity group in a transaction, but the example code for sharded > counters doesn't put its shards into a common entity group: > > def increment(name): > """Increment the value for a given sharded counter. > > Parameters: > name - The name of the counter > """ > config = GeneralCounterShardConfig.get_or_insert(name, name=name) > def txn(): > index = random.randint(0, config.num_shards - 1) > shard_name = name + str(index) > counter = GeneralCounterShard.get_by_key_name(shard_name) > if counter is None: > counter = GeneralCounterShard(key_name=shard_name, name=name) > counter.count += 1 > counter.put() > db.run_in_transaction(txn) > memcache.incr(name) > > Note that the GeneralCounterShard(...) constructor call doesn't set > the entity's parent. > > What's up with that? > > Thanks, > -n8 > > --http://n8gray.org --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Non-default app versions: Are they first-class apps?
It's been suggested that a website could use different app versions to support multiple languages or partitioning functions while still getting access to a single datastore. For example, see the last paragraph here (http://stackoverflow.com/questions/1085898/choosing- java-vs-python-on-google-app-engine/1087878#1087878). I was told, though, that non-default app versions could have some throttling or other limitations in place. If so, what are the limitations and is the App Engine team thinking of lifting those limits so all app versions are first-class apps? Thanks, Bill --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Text Search within data store
There are also posts in the language-specific Google groups for App Engine. The search that Barry suggested only covers the posts from the general App Engine group. (This is one problem of segregating all App Engine discussion across multiple language groups.) I've released an open source full text search module: http://billkatz.com/2009/6/Simple-Full-Text-Search-for-App-Engine There's also a system that's under dual-licensing (free/commercial): http://gae-full-text-search.appspot.com/ I think both have improved the situation since SearchableModel was the only alternative. -Bill On Aug 3, 1:18 pm, Barry Hunter wrote: > Try searching the > archiveshttp://groups.google.com/group/google-appengine/search?q=text+search&;... > > there have been a number of implementations released in recent months, > but all are quite limited much like SearchableModel. But if your > requiremens are basic may work for you. > > On 02/08/2009, Filosof wrote: > > > > > I have made an overview of datastore api and have not found the way > > how to perform usual textual search likewww.google.comdoes.. Is it > > possible/planned? Or is there any workaround how to perform quick > > search for entities on the datastore level? > > > Real use case for example. > > Somebody implemented messaging or notification or notes service and > > wants to provide a simple textual search in history.. > > > So I think it would be great feature to mark some property for simple > > text-search.. > > -- > Barry > > -www.nearby.org.uk-www.geograph.org.uk- --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Why Google App Engine is broken and what Google must do to fix it.
I just revisited this thread and noticed someone (probably one individual) gave "1" ratings to every person responding in the thread. Is there a way within Google groups to remove these kinds of individuals? A HackerNews thread discussed destructive people, but this is my first obvious exposure to how it can corrupt the ratings system for a community. Aral does not deserve to have a low rating with the information he's provided so I've counterbalanced with a "5". Something to remember when implementing a rating system for your app. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Any Site in Production Use ?
> def test(): > for i in range(1, 1000): > ferisrecord = Feris(pengarang = "Feris Thia" + str(i), content="Great > app" + str(i)) > ferisrecord.put() Try batching your puts. Right now you are doing serial put() on 1000 instances. I'd try batching those puts and see how much faster group puts can be: def test: batch = 100 num = 1000 / batch for i in xrange(0, num): ferisrecords = [] for j in xrange(0, batch): ferisrecords.append(new_ferisrecord(i, j)) db.put(ferisrecords) This discussion should probably be on a separate thread rather than take-over the original intent. -Bill --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Model.put() Best Practice
> If you check the SDK, I think there is no real difference between > put(entity) or put(entities). > > I don't think put(entities) will be a lot faster. > > Maybe in production, it is different I think in production it is different. See the comment by ryan (google datastore lead) on using multiple entity puts here: http://groups.google.com/group/google-appengine/msg/375e5adb642cd773 I also just discovered this excellent post after a quick search of the group: http://groups.google.com/group/google-appengine/msg/7ed92abad89500c5 It shows benchmarks for batching gets and provides code that you could modify to test puts in the same way. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] A fault-tolerant open source counter
I've extended my implementation of a sharded, memcached counter to allow more graceful handling of failed shard puts because of timeout, capability disabling, etc. The idea is to use memcache as a buffer for failed put data, which gets re-added to the next successful put. Longer writeup here: http://billkatz.com/2008/10/Fault-tolerant-counters-for-App-Engine Code is here: http://github.com/DocSavage/sharded_counter/tree/master/counter.py Let me know if you see holes in this approach. There might be a way to generalize this method to provide a more fault-tolerant datastore layer for more sophisticated models. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Linking directly to the Google Accounts "new account" page + feature request and UX complaint!
> As far as the UI concerns, what do you think about the "new account" > button for the sign-in page onhttp://moderator.appspot.com/?We are > experimenting there, and could make that the default for all > applications. It would be great if Google provided a customizable sign-in page like moderator.appspot.com. Right now, I have a two step process with graphics trying to explain what to do on the standard Google login page: http://workshops.writertopia.com/ You could put the disclaimer "Google is not affiliated with the contents of Writertopia or its owners. If you sign in..." over on the right by the login boxes if we could tailor content on the left part of the sign-in page. I've piggybacked onto an existing feature request for better Google Account login information: http://code.google.com/p/googleappengine/issues/detail?id=713 Star it if you'd like to see this feature. -Bill --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: datastore transaction question
If there aren't a huge number of Things for each User, then your approach sounds workable. A User would have a ListProperty of keys to Things. Your transaction on the User would add the key to a previously created (before the txn) Thing and adjust the money property. You could garbage collect unreferenced Things at a later time. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: datastore transaction question
> Have you seen: "Entity Groups, Ancestors and Paths" > ?http://code.google.com/appengine/docs/datastore/keysandentitygroups.html Good point. If a Thing will never be used for another User (e.g., it's not a product), you could make the Thing part of a User's entity group. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: webapp template inheritance problem
I copied the template code from google.appengine.ext.webapp and modified it to allow a template directory hierarchy under render() and load(). So my template.py has something like this: def render(template_path, template_dict, debug=False, template_dirs=()): """Renders the template at the given path with the given dict of values. Example usage: render("templates/index.html", {"name": "Bret", "values": [1, 2, 3]}) Args: template_path: path to a Django template template_dict: dictionary of values to apply to the template """ t = load(template_path, debug, template_dirs) return t.render(Context(template_dict)) template_cache = {} def load(path, debug=False, template_dirs=()): """Loads the Django template from the given path. It is better to use this function than to construct a Template using the class below because Django requires you to load the template with a method if you want imports and extends to work in the template. """ abspath = os.path.abspath(path) if not debug: template = template_cache.get(abspath, None) else: template = None if not template: directory, file_name = os.path.split(abspath) new_settings = { 'TEMPLATE_DIRS': (directory,) + template_dirs, 'TEMPLATE_DEBUG': debug, 'DEBUG': debug, } old_settings = _swap_settings(new_settings) try: template = django.template.loader.get_template(file_name) finally: _swap_settings(old_settings) if not debug: template_cache[abspath] = template def wrap_render(context, orig_render=template.render): URLNode = django.template.defaulttags.URLNode save_urlnode_render = URLNode.render old_settings = _swap_settings(new_settings) try: URLNode.render = _urlnode_render_replacement return orig_render(context) finally: _swap_settings(old_settings) URLNode.render = save_urlnode_render template.render = wrap_render return template I have a view.py that constructs a template_dirs tuple so I can have a base.html at a root directory and more specific templates for modules in subdirectories. Hope that helps, Bill --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: datastore exception handling examples
> Are there any articles or sample code, showing how to robustly handle > datastore exceptions, for production standard code? David, I recently modified my memcached, sharded counter system to handle various datastore exceptions: http://billkatz.com/2008/10/Fault-tolerant-counters-for-App-Engine You can look at or copy the open source code referenced in that article. It also depends on your app requirements. The above counter shouldn't be used for banking, for example, because if there are simultaneous memcache and datastore errors, you lose pending increments. But if you want something that minimizes the impact of exceptions for things like votes, it's a reasonable approach. -Bill --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Wordpress is supported on the Google App Engine?
No. Wordpress requires PHP, which is not currently supported on App Engine. If you want to use blog software on App Engine, you can checkout a few python alternatives over here: http://groups.google.com/group/google-appengine/web/google-app-engine-open-source-projects None of them will be as full-featured or mature as Wordpress. On Oct 20, 9:36 pm, Gampesh <[EMAIL PROTECTED]> wrote: > HI, > > does Wordpress is supported on the Google App Engine? if so then > please guide me how to do that. > > Thanks > Gampesh --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: "shell" security?
I believe the admin shell is still susceptible to cross-site scripting. (If you are logged in as admin, you could stumble across a hacker-placed script that calls the statement handler.) One way to defeat that would be some kind of token transmitted with a statement request, like a hash of the command salted with a unique site key. On Oct 22, 8:54 am, "Marzia Niccolai" <[EMAIL PROTECTED]> wrote: > If you use shell on your site, you should always restrict it to admin only. > The app has arbitrary access to your application's data. It basically > access as a Python command line interface to your app. > > -Marzia > > On Wed, Oct 22, 2008 at 2:21 AM, jeremy <[EMAIL PROTECTED]> wrote: > > > the shell app @http://shell.appspot.com/ - are there any security > > implications besides allowing users to use arbitrary quota resources > > (the url fetch in particular comes to mind)? > > > for example, could someone use the shell to retrieve someone else's > > session id? i'm looking at the code and it seems like the > > encapsulation of new.module is the extent of the separation between > > sessions. but i'm not sure to what extent new.module's encapsulation > > is hermetic. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Google App Engine Roadmap - Now Published
> * Support for a new runtime language Would you be able to say how different languages will be partitioned? Could you access a single datastore with more than one language or have different languages running on the same VM? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Implement AJAX in App Engine
There are at least two things you'd want to do in your case: 1) Figure out the necessary javascript to periodically issue AJAX calls. This will be dependent on the javascript framework (if any) you chose. 2) Setup a Django template helper to insert that javascript when the App Engine server parses your template file. See this cookbook recipe: http://appengine-cookbook.appspot.com/recipe/using-custom-django-template-helpers/?id=ahJhcHBlbmdpbmUtY29va2Jvb2tyjgELEgtSZWNpcGVJbmRleCI4YWhKaGNIQmxibWRwYm1VdFkyOXZhMkp2YjJ0eUZBc1NDRU5oZEdWbmIzSjVJZ1pFYW1GdVoyOE0MCxIGUmVjaXBlIjlhaEpoY0hCbGJtZHBibVV0WTI5dmEySnZiMnR5RkFzU0NFTmhkR1ZuYjNKNUlnWkVhbUZ1WjI4TTAM A lengthy intro to template helpers is here: http://docs.djangoproject.com/en/dev/howto/custom-template-tags/ And I recently discovered these rails helpers but haven't used them yet: http://code.google.com/p/django-helpers/ -Bill On Oct 12, 1:06 am, Shay Ben Dov <[EMAIL PROTECTED]> wrote: > I wonder is there a way to implement AJAX in an application like the > easy way it is done in Rails: > > you write 1 line in the header to include Javascript, defaults and > other > > then you issue a simple command in your HTML body like: > > > > > <%= periodically_call_remote(:update => 'alerts', > :url => { :controller => :alerts, > :action => :listen_for_requests }, > :frequency => 10 )%> > to issue a call to the server in the background. > > Enabling this kind of capabilities will make life easier and speed-up > development --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Google App Engine Roadmap - Now Published
> Lastly, concerning Bill's comment, I'm not entirely sure what you have in > mind when you ask if they will be running on the same 'VM', immediate plans > for new language support would most likely follow the same general theme as > python support. Meaning, you would upload your application to App Engine, > and the system manages all of serving infrastructure. Not knowing how apps really run in the cloud, I wondered if current python (and possibly future languages) got compiled to some App Engine VM which might allow cross-language calls (like using JVM for JRuby or the .NET CLR). It sounds like this is not the case but can different language files coexist within a single app and access the same datastore? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Microsoft Azure
I like Azure's blobs/tables/queue, like AWS, to handle different sized data. It looks like you can access different datastores from an app. When you get a data service timeout, you get partially completed data and a token to resume the operation. I think that's a good idea. No pricing or free version mentioned in low end for now. On Oct 27, 12:03 pm, jeremy <[EMAIL PROTECTED]> wrote: > I'm only skimming the description but i think the more familiar > relational sql storage will appeal to many people. > > On Oct 27, 11:16 am, "Andrew Badera" <[EMAIL PROTECTED]> wrote: > > > So, what's the GAE take on the MS Azure announcement at PDC today? > > > Is it going to be competitive, or not even in the same ballpark? > > > Will it force the GAE team to spend extra effort on a .NET implementation > > for GAE? > > > Thanks- > > - Andy Badera > > - [EMAIL PROTECTED] > > - (518) 641-1280 > > > -http://higherefficiency.net/ > > -http://changeroundup.com/ > > > -http://flipbitsnotburgers.blogspot.com/ > > -http://andrew.badera.us/ > > > - Google me:http://www.google.com/search?q=andrew+badera --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: counter use pattern
> With the new code, we have a python object rather than a model I can > get_or_insert My modified counter uses a simple python object (rather than a model) at the level above the shards. Shard models are still used so when you change a counter, you'll invoke a get_or_insert style transaction at the shard level. If you look at the other counter implementations, you'll see that a high-level Counter model stores data like counter name and number of shards. Since I handle counter names and # of shards at the program level, hardwired to particular tasks, my version simply bypasses those Counter entities as not necessary. Stat-like tasks like your use case are a good fit for the approach I've taken because counter accuracy (although very likely) is not guaranteed. For example, if you have a string of datastore timeouts and your memcache goes away, the counter will be inaccurate. In cases where it's essential for accurate counts or knowledge that it's failed, simply returning errors and handling it on the client side seems better. But unless I'm misunderstanding your question, you should be able to just always instantiate the Counter object and use it. -Bill --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: counter use pattern
> As I see it, any code that uses your new counter object will need to > deal with instantiating it somewhere ... > others might have a > similar question when wondering how best to actually implement it and > ensure the variable name they wish to use for the counter is not > taken, and that they're not unnecessarily instantiating multiple > Counter objects for no reason. The way I'm using counters, they are hardwired into the code right at the time counter info is needed to be retrieved or set. I do the instantiation right at that point. For my app, I guarantee variable names are OK just via code review and making each group of counters has a special prefix name so there can't be conflict across groups of counters. I'm still relatively new to python web apps, so I can use educating here as well, but my understanding was a request created a limited lifecycle for the request handler and instantiated models, unless you put it in the global space. So if you really want to be efficient, you could create the counters globally so they are kept across requests some times. But even if you are dynamically creating multiple instances of Counter objects, there shouldn't be much of a performance penalty or access issue because these objects are so lightweight and the very nature of a web app where the whole response gets recreated each request. Best, Bill --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Changing overriding the method(HTTP verb) of the request by extending RequestHandler
I made a decorator that checks for _method query strings or POST arguments and redirects to the appropriate request verb. Check out the methods_via_query_allowed decorator here: http://github.com/DocSavage/bloog/tree/master/handlers/restful.py It's use would be like so: import restful ... @restful.methods_via_query_allowed def post(self): pass If the current request has a _method with an overriding http verb, it'll re-route to the appropriate handler. -Bill On Oct 31, 2:32 pm, airportyh <[EMAIL PROTECTED]> wrote: > I want to changed the request verb of a request in the case a > parameter _method is provided, for example if a POST request comes in > with a parameter _method=PUT, I need to change the request to call the > put method of the handler. This is to cope with the way prototype.js > works with verbs like PUT and DELETE(workaround for IE). Here is my > first attempt: > > class MyRequestHandler(webapp.RequestHandler): > def initialize(self, request, response): > m = request.get('_method') > if m: > request.method = m.upper() > webapp.RequestHandler.initialize(self, request, response) > > The problem is, for some reason whenever the redirect is done, the > self.request.params are emptied by the time the handling method(put or > delete) is called, even though they were populated when initialize is > called. Anyone have a clue why this is? As a workaround I thought I > could clone the params at initialize() time, but .copy() did not work, > and I haven't found a way to do that either. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Google Account disabling and potential impact on App Engine apps
I had heard about some legitimate accounts getting disabled by Google because of AdWords or login issues. Today, there's this blog entry that's very troubling: http://blog.mibbit.com/?p=8 Aside from the obvious issues that might come up if we use Google accounts as our primary login system, could account disabling also bring down a Google App Engine app? For example, lets say I use [EMAIL PROTECTED] as my primary account to handle AdWords and be owner for an App Engine app. If there are issues with AdWords (or something else) and my [EMAIL PROTECTED] gets disabled, does that also bring down the associated App Engine app? Or would best practice here be to use another (non-gmail) account to also be administrator for the app so access to and running the app is preserved even if the primary account gets disabled? -Bill --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Google App Engine bloggers?
Jens has a good blog: http://blog.appenginefan.com/ These blogs aren't really updated anymore: http://kupuguy.blogspot.com/ http://appengineguy.com/ The cookbook and articles at the main App Engine site, though, are the best resources IMHO. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: counter use pattern
I probably should've piped up on the thread earlier. I'm currently looking at yejun's fork and will merge pending some questions I have on his optimizations. Here's one: My old code stored the counter name in each shard so I could get all shards with a single fetch. If you have 20 shards, you could have any number of actually created shards. In a very high transaction system, probably all 20 shards exist. In yejun's optimization, he's iterating through each shard using get_by_key_name and checking if it exists. Which is likely to be faster? A nice optimization by yejun is making count a TextProperty. This will prevent indexing and would probably save some cycles. Josh, you said "lines 144-145 should be under the try: on line 137. That way, the delayed counter count won't get reset to zero even in the case of a failed transaction." I thought any datastore errors are handled via the db.Error exception which forces a return before the delayed counter count is reset. (db.Error is defined in appengine/api/datastore_errors.py) On Josh's memcached buffering scheme, I can definitely see the utility if you're willing to sacrifice some reliability (we're down to one point of failure -- memcache) for possibly a lot of speed. Using memcache buffers for counters makes sense because it's easy to accumulate requests while for other model puts, like comments or other text input, the amount of memcache buffer could grow pretty large quickly. Does it make sense to use a sharded backend to a memcache buffer? Depends on the frequency of the final datastore writes, as mentioned above. (I'm not as concerned with complexity as yejun because I think this buffering is reasonably simple for counters.) So I think this would be a good thing to add onto the sharded counter code through a fork. Then people who want that kind of speed can opt for it. -Bill --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: SproutCore app on AppEngine, authentication concerns
Why do you need need two dev servers? From what I understand of SproutCore, you generate a bunch of static files that will be served up by either your local App Engine server (during development) or your production App Engine server in the cloud. Since the REST calls and static SC files are served from the same server, there's no problem with authentication cookies. So I'm not sure why your SC dev server ! = the App Engine server with REST services. On Nov 13, 2:00 am, "Devraj Mukherjee" <[EMAIL PROTECTED]> wrote: > Hi all, > > Our app engine application is going to use SproutCore as its > JavaScript framework. As SproutCore or JavaScript sits quite > independently of App Engine or any server side technology we are > assuming that this should work > (actuallyhttp://sproutcore-demo.appspot.com/confirms that it should not be a > problem). > > Authentication should be quite simple as well, as long as both our > HTML/JS deliver and REST services use Google authentication its should > all work fine. > > The question is while developing the SproutCore application we need to > run two dev servers, so if our rest services required authentication > and we deliver our SproutCore app from the SC dev server how can we > test that everything works? Although the two dev servers are running > at different ports do they still honour the cookies set by the domain > (localhost)? > > Our application is user centric so we will require the user to login. > > Any SproutCore / App Engine developers out there that could give me a > few pointers? > > Thanks for your time. > > -- > "I never look back darling, it distracts from the now", Edna Mode (The > Incredibles) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: ReferenceProperty could avoid unnecesary resolve
I've added the solution to the Cookbook and the FAQ: http://appengine-cookbook.appspot.com/recipe/getting-dbreferenceproperty-key-without-loading-entity On a previous thread, someone came up with a metaclass solution as well. -Bill On Nov 13, 10:01 am, "Chris Marasti-Georg" <[EMAIL PROTECTED]> wrote: > Wooble, that's incorrect. The solution is > here:http://stackoverflow.com/questions/141973/how-do-i-get-the-key-value-... > > On Thu, Nov 13, 2008 at 11:04 AM, Wooble <[EMAIL PROTECTED]> wrote: > > > How would that be more efficient? The entity isn't fetched until you > > try to use it as an instance anyway. > > > On Nov 13, 10:42 am, tigrillo <[EMAIL PROTECTED]> > > wrote: > > > For the sake of efficient, it will be nice if ReferenceProperty let's > > > us return the Key optionally instead of the Entity. > > > > Thanks --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: GAE will hurt Linux unless...
On Dec 3, 12:23 pm, Amir Michail <[EMAIL PROTECTED]> wrote: > The solution of course is for Google to release a user-friendly and > slick Google OS built on top of Linux that makes the web the main > source of apps. Strange you should mention that. A recent article wonders if there's an internal Google OS: http://www.internetnews.com/dev-news/print.php/3788821 About a third of the Google employees were using systems that specifically blocked detection of their browser's OS. Still a big leap to their conclusion, but I wouldn't be surprised if it were true. And I agree with a couple of the other posters that GAE won't hurt Linux use among web developers. A strong GAE platform is very friendly to a variety of web dev OSes, and Windows loses much of its advantages. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Problem adding my own domain , without right to change Cname record
I've used dnsmadeeasy.com for a while and like the service. It's not free, but at $15/year (almost $1/mo), it's very cheap and lets you manage all the DNS settings you mention. There's also some valuable add-ons for a little extra, like mail backup in case your mail port goes down, and failovers. There are some free DNS managers: FreeDNS and see http://hostingfu.com/free-dns I don't use them so can't comment, but google around for reviews. After you get a DNS service, point the domain name server record at your domain name registrar to your new DNS servers. Then modify CNAME like in the Google Apps docs. Good luck, Bill On Dec 8, 12:21 am, Possum <[EMAIL PROTECTED]> wrote: > Hi, guys, > maybe it's not the right place to ask this, but I think there must be > someone have the same problem with me : > > I register a domain name with an Australia company due to the cheap Au > $, but late find that I can't change any DNS info like CNAME by > myself . and it seems a must that if I want to map my app engine > application to my domain name . now what can I do ? > > now the only thing I can change is that under a "domain manager" I can > change the domain name server . it shows a Primary name server and a > secondary name server 's name and it's IP address . this company does > have a DNShosting serivce but compare to the price of domain > name ,it's price is rediculous. I kind of think this is a con. for I > know most domain register give us full access to manage the dns info > of the name . > > I am new to domain name (this might be the second domain name for me). > I just wondering ,is it possible that I use some "free dns service" > that can ensure me to have full acess to all my dns info like cname? > so that I can do the veryfication work for google app engine? and > after I verified it , I change it back (if that free service is not > reliable) to my name register. and everything will keep all right ? > > is there any service like this? and do you think there are problems to > do this?or what do you guys suggest? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: A newbie's site
Nice. I'd give more room to the Google Friend login area because most of it is cut off. Maybe use CSS and float it to right. On Dec 8, 12:04 am, lookon <[EMAIL PROTECTED]> wrote: > I've built a google app engine site based on the simpleviewer example > and FriendFeed Share Bookmarklet. The url ishttp://images.kangye.org/ > > It's a website to let people share images they saw on the Internet and > wish people can give the image a suitable caption. It has a > bookmarklet and in the detail page, people can click the caption to > edit. > > I'm new to GAE and python, and the site need to be improved a > lot...wish you can give me some suggeston~ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Datastore Class db.Model Relationships
Passing the "many" objects through a single db.put is probably the fastest way you can store separate entities. I believe there's still a relatively linear cost for putting some number of entities, so it's slower than doing a single entity put. The performance will depend on how many is "many" objects. If there are just a few, I wouldn't worry about it. If there are many, but the entities are small (like your example or a few strings), you might consider pickling your data if you don't need to get them individually. (There's a 1MB limit on entity sizes.) class One(db.Model): entitydata = db.BlobProperty() import pickle the_one = One() the_one.entitydata = pickle.dumps(some_python_list_or_dict) On Dec 7, 4:43 pm, lawnman <[EMAIL PROTECTED]> wrote: > Perhaps if I create the one-to-many relationships using reference > properties and put all of the models of the "many" objects using a > single db.put then performance may not be an issue. > > Is this a good assumption? > > On Dec 6, 8:58 am, kang <[EMAIL PROTECTED]> wrote: > > > maybe this helps:http://gaesql.appspot.com/ > > > On Sat, Dec 6, 2008 at 9:54 PM, lawnman <[EMAIL PROTECTED]> wrote: > > > > Is it possible to have a class based on db.Model that contains a list > > > of other classes also based on db.Model but where the second classes > > > are not stored in the Datastore separately but rather are stored with > > > the first class. In other words I want a one-to-many relationship but > > > do not want to store the "many's" as separate transactions for > > > performance reasons nor do I want use entity groups because I do not > > > have a need to reference the "many's" outside of referencing the > > > "one". > > > > for example: > > > > class entity(db.Model): > > > name = db.StringProperty() > > > > class one(db.Model): > > > entitylist = db.??? where ??? defines a list of entity. > > > > Thanks, > > > -- > > Stay hungry,Stay foolish. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: A newbie's site
On Dec 8, 1:15 am, kang <[EMAIL PROTECTED]> wrote: > I've noticed this problem...and will adjust it.. > Do you think the FriendConnect should be placed in the right? I'd place it on the right, particularly if it's optional and visitors can drill-down on images without logging in. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: marshal.load()
If this is some fixed data that needs to be searched almost every request, how about breaking it into a lot of chunks (e.g., if it's words, you could have 26 dicts, one for each starting letter) and hard- coding it as a python variable declarations in static files -- one dict per file. That would help you around the 1MB limits on variable and file size, right? When you read in each file as a module, the variable declarations would be automatically cached between requests, which would really decrease the load. On Dec 7, 10:10 am, adrian <[EMAIL PROTECTED]> wrote: > My app needs some fixed data that I need to search during almost every > request to my app. It's over 1 MB as a dict in a python file so I > can't do that. > > So I've been reading a binary picked file but this is taking almost 2 > CPU seconds and 900k function calls according to profile. I get a > CPU usage warning on every request. > > I tried replacing pickle with marshal and it reduced the function > calls to 15k for the same data, and was much faster on the dev > server. So I uploaded it to the production server, but it dies > saying that marshal has no load method. It appears that marshal.load > () is not supported on AppEngine, but if so, why is it available on > the dev server? > > I realize that marshal may change with python version updates, so this > is not an optimal solution, so I'm open for other solutions to try. > > Thanks --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Seemingly undoable to build a simple app
There's a definite need for a full-text search API (particularly substring matches) that just works for the datastore. For public-facing information like your products, you might consider outsourcing the search to something like Google Custom Search: http://www.google.com/coop/cse/ (There's an add-free option if you want to cough up some money.) Nick Johnson recently added Google CSE support to a branch of the Bloog project (http://github.com/Arachnid/bloog/commits/breaking), so you can see how he did it. The CSE approach can be tested over on his blog: http://blog.notdot.net/ -Bill On Dec 7, 1:37 pm, Koen Bok <[EMAIL PROTECTED]> wrote: > First off, I _really_ like the idea of AppEngine. We're hosting most > of our sites (versionsapp.com, checkoutapp.com) on it and it works > great for that purpose. Impressed as we were, I started trying to > build our first serious webapp on AppEngine; a small webstore. What I > basically wanted to do is building a page where you could browse 50 > products at a time, optionally limited by a search or tag, based on > Django. All went well until I wanted to implement search... > > I grabbed a few example datasets from users of our Mac POS app, they > mostly have around a 1000 products with a name, brand and description. > I knew substring search was hard to implement, so I skipped that and > used the undocumented ext.search models with my own set of search > words, mostly around 40 per product. Because some searches took quite > long I decided to slice the results by 50 at a time, not impressive > but good enough. And because product descriptions don't change often I > could use memcache to cache each result 'slice'. > > To do so, I figured I needed to at least order by name, so the slicing > would make sense. Therefore I had to create an index on name, > searchIndex (set of searchable words). Worked great, until I tried to > type two words. I told me I had to create another index: name, > searchIndex, searchIndex. Seemed logical, but then people may want to > type more then 2 words at the same time so I added 5 indexes to get up > to five words. > > From that moment inserting products started to take more then a > second, which I guess was due to the indexing, but even worse, I > started to get errors like BadRequestError: Too many indexed > properties for entity: app. I found out the max indexes for an entity > is capped at 5000, and calculated that if I were to use max 3 words > for a search I could only add 17 searchable words in an items' index > (17*17*17=4913). I'm still not sure that math is 100% correct, but at > least I was able to insert most of my products again, but searching is > just horribly slow. > > So now I'm stuck with a _lot_ of limitations for a simple app: only 3 > search terms, no substring searching, 50 results at a time and in my > opinion pretty bad performance, especially when inserting. Maybe I'm > doing something horribly wrong, and I don't consider myself the best > db programmer out there but this just seems silly. > > Again, this is not to bash AppEngine because I really see the > potential and like the idea. But as it stands now it's just unusable > for us (a small indie dev company that wants their products to scale), > and it's hard for me to imagine that anyone else can live with these > limitations. I guess what I'm saying is: I love this thing to work for > us! We want to use it and pay for it! Seriously, please fix the > searching... you're Google for god sake :-) > > Kindest regards, > > Koen Bok - madebysofa.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: non-users need easy url
Brian, Take a look at this recipe for creating id-based urls: http://appengine-cookbook.appspot.com/recipe/mapping-keys-to-urls Note that the put() returns a key, not the id as your line 93 would suggest. You should also look at some of the open source projects to see how others are doing their apps: http://groups.google.com/group/google-appengine/web/google-app-engine-open-source-projects Good luck, Bill On Dec 13, 2:51 pm, thebrianschott wrote: > Because no one has replied, perhaps I need to say more. > > There are two main pages in this application: > > 1 add_place.html which is directed toward the group "organizers" who > contribute mostly to the "class Group(db.Model)" , and > > 2 map.html which is used later by "individuals" who contribute > mostly to the "class Pins(db.Model)". > > Then, maps.py is the python program which does the magic by taking the > information from add_place.html and creates the information needed by > map.html. > > Currently on line 93 of maps.py you will find the "group.put()" which > is based on a default key ID, not on a key_name, but I think a > key_name is required -- check me on this, please -- in order to create > a simpler url that will direct the ultimate "individual" users to > their own map. I am having no luck implementing the key_name idea and > am looking for help doing that or for help doing this another way. > > 93 id=group.put() > 94 self.redirect("/?place=%s" % id) > > I have experimented with the following lines leading up to line 93, > but then errors occur near line 42 which baffle me. For example, do I > need to add key_name to the Model for group, and how? > > obj = Group.get_by_key_name(str(group.place)) > if not obj: > group = Group(key_name=str(group.place)) > id = group.put > () #notice this is > line 93 above > self.redirect("/?place=%s" % str(group.place)) > > On Dec 13, 10:21 am, thebrianschott wrote: > > > Carpoolfinder.appspot.com has two levels of users: "organizers" who > > must have a google account because they create ridesharing maps, and > > "individuals" who are *not* google account holders and who use one or > > more of the existing ridesharing maps after being informed of an > > existing map by an organizer. To route the individuals to their > > correct map, the organizer must provide his/her users with a distinct > > (unique) url that is easy to type in, so using the map's huge key is > > impractical (but that is as far as we have gotten). > > > How can we make each map easy to use for these individuals? I want the > > distinguishing feature of the url to be very similar to the > > group.place value of the map. Also, I need pretty detailed > > instructions because I am a very new web programmer. > > >http://code.google.com/p/carpoolfinder/ > > > Thanks, > > > Brian in Atlanta --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: non-users need easy url
Brian, That's quite an info dump :) Suggestion -- work your way through the documentation since it's pretty good and give yourself a few hours to test out ideas on the local app server. I think you'll learn a lot by using a Console (http://con.appspot.com/console/help/about). If you have these kind of questions, just interactively use the console. I don't really have the time for a complete analysis, but I think you may be confusing key_name with a key. Another page in the documentation not to be missed: http://code.google.com/appengine/docs/datastore/keysandentitygroups.html It says: "A key has several components: a path describing a parent-child relationship between the entity and another entity, the kind of the entity, and either a name assigned to the entity by the application or a numeric ID assigned by the datastore." The key_name is just one part of a Key object. When you get a string- encoded key, it's the mapping of the Key object data into URL-friendly characters. On Dec 13, 10:48 pm, thebrianschott wrote: > Looking at the rest of that last example, maybe they are saying not > that "susan5" is the key, but that using Key.from_path(,"susan5") can > get the key? > > m = Employee(name="Susan", key_name="susan5") > m.put() > e = Employee(name="Bob", manager=m.key()) > e.put() > > m_key = Key.from_path("Employee", "susan5") #did I miss the > importance of this? > e = Employee(name="Jennifer", manager=m_key) > > Oh, and I see a typo below where I meant key_name, I typed key_word. > > Brian > > On Dec 14, 1:02 am, thebrianschott wrote: > > > I know I am going in circles here, but it sure seems like my earlier > > solution using key_word should have worked and I am worried there is > > some bug. Consider the example at this link which uses the reserved > > word key_word. > > >http://code.google.com/appengine/docs/datastore/typesandpropertyclass... > > m = Employee(name="Susan", key_name="susan5") > > m.put() > > > What is going on? > > > Brian > > > On Dec 14, 12:22 am, thebrianschott wrote: > > > > Wait, I see a flaw: key_name is a reserved word only allowed in the > > > model class. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: 1 application, multiple datastores
It's currently not possible to address multiple datastores. Just looking at the API, it looks like addressing datastores should be possible because the keys include an app name, etc, but the App Engine team has said this feature is not coming any time soon. Cross-app datastore queries complicate the business model when you offer free apps. I think, though, that this is an important feature and should be supported under the pay-as-you-go option, i.e., if you want your datastore to be available cross-app, you elect to forfeit your free quota. Feel free to star this enhancement request: http://code.google.com/p/googleappengine/issues/detail?id=106 -Bill On Dec 19, 10:05 am, GTako wrote: > Hi, is it possible to maintain under 1 application, multiple > datastores that each datastore will be as if it is different app > engine account? > for example: i have a web application that should serve 2 companies, A > and B. I would want to open a google app engine account for the web > application files. the datastores for A and B could be 2 different > deployments under the same app engine account or under seperate > accounts. now assume i have N companies. what should i do? > the reason for seperation is that i dont want the datastores will be > dependent and under same account in case soemthing happens. please > advise. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Running multiple Dev Web Servers on same machine
Alex, I've had no problem running 2 dev web servers on my MacBook Pro (Leopard OS X). What OS are you using and are you launching dev_appserver.py with the port options? You have to give more detailed information. Best, Bill On Jan 8, 2:39 pm, Alex Popescu wrote: > I am trying to run 2 dev web servers on my local machine in order to > test some fetch functionality. Anyways, even if both apps are > configured to use different ports and they are running in 2 different > Python processes, this doesn't seem to work. Is this a known > limitation? Is there any workaround? > > tia, > > ./alex --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: How to determine top 10 scoring users from last 24 hours?
Amir, This is one of those cases that's tough because App Engine's datastore has no on-the-fly SUM ability. You'd have to write an updating process that makes periodic calls to your server app to process recent results and update the entities used for your 24-hour scoring table. Here's how I'd do it: - Record each event affecting scoring in an entity (call the model ScoreEvent) with a timestamp. The key_name of each ScoreEvent would be the timestamp + user info. On write of this entity, update a 24-hr score entity (call the model Score24) for this particular user. - Periodically do maintenance calls to your server to update the Score24 entities. Each maintenance call would sort ScoreEvents on the key and then fetch all ScoreEvents since the last Score24 maintenance call (use a key to mark your place in the ScoreEvents stream). Querying on keys was added to datastore in Nov: http://code.google.com/appengine/docs/datastore/queriesandindexes.html#Queries_on_Keys So for each Score24 maintenance call, you'd fetch some # of ScoreEvent entities, make a first pass so you combine all events into user deltas, then update the associated user Score24 entities. Then save the last ScoreEvent key that you processed in this maintenance call. On the next maintenance call, use that "last ScoreEvent key" in the query just as if you were paging: query = ScoreEvent.gql('WHERE __key__ > :1 ORDER BY __key__', last_key) So your Score24 entities will reflect each user's score over the last 24 hours with some error due to the frequency of your maintenance calls. Now getting the top X scoring users over the last 24 hours is a query on Score24 entities where you sort on score and fetch the top X entities. Best, Bill Amir Michail wrote: > Hi, > > How can this be done efficiently with the current version of the GAE? > > I suppose you could keep track of all events affecting scoring for the > last 24 hours. Whenever you handle a request -- even one that does > not affect scoring -- you could prune out events older than 24 hours > and add a new one if the player has received more points as a result > of the current request. > > As you take out old events and add in new ones, you also update the > top 10 ranking for the last 24 hours accordingly. > > The problem here is that taking out all the old events can be quite > expensive. As an extreme example, if you don't get any requests for > over 24 hours, then on the next request, you will need to clear out at > a day's worth of scoring events. > > Amir --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Creating a short unique URL for an entity
Why worry about risk of collision when you can generate relatively short urls using the Kind string and the id of the entity? Here's one of the first recipes in the cookbook: http://appengine-cookbook.appspot.com/recipe/mapping-keys-to-urls On Jan 11, 5:21 pm, Mahmoud wrote: > Just create a random n-character string for each document. Do that in > put() before the entity is saved for the first time. Choose n based on > what risk of collision (two documents having the same randomly > generated slug) is practical. I say n=6 is pretty good. > > On Jan 10, 10:24 am, Jesse Grosjean wrote: > > > I'm relatively new to web programming, so don't feel back about > > telling me the obvious :) > > > I have a simple model consisting of Account entities that can own many > > document entities. I'd like to create a short as possible, globally > > unique URL scheme that I can use to access any document. For testing > > right now I'm just using the documents .key() in the URL. So for > > example my urls look something like this: > > > /documents/ > > ag13cml0ZXJvb20tY29tchwLEgdBY2NvdW50GH4MCxIIRG9jdW1lbnQYiQEM > > > But that's really long. I'm now trying to figure out the easiest way > > to make it shorter. One option that I'm considering is to just put the > > account id and document id in the URL separated by a dash like this: > > > /documents/126-137 > > > And then I can reconstruct the Key like this: > > > db.Key.from_path('Account', account_key, 'Document', document_key) > > > I haven't actually tried this yet... but I think it should work? > > > My goal is a unique, short, permanent URL for every document in the > > system. If I'm going about this the wrong way or doing something dumb > > please let me know. > > > Thanks, > > Jesse --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Silicon Valley App Engine Developer meetup next Tuesday, Jan 20
The Silicon Valley App Engine Developer meetup is starting off the new year reinvigorated with a nice venue provided by TIPS Group in Palo Alto. Topic: Hacking App Engine (or how to dig into lower level APIs and customize them to your needs) Speaker: Jens Scheffler (author of App Engine Fan blog), http://blog.appenginefan.com/ Also Pete Koomen, Product Manager of App Engine, will talk about new features and provide an update on the platform progress. Location: TIPS Group, 1000 Elwell Ct, Palo Alto, CA 94303 (NOTE LOCATION CHANGE) Time: 7 pm, Tuesday, Jan 20, 2009 Schedule: - Update on Google App Engine - Main Talk on Hacking App Engine - Lightning Talks We'll have some time for prearranged lightning talks, so if you have a demo of something running on App Engine, drop me a line. For more details and to RSVP, visit: http://web.meetup.com/116/calendar/9332524/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Entities question
You could do an entity that has 50 genre counters. Some thoughts: 1) It seems like you can make genre part of a list property in a CD since each CD will only be a small number of genres. 2) I wouldn't use a Reference List for a User entity, because a User might have thousands of CDs. See Rafe Kaplan's article on modeling entity relationships (http://code.google.com/appengine/articles/ modeling.html).I'd model the User <-> CD relationship with a separate Ownership entity that has key references to a CD and User entity. You could add additional info into this relationship model, like whether the Owned CD has been lent to another User. 3) Are genres fixed or user-configurable (like a "tag" instead of a universal genre)? If they are like tags and user-configurable, you could but the genre list in each Ownership entity. Then if User X wants to know how many "1980s Top Hits" he owns, you could query the Ownership entities. You could then memcache the (User, Genre, #CDs for genre) tuple instead of keeping 50 counters in User and just query again if needed. -Bill On Jan 15, 2:23 am, Chen Harel wrote: > Hi, In case I want to represent a music album datastore, > I wish the user can add a cd that he likes (no need for information > other than cd name and genres) > Now, at first I looked at it as a RDBMS even in BigTable, meaning I > had > User entity > CD entity > Genre entity > With list of refrences keys... > Now after playing with it, I realized (at least I think so) that I am > killing my application, > Since I need to be able to supply the aggregations on "how many R&B > cd's user XXX has" > this query can't be optimized (at least with my current knowledge) on > BigTable, > So I need to start putting up counters... but I am questioning you > about the number of properties allowed (or make sense) > If I support 50 genres, can I simply do an entity that has: > User: > id > name > cds (list of refrences) > total_cd_counter > trance_cd_counter > house_cd_counter > rnb_cd_counter > .. (47 more counters) > > And when I add / remove a cd for a user, I shall update the > appropriate counter ... > This will make query for the favorite genre (which is the main goal :) > - Sorry I got to it only here) of the application... > > Please tell me if this will work, and will have sufficient > performance / scaling capabilities... --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: WYSIWYG Editor for App Engine
I would agree with Alexander that solid HTML/CSS knowledge is important, particularly if you are using another person's template and trying to add your own elements. CSSEdit has some good reviews for Mac HTML/CSS tweaking. I mostly use TextMate and Firefox. Adobe Dreamweaver does WYSIWYG HTML design and I believe there are some server-side plugins, but probably not for anything App Engine yet. On Jan 15, 1:30 am, Alexander Kojevnikov wrote: > The thing is, there's no substitute for a solid html/css knowledge. No > visual html editor, and there are lots of them floating around, will > produce a consistently valid and cross-browser compatible mark-up. > > Your best bet is to learn the basics, or to hire someone else to do > the job. > > On Jan 15, 6:05 pm, Ashu wrote: > > > Ok, so I am writing my application in > > 1) Python > > 2) Using the Default WebApp Framework (app engine). > > 3) Initially used Notepad++ (also configured Eclipse IDE) as the > > Editor > > > I also downloaded a free html template from the web and currently > > using that to display my data/content. However, I also need to > > customize some of the pages to take user input(aka add HTML Form > > elements). I did add them up, but it kind of screws up the look and > > feel of the page. I am still trying to work with CSS/HTML. So, I was > > wondering if there was a Visual Tool/plugin that I could use to design > > my pages by drag/drop and then connect it with my Python code. > > > On Jan 13, 3:54 am, Roberto Saccon wrote: > > > > there are tools and plugins for everything available on the Internet. > > > Make your question more precise. Is the editor for natural language or > > > programming language (which one) ? If the later is the case, I > > > recommend CodeMirror, it has pluggable parsers, is framework > > > independent and if you look at the source code, it is one of the most > > > amazing pieces of JavaScript I have seen in the last years (and it is > > > also used at Google, but I forgot which project) > > > > regards > > > Roberto > > > > On Jan 13, 7:21 am, Ashu wrote: > > > > > I have coded up the necessary logic, but now I am stuck trying to come > > > > with a descent looking Form to take input from the user. Are there any > > > > plugins or tools available to get the HTML part coded with ease. > > > > > Thanks. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Entities question
> I'm not sure I've followed you with whole Ownership model.. > Isn't that a RDBMS approach to the data and not BigTable's? The two approaches aren't mutually exclusive. In the case of User <-> CD, a User can have thousands or more CDs and a CD could be purchased by millions. Under those circumstances, storing the ownership relationship in a list of references under either User or CD entities will really impact performance. In Rafe Kaplan's article he says: "Another more important one is that you want to avoid storing overly large lists of keys in a ListProperty." The straightforward alternative is a separate Ownership model and now there's a separate issue of constructing a transaction that handles updating your genre counter upon a successful Ownership change. It'll be interesting if someone comes up with another approach, maybe using multiple User entities (under one entity group) that all map to a single user, with each User entity holding a subset of all the CDs and genre counters. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Entities question
Chen, Some differences between using an Ownership model and using a list of references to CDs in a User entity: 1) You can store info associated with Ownership as mentioned in my previous message. So if you want to print all the CD titles owned by a user, you could denormalize CD titles and duplicate that information in Ownership entities. Then, you do a query on Ownership entities filtering for a specific User and the titles would be fetched. In the list of references, you'd have to actually retrieve each CD title or store titles in the single User entity, which leads us to #2... 2) There's a cap on number of indexed properties. See http://groups.google.com/group/google-appengine/browse_thread/thread/d5f4dcb7d00ed4c6 I haven't tested this yet, but the cap on indexed properties will really hamper your ability to have huge reference lists within a User entity. However, you could pickle/serialize your CD data into a blob or text property which aren't indexed. If you start storing the CD lists as blobs, though, you lose the ability to do queries like "Show me all users that own CD X" On Jan 16, 11:05 pm, Chen Harel wrote: > Bill if I use an ownership model, > How do you fetch all the cds of a user? > you select on Ownership, then get the tuples with user id / cd id > Now you need to fetch many cd id from the CD entity.. > So what's the difference? > If you store them as a list of IDs in the User, you still have to go > over the CD and fetch for every ID, > This still doesn't allow you to properly index your data, does it? > > On Jan 16, 8:14 pm, Bill wrote: > > > > I'm not sure I've followed you with whole Ownership model.. > > > Isn't that a RDBMS approach to the data and not BigTable's? > > > The two approaches aren't mutually exclusive. In the case of User <-> > > CD, a User can have thousands or more CDs and a CD could be purchased > > by millions. Under those circumstances, storing the ownership > > relationship in a list of references under either User or CD entities > > will really impact performance. > > > In Rafe Kaplan's article he says: > > "Another more important one is that you want to avoid storing overly > > large lists of keys in a ListProperty." > > > The straightforward alternative is a separate Ownership model and now > > there's a separate issue of constructing a transaction that handles > > updating your genre counter upon a successful Ownership change. > > > It'll be interesting if someone comes up with another approach, maybe > > using multiple User entities (under one entity group) that all map to > > a single user, with each User entity holding a subset of all the CDs > > and genre counters. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: index.yaml not being updated
Just to make sure, you have forced the local dev server to execute a number of queries that require indices, right? On Jan 18, 6:50 am, Dave wrote: > Hi, In chasing a query issue, i've noticed my index.yaml file is not > being updated at all. I've tried to delete it and restart server ($ > python manage.py runserver) and all that happens is a new index.yaml > is created as below. Note there is nothing about indexes in it? I have > a previous app where the index.yaml file was updated and I could see > all the autogenerated index info. Any ideas? btw, on mac 10.4.11. Also > I've search all over for a 're-index' command for manage.py but so far > have not found a thing. > > thx... > > indexes: > > # AUTOGENERATED > > # This index.yaml is automatically updated whenever the dev_appserver > # detects that a new type of query is run. If you want to manage the > # index.yaml file manually, remove the above marker line (the line > # saying "# AUTOGENERATED"). If you want to manage some indexes > # manually, move them above the marker line. The index.yaml file is > # automatically uploaded to the admin console when you next deploy > # your application using appcfg.py. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: index.yaml not being updated
Yes, that was what I was getting at. Dave, take a look at Ryan's article on index building: http://code.google.com/appengine/articles/index_building.html Your index.yaml won't be updated unless your queries require developer- defined or composite indexes. When you do have more complex queries, you can manually insert the required indexes or make sure you execute them on your dev server. On Jan 18, 2:14 pm, djidjadji wrote: > These simple queries don't need entries in the index.yaml file. > For single attribute queries there are implicit indices constructed. > Even is you filter multiple times for equality you don't need an > index.yaml entry, and maybe then only when you want a DESC sorting. > > If the production server gives the correct result without raising a > NeedIndexError you use some implicit index. > > 2009/1/18 Dave : > > > request.user.invites_in.filter('invited_person =',request.user) > > request.user.invites_out.filter('person_inviting =',request.user) > > user.connections.filter('user =',request.user) > > Dave --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Entities question
By denormalize, I mean storing the CD titles in the Ownership entities as well. Yep, there'll be duplication to avoid the join, but you'll be able to get the CD titles from the fetched Ownership entities in one query. I'm still trying to figure out best practices myself with the datastore. When joins are expensive or impossible, you can turn to denormalization but just how you do it depends on the queries you expect to run (and optimize for). Here's an interesting article that has some relevant links: http://www.codinghorror.com/blog/archives/001152.html You can see your initial User entity with all the counters was an intuitive use of denormalization. The only problem is the added constraint of indexed properties caps. On Jan 18, 12:07 pm, Chen Harel wrote: > Ok, If I go with number 1, I'm not sure how can I denormalize it to > print the CD titles... > If I can't "join" on the CD titles them selves... How can I fetch the > names of the CDs if the select is > SELECT * FROM Ownership WHERE user_id = ? > > Bill, 10x a lot for your help... > > On Jan 17, 10:13 am, Bill wrote: > > > Chen, > > > Some differences between using an Ownership model and using a list of > > references to CDs in a User entity: > > > 1) You can store info associated with Ownership as mentioned in my > > previous message. So if you want to print all the CD titles owned by > > a user, you could denormalize CD titles and duplicate that information > > in Ownership entities. Then, you do a query on Ownership entities > > filtering for a specific User and the titles would be fetched. In the > > list of references, you'd have to actually retrieve each CD title or > > store titles in the single User entity, which leads us to #2... > > > 2) There's a cap on number of indexed properties. > > Seehttp://groups.google.com/group/google-appengine/browse_thread/thread/... > > > I haven't tested this yet, but the cap on indexed properties will > > really hamper your ability to have huge reference lists within a User > > entity. However, you could pickle/serialize your CD data into a blob > > or text property which aren't indexed. > > > If you start storing the CD lists as blobs, though, you lose the > > ability to do queries like "Show me all users that own CD X" > > > On Jan 16, 11:05 pm, Chen Harel wrote: > > > > Bill if I use an ownership model, > > > How do you fetch all the cds of a user? > > > you select on Ownership, then get the tuples with user id / cd id > > > Now you need to fetch many cd id from the CD entity.. > > > So what's the difference? > > > If you store them as a list of IDs in the User, you still have to go > > > over the CD and fetch for every ID, > > > This still doesn't allow you to properly index your data, does it? > > > > On Jan 16, 8:14 pm, Bill wrote: > > > > > > I'm not sure I've followed you with whole Ownership model.. > > > > > Isn't that a RDBMS approach to the data and not BigTable's? > > > > > The two approaches aren't mutually exclusive. In the case of User <-> > > > > CD, a User can have thousands or more CDs and a CD could be purchased > > > > by millions. Under those circumstances, storing the ownership > > > > relationship in a list of references under either User or CD entities > > > > will really impact performance. > > > > > In Rafe Kaplan's article he says: > > > > "Another more important one is that you want to avoid storing overly > > > > large lists of keys in a ListProperty." > > > > > The straightforward alternative is a separate Ownership model and now > > > > there's a separate issue of constructing a transaction that handles > > > > updating your genre counter upon a successful Ownership change. > > > > > It'll be interesting if someone comes up with another approach, maybe > > > > using multiple User entities (under one entity group) that all map to > > > > a single user, with each User entity holding a subset of all the CDs > > > > and genre counters. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Silicon Valley App Engine meetup *tonight*
Topic: Hacking App Engine (or how to dig into lower level APIs and customize them to your needs) Speaker: Jens Scheffler (author of App Engine Fan blog), http://blog.appenginefan.com/ Also Pete Koomen, Product Manager of App Engine, will talk about new features and provide an update on the platform progress. RSVP here: http://web.meetup.com/116/calendar/9332524/ Location: TIPS Group, 1000 Elwell Ct, Palo Alto, CA 94303 (NOTE LOCATION CHANGE) Time: 7 pm, Tuesday, Jan 20. Schedule: - Update on Google App Engine - Main Talk on Hacking App Engine - Lightning Talks (at least 3 good ones are slated) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Model.put() Override?
The most recent SDK (1.1.8) now allows hooks, and that seems to be the best way to hack datastore puts. Jens (http://blog.appenginefan.com/) has started to blog about use of the hook feature and gave two examples at our local App Engine meetup tonight. Stay tuned to his blog for a good explanation. In the meantime, look at appengine.api.apiproxy_stub_map.py and you'll see some hook methods. You can register methods to run pre- or post- put. If you supply a hook at this level of the API, you can use your hack even when bypassing the Model API. On Jan 17, 11:15 am, James Ashley wrote: > The "simple" answer seems to be "override the db class." That's > totally off the top of my head, and it may be completely unrealistic. > > Something like (totally untested and almost definitely wrong. Consider > it pseudo-code): > > class my_db(google.ext.db): > def put(self, *objects, **kwargs): > for o in objects: > o.pre_put() > super(google.ext.db, self).put(*objects, **kwargs) > > Then the models that were overriding put() could put that override > code in pre_put() instead (and change put() to just call that before > calling super().put()...). > > You'd have to juggle the nuances of things like post_put(), and any > side-effects. But it seems like a reasonable compromise. > > On Jan 14, 2:22 pm, boson wrote: > > > That is a curious situation. I would like to know also if there is a > > way to inject code into the put-stream regardless of method called, as > > I have various put() overrides as well. > > > On Jan 13, 9:45 pm, Devel63 wrote: > > > > I overrode the put method on a class derived from Model, primarily > > > because I wanted to do some accounting work when a new object was > > > saved (in the simplest case, just keeping track of a counter). It > > > does its work, then calls super. > > > > It works fine. > > > > Until I use db.put([ob1,obj2,obj3]) to save a bunch of these, and then > > > of course the individual object.put() functions are never called. > > > > Is there a best practice for this? I really liked having it in the > > > "put", because then no caller needed to know about it, and it just > > > took care of itself. Is the best thing to precede every "put" call > > > with an explicit "put_prep" call? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Site map placement
Does anyone know how to place a site map or robots.txt so that it is visible on your site's "root" directory in GAE? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Site map placement
Thanks Barry, the second link had the syntax I needed! On Jan 22, 1:10 pm, Barry Hunter wrote: > Seehttp://code.google.com/appengine/docs/python/gettingstarted/staticfil... > andhttp://code.google.com/appengine/kb/general.html#erroruris > for an example > > 2009/1/22 Bill : > > > > > Does anyone know how to place a site map or robots.txt so that it is > > visible on your site's "root" directory in GAE? > > -- > Barry > > -www.nearby.org.uk-www.geograph.org.uk- --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Google's Source Control Vision
I use git for controlling my App Engine source code. ( For example, I released my early attempt at a blog over at http://github.com/DocSavage/bloog ) When you deploy on App Engine, you can set the version ID of your code. I've created a shell script which automatically generates the git version hash and puts it in the app.yaml "version" key/value. App Engine will let you use any alphanumeric string for versions, so git hashes are OK. So your team can use whatever source control system for the code (git is great for distributed teams), and when you deploy, you have multiple versions of your app in the cloud ready to run. Each version's ID tells you the exact repository state. The App Engine server console lets you easily switch the app version used for your main url, and it even lets you test each version separately through longer urls at appspot.com. The system is pretty nice. -Bill On Jan 22, 5:33 pm, GenghisOne wrote: > Does the current version of Google's App Engine vision address > distributed software development and source control? > > Or put more practically, how can a couple of distributed software > developers collaborate on a Google App? For instance, how can I pull > down the latest source and update it? > > Thx. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: hiding private key
Brian, If you plan on storing the key in a file, I would suggest using the svn:ignore feature: http://svnbook.red-bean.com/en/1.5/svn-book.html#svn.advanced.props.special.ignore You can tell svn to not check in any file that matches your private filename. Just let your collaborators know they're supposed to add their version of the private file after svn checkout. -Bill On Jan 23, 8:21 pm, thebrianschott wrote: > Dave, > > Thank you for clarifying that. I guess I would have to put the key > into the datastore, but that may not be practical for me either. I > have not decided yet how much access to the datastore I will give > users of my app, yet. I don't think they need any, but I cannot be > sure now. > > In any case, thanks again for sticking with this discussion. > > On Jan 23, 6:51 pm, David Symonds wrote: > > > On Sat, Jan 24, 2009 at 7:41 AM, thebrianschott > > wrote: > > > I think I can save my secretkeyin a static file that is not "add"ed > > > to the svn and I was told by David that everything (source, etc) is > > > secreat that is committed to the appspot. So as long as the static > > > file is not "add"ed to the svn, it's secure, isn't it? > > > If it's in a static place (i.e. you use static_files or static_dir > > directives in app.yaml), it'll be accessible by any web user. Nothing > > else is directly accessible, though, so you'd be fine. > > > Dave. > > Brian in Atlanta --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: hiding private key
I'm not sure what you mean, so I'll simply restate what I believe is the situation and how I would handle it. You need to put some "private" info into a .py file that will be part of your App Engine app, similar to a config.py. When your app runs, it will need access to the private code in this .py file, which can be done by a simple python import. The problem: You also want to make the source files for your app available in a public svn repository. The solution: On your local machine, you put the private key into some file, call it private.py. Your main.py or some other publicly- released code will import private.py. You want this private.py to be on your local machine and in the App Engine cloud, but NOT in the public svn repository. So, all you have to do is set the svn:ignore property to ignore private.py. This means private.py is not under revision control and won't be committed to the public svn repository. The private.py file will be on your local machine and when you "appcfg.py update", the script will see private.py and put it in the cloud as well. Just make sure you keep a backup of the code separate from the svn repository (or keep private.py a one-liner), because if your local machine's disk crashes, you won't have a backup in the public svn repo for private.py. -Bill On Jan 24, 6:25 am, thebrianschott wrote: > Bill, > > So, you are not suggesting that Dave's recommendation to NOT put my > secret file in the material committed to the appspot, because it would > be available to anyone on the web, right? But you are saying that if I > learn to write a "little script" that pulls my secret key [see > the quote from Dave, below], then the ignore feature would work nicely > for the little script and the tiny.py file Dave mentions. And I agree > that the ignore would be dandy and I appreciate your jumping in here. > I sure wish such a little script and tiny .py file example existed for > me to adapt, because I'm very shaky on python programming, especially > creating (,linking) and editing existing editable source files. > > [Below is a quote from Dave's earlier message in the thread] > Okay, I see where you're coming from. Your best bet is probably to > auto-generate or manually edit a tiny .py file that you don't check > in > to SVN (or maybe check in a secret-key.py.template with the key > replaced with "InsertKeyHere"). Then you could write a little script > that pulls your real secret key from a file on your own computer > before uploading your app. > [Above is a quote from Dave's earlier message in the thread] > > On Jan 24, 1:35 am, Bill wrote: > > > > > Brian, > > > If you plan on storing thekeyin a file, I would suggest using the > > svn:ignore > > feature:http://svnbook.red-bean.com/en/1.5/svn-book.html#svn.advanced.props.s... > > > You can tell svn to not check in any file that matches yourprivate > > filename. Just let your collaborators know they're supposed to add > > their version of theprivatefile after svn checkout. > > > -Bill > > > On Jan 23, 8:21 pm, thebrianschott wrote: > > > > Dave, > > > > Thank you for clarifying that. I guess I would have to put thekey > > > into the datastore, but that may not be practical for me either. I > > > have not decided yet how much access to the datastore I will give > > > users of my app, yet. I don't think they need any, but I cannot be > > > sure now. > > > > In any case, thanks again for sticking with this discussion. > > Brian in Atlanta --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Dynamically Choosing Which Properties to Write to the Datastore
Hi David, On Jan 24, 8:39 am, David Kamenetz wrote: > However, if the user only enters/changes, say, the txt field on the > form I only POST the txt field to the server. I don't send all the > fields. My POST only has data for txt. If I use the code above on an > existing entity, it will erase the title property (formitem.title). > > Is there an elegant way to write only the txt element to the txt > property? As far as I know, you can't write only one property when replacing an existing entity, but maybe there's something that can be done at the lowest levels just before going to the protocol buffer. (See datastore.py and datastore_pb.py in the SDK where entity dicts are being passed.) I use POSTs to create new entities and PUTs to modify existing ones or store a new entity in a known url. With the PUTs, if there's an existing entity, you need to read it in and then selectively modify key/values depending on what's been passed from your form. So creating new entities through POST is simple. Just set whatever property you want to set on model initialization, and all others are not stored. You can automate the whole process by creating a Model you'll inherit from (I call mine SerializableModel), and create a method that iterates through all model properties, calls request.get() on them, and if the get isn't None (i.e., there's a value passed in from your form), you add that key/value to your entity dict. You later pass the entity dict as an initializer into your Model constructor. For example: def get_entity_dict(model_class, get_func): entity_dict = {} for prop_name, prop_class in model_class.properties().iteritems(): value = get_func(prop_name) if value: entity_dict[prop_name] = model_class.deserialize (prop_class, value) return entity_dict So in the above, get_func is set to: get_func = lambda x : some_handler.request.get(x, default_value=None) We only set key/value pairs in entity_dict for properties that are set in form. The model_class.deserialize() is a routine that takes strings from the form and converts them into appropriate datastore objects. So in your handler you'd have something like this in simplified form: def post(self): get_func = lambda x : some_handler.request.get(x, default_value=None) props = get_entity_dict(MyModel, get_func) obj = MyModel(**props) obj.put() For the PUT case, you read the entity first, set the entity_dict to the current entity values, and then do the above. I might open source the model system I've created that does all this stuff. -Bill --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Unable to access my admin console, Yes i have read the Docs and FAQ ;)
If you go to http://appengine.google.com/start after logging in as the administrator of your app, you should see a "My Applications" page with a list of your apps. Each app name should have a link. Clicking on the link will bring up the admin console for your app. The direct link to the console looks something like this: http://appengine.google.com/dashboard?&app_id= On Feb 2, 1:44 am, V1 wrote: > Hello, > > I have successfully deployed my first app Enginehttp://spry-it.appspot.com/. > When I go to:http://appengine.google.com/it automatically redirects > me tohttp://appengine.google.com/startand all i see is that i can > create a app Engine.. I already done that ;) > > I also checkedhttp://appengine.google.com/a/your-domain.com > > because I registered with an email address that is linked to my google > apps. But that just redirects to a dead page. :) > > Am I doing something wrong here? > > Also pushing on the GoogleAppEngine SDK dashboard button redirects me > to the start page where i can only create apps, not view. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: SearchableModel problems
The exception is doing a good job explaining what SearchableEntity wants. For each number of keywords you'll accept during a search, you'll have to have an index with that many "name: __searchable_text_index" And you could easily start hitting the maximum indexed properties limit of 5000: http://groups.google.com/group/google-appengine/browse_thread/thread/d5f4dcb7d00ed4c6 See here for exploding indexes: http://code.google.com/appengine/docs/python/datastore/queriesandindexes.html So full-text search is not very good right now with SearchableEntity. I think Google Custom Search API is a better way to go if you can. -Bill On Feb 6, 4:41 pm, Ilia Lobsanov wrote: > I have entities (model name is Contract) in the datastore created > using search.SearchableEntity. > > This is how I'm searching these entities: > > query = Contract.all().search(keyword) > query.order("-contract_date").fetch(10) > > When the keyword variable is one word, no exception is raised. If I > use two words for the keyword, the following exception is raised. If I > use three words, the exception will talk about wanting three indexes > on __searchable_text_index > > Exception in request: > Traceback (most recent call last): > File "/base/data/home/apps/govtenders/123.331195053078948310/common/ > zip-packages/django.zip/django/core/handlers/base.py", line 86, in > get_response > response = callback(request, *callback_args, **callback_kwargs) > File "/base/data/home/apps/govtenders/123.331195053078948310/goat/ > views.py", line 72, in search_by_keyword > more_template_params = _process_query(request, query) > File "/base/data/home/apps/govtenders/123.331195053078948310/goat/ > views.py", line 139, in _process_query > results = query.order("-contract_date").fetch(PAGESIZE+1); > File "/base/python_lib/versions/1/google/appengine/ext/db/ > __init__.py", line 1390, in fetch > raw = self._get_query().Get(limit, offset) > File "/base/python_lib/versions/1/google/appengine/api/ > datastore.py", line 942, in Get > return self._Run(limit, offset)._Next(limit) > File "/base/python_lib/versions/1/google/appengine/api/ > datastore.py", line 891, in _Run > str(exc) + '\nThis query needs this index:\n' + yaml) > NeedIndexError: no matching index found. > This query needs this index: > - kind: Contract > properties: > - name: __searchable_text_index > - name: __searchable_text_index > - name: contract_date > direction: desc --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: SDK version 1.1.9 Released
Buried in the change log is an innocuous line about the remote API. >From the module: "This allows easy remote access to the App Engine datastore, and potentially any of the other App Engine APIs, using the same interface you use when accessing the service locally." On Feb 9, 6:12 pm, Jeff S wrote: > Today we released version 1.1.9 of our SDK. Here's what's new in this > release: > > * You can now use the Python standard libraries urllib, urllib2 > or httplib to make HTTP requests. This has been a frequent request on > our issue tracker. > * We've been working on a set of tools that will make the > process of uploading and downloading data from App Engine applications > easier. Today we're excited to announce an early release of our new > bulk uploading client. You can try it out here. Let us know what you > think in our Google Group! > * Several updates to our datastore, including the automatic > generation of single property indexes and the addition of IN and != > operators to db.Query. See the Datastore API docs for more details. > * A bunch of additional bugfixes and enhancements, listed in our > Release Notes. > > See the blog > post:http://googleappengine.blogspot.com/2009/02/sdk-version-119-released > > and the downloads are available > here:http://code.google.com/appengine/downloads.html > > Happy coding, > > Jeff --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: How do I "select * from entity where 'foreignkey' = blah"
I don't understand why you are modeling it that way. Why have the ReferenceProperty in Location? Given your case of wanting all people with a given birthplace, it seems like this makes sense: class Person(db.Model): name = db.StringProperty() birthplace = db.ReferenceProperty(Location, default="unknown", collection_name='born_here') class Location(db.Model): place = db.StringProperty() Since a person is likely only born in one place, you put the reference with the Person. Now you can do: birthplace = my_person.birthplace# gets Location and some_location.born_here # list of Person born here On Feb 9, 9:27 pm, adelevie wrote: > I know the datastore is not relational but this should still be > simple. > > I have two models: > > class Person(db.Model): > name = db.StringProperty() > > class Location(db.Model): > birthplace = db.StringProperty() > name = db.ReferenceProperty() > > So I want to be able to select all People given a birthplace. I tried > messing with keys, but to no avail. > > thanks, > Alan --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: How to use remote_api?
I've written a blog entry describing how I've gotten the remote_api to work for me: http://billkatz.com/2009/2/Remote-API-Hello-World Hope it helps, Bill On Feb 13, 3:24 am, lookon wrote: > I am under Windows. Anyone can tell me how to use remote_api? Thanks. > > p.s. I met the problem that the appengine module cannot be easily > imported. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Grouping and counting in GAE... advice needed
This is a brain dump (LONG) as I work through this problem. Hope it helps others or others can help me by correcting mistakes or suggesting alternatives. First, look at this article on Digg-like functionality: http://code.google.com/appengine/articles/overheard.html That article suggests this formula for a decaying vote ranking in models.set_vote(): rank = quote.created * DAY_SCALE + quote.votesum Each new vote causes this rank to be recalculated. The keys to this approach are (1) you only compute on write and (2) the passage of time increases rank values so all untouched (i.e., not recently voted on) articles don't have to be updated. In the example, DAY_SCALE = 4 and quote.created is # of days, so an article a day older would need 4 votes more than a new article to rank the same. In your case, you want an "ordered list of URLs with most votes in the last hour."This means a new vote is worth infinitely more than a vote more than an hour ago. (And you want a step function -- all votes within an hour are treated equally.) Any solution, to be manageable, should not require periodic updates to all articles, even untouched ones. First, put a property last_vote_time for the most recent vote in the model. So we can simply ignore every link that has last_vote_time more than an hour ago. Now we're left with all links that received a vote in the last hour, but rank scores will contain stale votes. This is where I run into trouble figuring out how to adapt the previous method to this problem. So I'm going to have to think of another approach. Let's start with a simpler problem where we only use non-overlapping hour windows, i.e., the rankings get reset every hour on the hour. In this case, the first vote after an hour boundary gets crossed resets the rank score. Rank is just votesum for this hour window. Now what happens if we use more of these hour windows, but stagger them across an hour: votesum_0_min = db.IntegerProperty() votesum_5_min = ... votesum_10_min = ... ... votesum_55_min = ... In the above, votesum_5_min covers the future hour span that starts 5 minutes into the hour, e.g. 6:05 to 7:05. On new vote, we look at the current time and the last_vote_time, then clear every votesum that straddles its boundary. Example: Last vote time was 8:16, current time of new vote is 8:33. We clear votesum_20_min, votesum_25_min, votesum_30_min, and set all of them to 1 vote for this new vote. All other votesums += 1. Let's say we it's now 8:38 pm and we want to see all the links with most votes for the last hour. Call our model URLModel. q = URLModel.gql('ORDER by votesum_40_min DESC') urls = q.fetch(...) We order on the window that has been active the longest for the current time. Looking at the query above, you notice that we have to remove all URLModel entities with last_vote_time more than an hour ago, but we want to order on votesum_40_min. We could do: q = URLModel.gql('WHERE last_vote_time > :hour_ago ORDER BY last_vote_time, votesum_40_min DESC') but we'll have to do the real votesum_40_min ordering after the fetch. We could use an equality filter that pretty much removes all entities with last_vote_time more than an hour ago. One solution would be to add a StringListProperty, call it last_vote_hours, that contains strings corresponding to the last vote's hour in 00-23 format and its hour + 1. So a vote at 7:45 pm would store ['19', '20] in last_vote_hours. Back to our query at 8:38 pm: q = URLModel.gql('WHERE last_vote_hours = '20' ORDER by votesum_40_min DESC') urls = q.fetch(...) Then just remove any urls with last_vote_hours more than an hour ago. I think that may be the most efficient way to do it but look forward to hearing from others if they've actually read this far :) -Bill --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Grouping and counting in GAE... advice needed
On Feb 24, 2:29 am, Bill wrote: > This is a brain dump (LONG) as I work through this problem. Hope it > > We could use an equality filter that pretty much removes all entities > with last_vote_time more than an hour ago. One solution would be to > add a StringListProperty, call it last_vote_hours, that contains > strings corresponding to the last vote's hour in 00-23 format and its > hour + 1. So a vote at 7:45 pm would store ['19', '20] in > last_vote_hours. > > Back to our query at 8:38 pm: > > q = URLModel.gql('WHERE last_vote_hours = '20' ORDER by votesum_40_min > DESC') > urls = q.fetch(...) To avoid past days from getting into results, the last_vote_hours would have a full date stamp + the hour, something like "2009-02-23 19" and "2009-02-23 20." Any alternatives or questions? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Clarification of storage pricing requested
There seems to be an inconsistency in the cited storage pricing. The blog post says $0.15/GB/mo, but the docs and my app dashboard say $0.005/GB/mo, which is a huge drop in pricing if correct. Which one is correct? And congratulations to the entire App Engine team for rolling out this much requested feature. -Bill --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---