[google-appengine] Re: Move app to another Google Account
Thank you all for your responses. What you suggested worked. I invited the other developer on another Google Account. Then on that new developer's Google Account I deleted the original creator of the application. That worked. Thanks again. On Oct 24, 8:17 pm, Diego <[EMAIL PROTECTED]> wrote: > Hi all, > > I was wondering if it was possible tomovea GogleAppEngine > application from oneGoogleAccounttoanother? > > Cheers, > Diego --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: JSONP Gateway
Just a suggestion: you could add a "context" parameter for applications where multiple dispatch is required. Jean-Lou. On Oct 31, 3:00 pm, "Malte Ubl" <[EMAIL PROTECTED]> wrote: > Hey, > > I've just released a very simple application that can be used to turn > any JSON webservice into a JSONP webservice. > It takes an url parameter, fetches that url, validates the json and > then returns it wrapped in a callback function. > > Is this ok with the terms of service? Do you think the problems with > respect to security are too serious to leave this online? > An example is > here:http://jsonpgateway.appspot.com/?url=http%3A//search.yahooapis.com/Im... > > Bye > Malte --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Confusions on transactions and concurrent writes to entity groups
"All writes are transactional."
Does this mean updating values on a single entity will lock the entire
group when put() is called?
On Sep 22, 10:52 am, Jeff S <[EMAIL PROTECTED]> wrote:
> To further clarify. All writes are transactional. Details on how the
> transactions work can be found in ryan's presentation from Google
> I/O:http://snarfed.org/space/datastore_talk.htmlThe section on
> transactions specifically begins at slide 49. You can also watch the
> video
> here:http://sites.google.com/site/io/under-the-covers-of-the-google-app-en...
>
> Cheers,
>
> Jeff
>
> On Sep 22, 10:36 am, Jeff S <[EMAIL PROTECTED]> wrote:
>
> > Hi David,
>
> > Even if a put request to the datastore is not run in a transaction,
> > the operation is automatically retried. Contention is not unique to
> > transactions. The benefit of using transactions, is that if one write
> > in the transaction times out (due to too much contention or some other
> > issue) the other parts of the transaction will not be applied. For
> > more details
> > see:http://code.google.com/appengine/docs/datastore/transactions.html#Usi...
>
> > Happy coding,
>
> > Jeff
>
> > On Sep 18, 6:25 pm, DXD <[EMAIL PROTECTED]> wrote:
>
> > > I appreciate any clarifications on my situation as follows. I have an
> > > entity group whose the root entity is called "root". When a particular
> > > URL is requested, a new entity is added to this group as a direct
> > > child of root. The code looks similar to this:
>
> > > def insert():
> > > root = Root.get_by_key_name('key_name')
> > > child = Child(parent=root)
> > > child.put()
>
> > > Note that the insert() function is not run in a transaction (not
> > > called by db.run_in_transaction()).
>
> > > I spawned many concurrent requests to this URL. The log shows that
> > > there are many failed requests with either "TransactionFailedError:
> > > too much contention on these datastore entities. please try again" or
> > > "DeadlineExceededError". Since I'm still a bit unclear about the
> > > internal working of the datastore, these are my explanations for what
> > > happened. Pls correct me where I'm wrong:
>
> > > 1. when one child entity is being inserted, it locks the entire group.
> > > All other concurrent requests are blocked, and their child.put()
> > > statement exclusively is retried a number of times. Say the limit
> > > number of retry is r.
>
> > > 2. If child.put() is retried r times but still doesn't go through, it
> > > gives up and yields the "too much contention" error.
>
> > > 3. If child.put() does not yet reach r times of retry, but its session
> > > already reaches the time limit t, then it fails yielding the
> > > "DeadlineExceededError".
>
> > > If my explanations are correct, isn't it true that the insert()
> > > function is exactly equivalent to this version?:
>
> > > def insert():
> > > root = Root.get_by_key_name('key_name')
> > > child = Child(parent=root)
> > > def txn()
> > > child.put()
> > > db.run_in_transaction(txn)
>
> > > Or more generally, is it true that all API operations that write to
> > > the datastore have exactly the same effect with transaction
> > > (automatically retried if failed, and so on)?
>
> > > Thanks for clarifications,
> > > David.
>
>
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] SMS Verification
I'm using a Dell Axim x51v Pocket PC, which is not a mobile phone, but App Engine reqires SMS verification. Obviously, I can't recieve SMS messages on it; is there a way to use my device without this verification? (Email would be fine) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: App Engine 'session' questions
> Which is why it was critical for me to know how much memory I had to > work with... how often GAE will invalidate my serverside data > structures and if I should expect my user's requests to be hitting > instances of my application on different servers / with different > IPs / etc if anyone has any info/tips on the rules GAE uses or > how it behaves I would love to hear!!! Otherwise I will continue down > the route of experimentation... I'll update this list if I can find > anything, It's likely that all of this stuff is dynamic, depending on activity by your application and by other GAE applications, and even time of day. (If the total GAE application load varies by time of day, Google is likely to vary the number of servers handling GAE requests as well.) Moreover, absent some explicit guarantee by Google, even if you observe some behavior that you can exploit, there's no reason to believe that such behavior will continue. Google will optimize optimize (read "change") this sort of thing to make their system run efficiently. On Oct 31, 1:28 pm, sal <[EMAIL PROTECTED]> wrote: > Joseph - > > Thanks for the link!! I did see this project earlier when hunting for > some info on the GAE memory behavior... > > The system looks great. I was hoping to however store 'plain' python > objects in RAM on the serverside - the reason for this is to not incur > any quota 'hits' by hitting the datastore... also I didn't want to > have to restrict to only pickle-able objects for the session data. > > The way I was hoping to architect my application - highly volatile / > complex data structures would be stored in the server's RAM as plain > python objects, critical data stored into the datastore - and if the > GAE invalidates the memory of my application serverside I would re- > construct those RAM structures algorithmically from some info stored > in the datastore. > > Which is why it was critical for me to know how much memory I had to > work with... how often GAE will invalidate my serverside data > structures and if I should expect my user's requests to be hitting > instances of my application on different servers / with different > IPs / etc if anyone has any info/tips on the rules GAE uses or > how it behaves I would love to hear!!! Otherwise I will continue down > the route of experimentation... I'll update this list if I can find > anything, > > thanks, > > On Oct 31, 1:51 pm, "[EMAIL PROTECTED]" > > > > <[EMAIL PROTECTED]> wrote: > > If you're trying to persist sessions per user across the datastore, > > gaeutilities already has a class you can use for that which uses both > > memcache and the datastore. Reads go to memcache first and then to the > > datastore if it's not found in memcache (memcache can't be trusted to > > be persistent). > > > If you're looking for something which can store data for all sessions, > > gaeutilities also has cache, which uses the same memcache/datastore > > logic. It also allows you to set expiration dates on the individual > > cache items. These cache items are accessible by all requests. > > > Both use standard Python dictionary methods for access > > > session['user'] = "bob" > > cache['myhit'] = "hit" > > > Both also support storage of any object that can be pickled, so you > > can store dictionaries, lists, and such. > > >http://gaeutilities.appspot.com/ > > > On Oct 31, 1:43 pm, sal <[EMAIL PROTECTED]> wrote: > > > > Nikola - thanks much for your input, I've been researching this for a > > > day or so now... and have found some things out in case anyone else is > > > wondering also. > > > > It looks like you can declare 'global' variables, and they stay in the > > > server's memory across all HTTP requests. I've created a hashmap > > > (err... what was the python version again? dict? :) I'm a J2EE guy) > > > and am using this to store a map of session cookies to the user's > > > requests, so I have now a working session. > > > > I still don't know how long the data in these global variables will > > > persist, or how much memory I have allocated to use, and I'm only > > > working locally on the SDK so I haven't tested on GAE yet. I am > > > planning to extend the session manager I created to work with memcache > > > and datastore to automatically persist sessions if/when I do find out > > > the real limits. > > > > The other question still open - is if GAE will be splitting multiple > > > HTTP requests over multiple copies of my 'in memory globals' > > > simultaneously... this will cause some more issues to work around... > > > if anyone has experience with this yet do let me know!! > > > > Thanks, > > > > On Oct 31, 7:52 am, Nikola <[EMAIL PROTECTED]> wrote: > > > > > GAE has no "global memory" - every handler runs in it's own address > > > > space. You can persist data in the datastore (perhaps caching with > > > > memcache for performance), or in browser cookies. A lot of > > > > applications can do with the Users API instead of full-blown sessions. > >
[google-appengine] Re: Modeling Hierarchical Data
Yes, if you use the lexical path you are limited to depths - which is not a problem for me as they are all shown on the same page and it gets far too complicated to read with many depths (see something like digg which goes 3 or 4 deep). I also use just a-z for the path encoding, which limits us again but keeps it readable. Another big problem with this method is it uses up your inequality filter to get the path. I've also been looking at using lists to store the parents for example.. Comment 0 - [A] Comment 1 - [A,B] Comment 2 - [A,B,C] Comment 3 - [D] Comment 4 - [E] Comment 5 - [E,F] Where A,B,C etc. are keys of its parent. For adding children, you grab the parents list, and add the parents key to the list. Then you can use a single filter to find all comments of parent 'A', or combinations using IN filters. If you need depths just store the list length also. Again, there are downsides as you will be limited to the exploding index limit, and puts will be slower when you get very deep - but the bonus is you get your inequality filter back so you can add ratings or something. Anthony On Oct 31, 9:33 pm, Chris Tan <[EMAIL PROTECTED]> wrote: > Thanks for the heads up on that interesting method. > > Just wondering, from your comment in the other thread: > > "Or, stick with the lexical path but use the full byte rather than > decimal to store the number.. then just 2 bytes give you a max of 64k > comments per depth and 200+ deep. " > > The sort order for StringProperty is in Unicode, so wouldn't you have > to use > an something like UTF-8? > > Some pros of this method: > - Fast query for children > - Possible to reconstruct hierarchy > > Cons: > - Limited depth > - Need to do query for siblings before setting the path > - Somewhat complex. Need to increment Unicode character > > On Oct 31, 3:34 am, Anthony <[EMAIL PROTECTED]> wrote: > > > Store all the parent ids as a hierarchy in a string... > > > h="id1/id2/" > > h="id1/id2/id3" > > h="id1/id2/id3/id4" > > h="id5/" > > h="id5/id6" > > > You can then filter WHERE h > "id1/" and h < "id5/" to get all > > children of id1. > > > Or you can use entity groups & ancestors. > > > More details > > here...http://groups.google.com/group/google-appengine/browse_thread/thread/... > > > On Oct 31, 8:31 am, Chris Tan <[EMAIL PROTECTED]> wrote: > > > > I'm wondering how other people have been modeling hierarchical > > > information > > > such as nested comments. > > > > My first thought was to do something like this: > > > > class Comment(db.Model): > > > reply_to = db.SelfReferenceProperty(collection_name="replies") > > > author = db.UserProperty() > > > content = db.TextProperty() > > > > However, this isn't scalable, as a query is made for every > > > comment in the thread. > > > > Any ideas? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: patterns for expiration settings for static files
Yes, carefull there is a bug in the SDK, "expiration" does not work. default_expiration works Another thing, for your second handler, if it's a file you should use "static_files" and not "static_dir" On 31 oct, 21:20, dobee <[EMAIL PROTECTED]> wrote: > is it possible to serve a static directory with a given expiration > time while serving a contained file with a different expiration. > > for example this does not work for me: > > - url: /static > expiration: 1d > static_dir: static > > - url: /static/admin/xy\.nocache\.js > expiration: 60s > static_dir: static > > thx in advance, bernd --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Changing overriding the method(HTTP verb) of the request by extending RequestHandler
I want to changed the request verb of a request in the case a
parameter _method is provided, for example if a POST request comes in
with a parameter _method=PUT, I need to change the request to call the
put method of the handler. This is to cope with the way prototype.js
works with verbs like PUT and DELETE(workaround for IE). Here is my
first attempt:
class MyRequestHandler(webapp.RequestHandler):
def initialize(self, request, response):
m = request.get('_method')
if m:
request.method = m.upper()
webapp.RequestHandler.initialize(self, request, response)
The problem is, for some reason whenever the redirect is done, the
self.request.params are emptied by the time the handling method(put or
delete) is called, even though they were populated when initialize is
called. Anyone have a clue why this is? As a workaround I thought I
could clone the params at initialize() time, but .copy() did not work,
and I haven't found a way to do that either.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: Modeling Hierarchical Data
Hi, With the example of threaded comments, I'm not entirely convinced that there would be anything wrong with doing a SelfReferenceProperty to store the link between a reply to a comment and the original comment. Let me explain to you how I might approach this problem, and obviously you can modify this for your own needs. I would probably turn threading off by default, allowing comments to be displayed relatively chronologically, but also allowing the user to view comment threads as he or she desired. My comment model would probably be something like: class Comment(db.Model) reply_to = db.SelfReferenceProperty() has_reply = db.BooleanProperty(default=False) comment_index = db.StringProperty() author = db.UserProperty() comment = db.TextProperty() post_time = db.DateTimeProperty() I would just use the system that Brett discussed in Building Scalable Web Application ( http://sites.google.com/site/io/building-scalable-web-applications-with-google-app-engine) to ensure that all comments were more or less sorted. To just view in order, I would sort by the comment index, and be done with that. A user could either reply to a comment, or post an entirely new comment. If a person replied to a comment, I would store with the reply comment a reference to the 'parent' comment (here I don't mean parent in terms of txns). In the 'parent' comment, I would store as a boolean property parent_comment.has_reply=True. This is just for ease, so when rendering the template, instead of doing a query for back references to the comment, I could just display a link below the comment 'show comment thread'. When the user clicked on 'show comment thread' you could just do a back reference query on all of the replies to that comment. You'd fetch the first, say, 11 of those comments, display the first 10, and a link to show more if the 11th existed. This could work to have comments nested as deep as you like. All of the replies could also have replies, and you could display a sub-thread just by checking to see if the reply has a reply. Hopefully this response makes sense. I'm sure there are other approaches to this problem, but I'm fairly certain this could work well, and would be easy to render and query for as well as being fairly efficient. -Marzia On Fri, Oct 31, 2008 at 2:33 PM, Chris Tan <[EMAIL PROTECTED]> wrote: > > Thanks for the heads up on that interesting method. > > Just wondering, from your comment in the other thread: > > "Or, stick with the lexical path but use the full byte rather than > decimal to store the number.. then just 2 bytes give you a max of 64k > comments per depth and 200+ deep. " > > The sort order for StringProperty is in Unicode, so wouldn't you have > to use > an something like UTF-8? > > > Some pros of this method: > - Fast query for children > - Possible to reconstruct hierarchy > > Cons: > - Limited depth > - Need to do query for siblings before setting the path > - Somewhat complex. Need to increment Unicode character > > > > On Oct 31, 3:34 am, Anthony <[EMAIL PROTECTED]> wrote: > > Store all the parent ids as a hierarchy in a string... > > > > h="id1/id2/" > > h="id1/id2/id3" > > h="id1/id2/id3/id4" > > h="id5/" > > h="id5/id6" > > > > You can then filter WHERE h > "id1/" and h < "id5/" to get all > > children of id1. > > > > Or you can use entity groups & ancestors. > > > > More details here... > http://groups.google.com/group/google-appengine/browse_thread/thread/... > > > > On Oct 31, 8:31 am, Chris Tan <[EMAIL PROTECTED]> wrote: > > > > > I'm wondering how other people have been modeling hierarchical > > > information > > > such as nested comments. > > > > > My first thought was to do something like this: > > > > > class Comment(db.Model): > > > reply_to = db.SelfReferenceProperty(collection_name="replies") > > > author = db.UserProperty() > > > content = db.TextProperty() > > > > > However, this isn't scalable, as a query is made for every > > > comment in the thread. > > > > > Any ideas? > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Modeling Hierarchical Data
I think you misunderstand my point then, because while I did say you might use any representation that feels natural, I also said that expensive operations do need optimizations. In this case, building an index. And of course, if an operation is expensive: cache it! On Oct 31, 2008 5:11 PM, "Steve Schwarz" <[EMAIL PROTECTED]> wrote: On Fri, Oct 31, 2008 at 3:18 PM, Calvin Spealman <[EMAIL PROTECTED]> wrote: > > It seems to me that the question at hand isn't hierarchial entities, per se. We have quite a few... Calvin, I very new to GAE but that does seem to be the issue Chris was posing. While the "natural" representation is a tree structure, the cost of getting each comment might exceed the compute/selects allowed by GAE w/in a request. Imagine if these were the comments for some popular Slashdot article. At what point do you optimize for read speed? I'd guess the use case for comments is "read many write few". It could be that the best model is to actually update a single comment object with new comments (it would store the hierarchy) so that the read case involves querying for a single large object. An alternative is to store each comment in a hierarchy as described in Chris' email and have an external periodic job hit the server and update a single comment instance with the hierarchy when it detects that new comments have been added (a kind of external mapreduce). Or a combination that updates the "master" comment with the new comment when it is added. If you want to do moderation the moderation step could be the one that appends the new comment to the master comment. I'm thrashing on a similar problem. I want to serve a relatively large XML file containing a couple thousand elements. It will have very few updates and many reads. I'd like to be able to perform CRUD operations on the elements and then have the XML file be updated when any of those elements change. I don't want to prematurely optimize, but then again don't want to have a "non-starter" either. >From reading this list and the docs it seems there are times when traditional DB backed designs have to be modified to work well in the GAE environment. I'm also looking for direction/patterns/best practices. Best Regards, Steve --~--~-~--~~~---~--~~ You received this message because you are s... --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Modeling Hierarchical Data
Hi Steve, I was thinking along the same lines. Having the root comment or thread hold a cache of the whole tree, which would be updated whenever an entity is updated, added or removed seems like the most efficient way for large hierarchies. You could have your cached tree represented using nested dictionary objects on your root object using a PickleProperty (see: http://groups.google.com/group/google-appengine/msg/8433525107a8bb92) to be converted to XML as required. The comment class could have a path property (e.g. "id1/id2/id3") which could be quickly traversed to retrieve or update child comments. Since it doesn't need to be indexed, it can be a TextProperty() which isn't limited to 500 bytes. CRUD operations could be overridden in your Model class: class Comment(db.Model): path = TextProperty() def put(self): # update root comment super(Comment, self).save() def delete(self): # update root comment super(Comment, self).delete() Pros: - Reads only need to fetch 1 (root tree) or 2 entities (look-up child path & return sub-tree from root) - Not limited by depth Cons: - Writes update 2 entities. - Pickling/Unpickling could be resource intensive On Oct 31, 2:11 pm, "Steve Schwarz" <[EMAIL PROTECTED]> wrote: > On Fri, Oct 31, 2008 at 3:18 PM, Calvin Spealman <[EMAIL PROTECTED]>wrote: > > > It seems to me that the question at hand isn't hierarchial entities, per > > se. We have quite a few options for representing them and any will do, > > truthfully. > > > It is the operations on them which pose issues. Notably, how do we make our > > queries for decendants cost-effective? > > > I say use whatever representation feels comfortablee to you. Other issues > > you may deal with are a matter of indexing and other operations both > > important and separate from the exact representation. If the need is "give > > me all of this node's decendants" then we may create simple index entries > > mapping a node to each of its decendants and mmay be in a group without > > impacting the nodes themselves or other nodes' decendant indexes. > > Calvin, > I very new to GAE but that does seem to be the issue Chris was posing. While > the "natural" representation is a tree structure, the cost of getting each > comment might exceed the compute/selects allowed by GAE w/in a request. > Imagine if these were the comments for some popular Slashdot article. At > what point do you optimize for read speed? I'd guess the use case for > comments is "read many write few". It could be that the best model is to > actually update a single comment object with new comments (it would store > the hierarchy) so that the read case involves querying for a single large > object. > > An alternative is to store each comment in a hierarchy as described in > Chris' email and have an external periodic job hit the server and update a > single comment instance with the hierarchy when it detects that new comments > have been added (a kind of external mapreduce). Or a combination that > updates the "master" comment with the new comment when it is added. If you > want to do moderation the moderation step could be the one that appends the > new comment to the master comment. > > I'm thrashing on a similar problem. I want to serve a relatively large XML > file containing a couple thousand elements. It will have very few updates > and many reads. I'd like to be able to perform CRUD operations on the > elements and then have the XML file be updated when any of those elements > change. I don't want to prematurely optimize, but then again don't want to > have a "non-starter" either. > > From reading this list and the docs it seems there are times when > traditional DB backed designs have to be modified to work well in the GAE > environment. I'm also looking for direction/patterns/best practices. > > Best Regards, > Steve --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Modeling Hierarchical Data
Thanks for the heads up on that interesting method. Just wondering, from your comment in the other thread: "Or, stick with the lexical path but use the full byte rather than decimal to store the number.. then just 2 bytes give you a max of 64k comments per depth and 200+ deep. " The sort order for StringProperty is in Unicode, so wouldn't you have to use an something like UTF-8? Some pros of this method: - Fast query for children - Possible to reconstruct hierarchy Cons: - Limited depth - Need to do query for siblings before setting the path - Somewhat complex. Need to increment Unicode character On Oct 31, 3:34 am, Anthony <[EMAIL PROTECTED]> wrote: > Store all the parent ids as a hierarchy in a string... > > h="id1/id2/" > h="id1/id2/id3" > h="id1/id2/id3/id4" > h="id5/" > h="id5/id6" > > You can then filter WHERE h > "id1/" and h < "id5/" to get all > children of id1. > > Or you can use entity groups & ancestors. > > More details > here...http://groups.google.com/group/google-appengine/browse_thread/thread/... > > On Oct 31, 8:31 am, Chris Tan <[EMAIL PROTECTED]> wrote: > > > I'm wondering how other people have been modeling hierarchical > > information > > such as nested comments. > > > My first thought was to do something like this: > > > class Comment(db.Model): > > reply_to = db.SelfReferenceProperty(collection_name="replies") > > author = db.UserProperty() > > content = db.TextProperty() > > > However, this isn't scalable, as a query is made for every > > comment in the thread. > > > Any ideas? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Modeling Hierarchical Data
On Fri, Oct 31, 2008 at 3:18 PM, Calvin Spealman <[EMAIL PROTECTED]>wrote: > It seems to me that the question at hand isn't hierarchial entities, per > se. We have quite a few options for representing them and any will do, > truthfully. > > It is the operations on them which pose issues. Notably, how do we make our > queries for decendants cost-effective? > > I say use whatever representation feels comfortablee to you. Other issues > you may deal with are a matter of indexing and other operations both > important and separate from the exact representation. If the need is "give > me all of this node's decendants" then we may create simple index entries > mapping a node to each of its decendants and mmay be in a group without > impacting the nodes themselves or other nodes' decendant indexes. > Calvin, I very new to GAE but that does seem to be the issue Chris was posing. While the "natural" representation is a tree structure, the cost of getting each comment might exceed the compute/selects allowed by GAE w/in a request. Imagine if these were the comments for some popular Slashdot article. At what point do you optimize for read speed? I'd guess the use case for comments is "read many write few". It could be that the best model is to actually update a single comment object with new comments (it would store the hierarchy) so that the read case involves querying for a single large object. An alternative is to store each comment in a hierarchy as described in Chris' email and have an external periodic job hit the server and update a single comment instance with the hierarchy when it detects that new comments have been added (a kind of external mapreduce). Or a combination that updates the "master" comment with the new comment when it is added. If you want to do moderation the moderation step could be the one that appends the new comment to the master comment. I'm thrashing on a similar problem. I want to serve a relatively large XML file containing a couple thousand elements. It will have very few updates and many reads. I'd like to be able to perform CRUD operations on the elements and then have the XML file be updated when any of those elements change. I don't want to prematurely optimize, but then again don't want to have a "non-starter" either. >From reading this list and the docs it seems there are times when traditional DB backed designs have to be modified to work well in the GAE environment. I'm also looking for direction/patterns/best practices. Best Regards, Steve --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: App Engine 'session' questions
Joseph - Thanks for the link!! I did see this project earlier when hunting for some info on the GAE memory behavior... The system looks great. I was hoping to however store 'plain' python objects in RAM on the serverside - the reason for this is to not incur any quota 'hits' by hitting the datastore... also I didn't want to have to restrict to only pickle-able objects for the session data. The way I was hoping to architect my application - highly volatile / complex data structures would be stored in the server's RAM as plain python objects, critical data stored into the datastore - and if the GAE invalidates the memory of my application serverside I would re- construct those RAM structures algorithmically from some info stored in the datastore. Which is why it was critical for me to know how much memory I had to work with... how often GAE will invalidate my serverside data structures and if I should expect my user's requests to be hitting instances of my application on different servers / with different IPs / etc if anyone has any info/tips on the rules GAE uses or how it behaves I would love to hear!!! Otherwise I will continue down the route of experimentation... I'll update this list if I can find anything, thanks, On Oct 31, 1:51 pm, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > If you're trying to persist sessions per user across the datastore, > gaeutilities already has a class you can use for that which uses both > memcache and the datastore. Reads go to memcache first and then to the > datastore if it's not found in memcache (memcache can't be trusted to > be persistent). > > If you're looking for something which can store data for all sessions, > gaeutilities also has cache, which uses the same memcache/datastore > logic. It also allows you to set expiration dates on the individual > cache items. These cache items are accessible by all requests. > > Both use standard Python dictionary methods for access > > session['user'] = "bob" > cache['myhit'] = "hit" > > Both also support storage of any object that can be pickled, so you > can store dictionaries, lists, and such. > > http://gaeutilities.appspot.com/ > > On Oct 31, 1:43 pm, sal <[EMAIL PROTECTED]> wrote: > > > Nikola - thanks much for your input, I've been researching this for a > > day or so now... and have found some things out in case anyone else is > > wondering also. > > > It looks like you can declare 'global' variables, and they stay in the > > server's memory across all HTTP requests. I've created a hashmap > > (err... what was the python version again? dict? :) I'm a J2EE guy) > > and am using this to store a map of session cookies to the user's > > requests, so I have now a working session. > > > I still don't know how long the data in these global variables will > > persist, or how much memory I have allocated to use, and I'm only > > working locally on the SDK so I haven't tested on GAE yet. I am > > planning to extend the session manager I created to work with memcache > > and datastore to automatically persist sessions if/when I do find out > > the real limits. > > > The other question still open - is if GAE will be splitting multiple > > HTTP requests over multiple copies of my 'in memory globals' > > simultaneously... this will cause some more issues to work around... > > if anyone has experience with this yet do let me know!! > > > Thanks, > > > On Oct 31, 7:52 am, Nikola <[EMAIL PROTECTED]> wrote: > > > > GAE has no "global memory" - every handler runs in it's own address > > > space. You can persist data in the datastore (perhaps caching with > > > memcache for performance), or in browser cookies. A lot of > > > applications can do with the Users API instead of full-blown sessions. > > > > -Nikola > > > > On Oct 30, 10:11 pm, sal <[EMAIL PROTECTED]> wrote: > > > > > I tried to RTFM a bit on the topic and wasn't able to find what was > > > > needed - if someone with more App Engine experience could point me in > > > > the right direction that would be great! > > > > > I understand that any cloud computing environment needs to allow for > > > > dynamic allocation of computing resources - I am just looking to > > > > discover what some of the operating parameters are so I can avoid > > > > errors with my apps. My questions: > > > > > - Regarding the behavior of regular python data structures: Assume I > > > > have an object (an instance of a class) in global memory (for > > > > performance reasons) which contains various fields holding values. > > > > How long will a single user's web requests hit the same instance of > > > > that object? Will multiple requests ever go to different 'cached' > > > > instances of that object or another instance of that in-memory data > > > > structure located on a different server? Do I need to anticipate > > > > requests coming from the same user going to different web servers that > > > > have independant cached versions of my application's data structures, > > > > simult
[google-appengine] patterns for expiration settings for static files
is it possible to serve a static directory with a given expiration time while serving a contained file with a different expiration. for example this does not work for me: - url: /static expiration: 1d static_dir: static - url: /static/admin/xy\.nocache\.js expiration: 60s static_dir: static thx in advance, bernd --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Modeling Hierarchical Data
It seems to me that the question at hand isn't hierarchial entities, per se. We have quite a few options for representing them and any will do, truthfully. It is the operations on them which pose issues. Notably, how do we make our queries for decendants cost-effective? I say use whatever representation feels comfortablee to you. Other issues you may deal with are a matter of indexing and other operations both important and separate from the exact representation. If the need is "give me all of this node's decendants" then we may create simple index entries mapping a node to each of its decendants and mmay be in a group without impacting the nodes themselves or other nodes' decendant indexes. On Oct 31, 2008 3:21 PM, "Sal" <[EMAIL PROTECTED]> wrote: Yes, entity groups should be small since 'touching' an entity in the group/path/hierarchy, blocks the whole group. Entity groups are really only meant to be used in situations where you need transaction support. In fact, quoting the GAE documentation, "Only use entity groups when they are needed for transactions. For other relationships between entities, use ReferenceProperty properties and Key values". On Oct 31, 11:23 am, yejun <[EMAIL PROTECTED]> wrote: > I think I have read somewhere in documents, ... --~--~-~--~~~---~--~~ You received this message because you are subs... --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] JSONP Gateway
Hey, I've just released a very simple application that can be used to turn any JSON webservice into a JSONP webservice. It takes an url parameter, fetches that url, validates the json and then returns it wrapped in a callback function. Is this ok with the terms of service? Do you think the problems with respect to security are too serious to leave this online? An example is here: http://jsonpgateway.appspot.com/?url=http%3A//search.yahooapis.com/ImageSearchService/V1/imageSearch%3Fappid%3DYahooDemo%26query%3DMadonna%26output%3Djson Bye Malte --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Modeling Hierarchical Data
Yes, entity groups should be small since 'touching' an entity in the group/path/hierarchy, blocks the whole group. Entity groups are really only meant to be used in situations where you need transaction support. In fact, quoting the GAE documentation, "Only use entity groups when they are needed for transactions. For other relationships between entities, use ReferenceProperty properties and Key values". On Oct 31, 11:23 am, yejun <[EMAIL PROTECTED]> wrote: > I think I have read somewhere in documents, the entity groups size > need to keep small. > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Writing binary data in the HTTP response
I tried your code. No error whatsoever and working as expected.
On Oct 31, 12:33 pm, Sito <[EMAIL PROTECTED]> wrote:
> Hi all,
>
> I can't find a way to write arbitrary binary data in my response. I
> have a big list of ints that I would like to serve in a response as
> compact as possible. For this, I create an array of shorts (my ints
> never take more than two bytes). I create my array like this:
>
> from array import array
>
> int_bytes = array('H')
> for p in int_data:
> int_bytes.append(p)
>
> later on, when I want to write the response, I do this:
>
> self.response.out.write(int_bytes.tostring())
>
> but I get this nasty error:
>
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xf1 in position
> 0: ordinal not in range(128)
>
> I understand from this that StringIO, of which self.response.out is an
> instance, does not accept bytes that cannot be encoded to ascii.
>
> Does anyone know how I can do to write this kind of binary data?
>
> Thank you,
>
> Alfonso
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] AppEngine Hack-a-thon at the Ann Arbor Google office 11/17
Hello all, If you are not interested in learning about AppEngine feel free to move on to the next email. The Michigan Python Users group in conjunction with Google are happy to present: Google App Engine Hack-a-thon: Ann Arbor What: An App Engine developer event for Google App Engine! * Learn about Google App Engine: We will have talks on the major features of Google App Engine at different points throughout the day. We will run through developing an app with the SDK and show you how to deploy and manage applications on Google App Engine. * Build With Us, or Build Your Own: You are welcome to bring along anything you can prepare ahead of time (sketches, designs, web page mock ups, etc.) and use the time and information provided to develop your idea into a working application, then share it with the world. Or, you can code along with us in building a Google App Engine application from start to finish. Who: You! Your ideas and your enthusiasm complete the mix. We will assume some basic skills and preparation for the event, including an existing knowledge of the Python programming language. We'll provide power, copies of the SDK, and an awesome ambiance. Just bring yourself and your laptop. Some light snacks and beverages will be available, as will a pizza lunch. If you decide to bring extra food, please make sure it is computer friendly. :) When: Monday Nov 17th, 2008 10AM-6PM Where: Google's Ann Arbor Office: 201 S. Division St. Ann Arbor, MI 48104 You can read more about previous hack-a-thons at the AppEngine blog (http://googleappengine.blogspot.com) RSVP for the Ann Arbor event with the link below. http://hackathon.appspot.com/index.html?event=agloYWNrYXRob25yDAsSBUV2ZW50GLw2DA Looking forward to seeing you there! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Modeling Hierarchical Data
I think I have read somewhere in documents, the entity groups size need to keep small. On Oct 31, 6:34 am, Anthony <[EMAIL PROTECTED]> wrote: > Store all the parent ids as a hierarchy in a string... > > h="id1/id2/" > h="id1/id2/id3" > h="id1/id2/id3/id4" > h="id5/" > h="id5/id6" > > You can then filter WHERE h > "id1/" and h < "id5/" to get all > children of id1. > > Or you can use entity groups & ancestors. > > More details > here...http://groups.google.com/group/google-appengine/browse_thread/thread/... > > On Oct 31, 8:31 am, Chris Tan <[EMAIL PROTECTED]> wrote: > > > I'm wondering how other people have been modeling hierarchical > > information > > such as nested comments. > > > My first thought was to do something like this: > > > class Comment(db.Model): > > reply_to = db.SelfReferenceProperty(collection_name="replies") > > author = db.UserProperty() > > content = db.TextProperty() > > > However, this isn't scalable, as a query is made for every > > comment in the thread. > > > Any ideas? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Add a flag to the appcfg.py to zip/upload the sources
Even though I shudder at the thought of not using source control, Sylvain's current solution is an elegant way of storing a backup copy of each version of her app. It sounds like Mahmoud's group has their own process that works for them using SVN. I think the idea of automatically backing up a project every time you deploy it is great. I don't think it should be part of appcfg.py though. Everyone will have their own idea of what the ideal backup solution is - some will want a flag to commit to SVN on every upload, some will want to zip the files, some will want to 7zip the files, some will want a tar.gz, and yet others will want git or CVS or whatever. It's simple enough, as we see above (5 or 6 lines of boilerplate) to script a solution custom tailored to each project's needs. -peter On Oct 31, 11:16 am, Mahmoud <[EMAIL PROTECTED]> wrote: > In our project, we pack the javascript and CSS and switch some flags > around before uploading the source to the server. So what gets > deployed is a bit different from what is on our SVN trunk (but we tag > it before uploading). I'm still not sure why you'd want to download > your source from GAE, if you can easily and securely check it out from > SVN. > > -Mahmoud > > On Oct 31, 5:36 am, Sylvain <[EMAIL PROTECTED]> wrote: > > > It seems that a lot of developers would like to download their full > > project (source + static). > > Currently, there is no solution (even a source constrol). > > > For my project, I use a script like this one (with 7zip): > > > --- START BAT --- > > 7za a -tzip src\source.zip src\ -xr!*.pyc > > appcfg.py update src/ > > --- END BAT --- > > > APP.YAML --- > > - url: /source.zip > > static_files: source.zip > > upload: source.zip > > login: admin > > -- > > > But I think all this script could be added to the "appcf.py update" > > layer with a flag like one "source=source.zip" > > > I've created a feature request > > :http://code.google.com/p/googleappengine/issues/detail?id=831 > > > Regards --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Memcache exhaustion policy question
Hi, Memcache uses a shared pool of cache memory across applications, and at any given time the amount of memory free in the pool may vary. Memcache resources are managed so that we can maintain the highest cache hitrate possible, which ensures your application is running as efficiently as possible. For an application with appreciable traffic, you can expect 100MB of Memcache resources, increasing proportionally as your app gets larger. However, there are no guarantees that any entry can be found in your cache at any given time. It's important to write your application robustly so that when an entry is not found in the cache, you retrieve the resource elsewhere. -Marzia On Tue, Oct 28, 2008 at 8:57 AM, pr3d4t0r <[EMAIL PROTECTED]> wrote: > > Greetings. > > Is the cache exhaustion policy for Memcache defined somewhere? The > documentation indicates that data are evicted due to memory exhaustion > in the cache, but gives no specific volumes about how that works. Is > the memory limit documented anywhere, or can you provide an insight? > > Thanks in advance and have a great day, > > pr3d4t0r > http://teslatestament.com > http://istheserverup.net > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: App Engine 'session' questions
If you're trying to persist sessions per user across the datastore, gaeutilities already has a class you can use for that which uses both memcache and the datastore. Reads go to memcache first and then to the datastore if it's not found in memcache (memcache can't be trusted to be persistent). If you're looking for something which can store data for all sessions, gaeutilities also has cache, which uses the same memcache/datastore logic. It also allows you to set expiration dates on the individual cache items. These cache items are accessible by all requests. Both use standard Python dictionary methods for access session['user'] = "bob" cache['myhit'] = "hit" Both also support storage of any object that can be pickled, so you can store dictionaries, lists, and such. http://gaeutilities.appspot.com/ On Oct 31, 1:43 pm, sal <[EMAIL PROTECTED]> wrote: > Nikola - thanks much for your input, I've been researching this for a > day or so now... and have found some things out in case anyone else is > wondering also. > > It looks like you can declare 'global' variables, and they stay in the > server's memory across all HTTP requests. I've created a hashmap > (err... what was the python version again? dict? :) I'm a J2EE guy) > and am using this to store a map of session cookies to the user's > requests, so I have now a working session. > > I still don't know how long the data in these global variables will > persist, or how much memory I have allocated to use, and I'm only > working locally on the SDK so I haven't tested on GAE yet. I am > planning to extend the session manager I created to work with memcache > and datastore to automatically persist sessions if/when I do find out > the real limits. > > The other question still open - is if GAE will be splitting multiple > HTTP requests over multiple copies of my 'in memory globals' > simultaneously... this will cause some more issues to work around... > if anyone has experience with this yet do let me know!! > > Thanks, > > On Oct 31, 7:52 am, Nikola <[EMAIL PROTECTED]> wrote: > > > GAE has no "global memory" - every handler runs in it's own address > > space. You can persist data in the datastore (perhaps caching with > > memcache for performance), or in browser cookies. A lot of > > applications can do with the Users API instead of full-blown sessions. > > > -Nikola > > > On Oct 30, 10:11 pm, sal <[EMAIL PROTECTED]> wrote: > > > > I tried to RTFM a bit on the topic and wasn't able to find what was > > > needed - if someone with more App Engine experience could point me in > > > the right direction that would be great! > > > > I understand that any cloud computing environment needs to allow for > > > dynamic allocation of computing resources - I am just looking to > > > discover what some of the operating parameters are so I can avoid > > > errors with my apps. My questions: > > > > - Regarding the behavior of regular python data structures: Assume I > > > have an object (an instance of a class) in global memory (for > > > performance reasons) which contains various fields holding values. > > > How long will a single user's web requests hit the same instance of > > > that object? Will multiple requests ever go to different 'cached' > > > instances of that object or another instance of that in-memory data > > > structure located on a different server? Do I need to anticipate > > > requests coming from the same user going to different web servers that > > > have independant cached versions of my application's data structures, > > > simultaneously? > > > > - How much 'memory' or 'ram' (Regular python memory') can I consume > > > via my python code... in the form of lists, Strings, dicts, etc. > > > before I encounter an exception? Is the limit per-user-session or per- > > > app? > > > > - I noticed from limited testing that there is at least a small > > > semblance of a session... what is the token used to arrbitrate them? > > > ie: does GAE use a browser cookie, the IP of the user - or a cookie > > > containing the user id? (I want to make sure that I don't invalidate > > > whatever the mechanic is, so I don't destroy the user's sessions) > > > > I'll try to keep searching documentation for some answers... if anyone > > > can point me to any info in the right direction I *greatly* > > > appreciate, in advance, all your help, > > > > - Sal > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: App Engine 'session' questions
Nikola - thanks much for your input, I've been researching this for a day or so now... and have found some things out in case anyone else is wondering also. It looks like you can declare 'global' variables, and they stay in the server's memory across all HTTP requests. I've created a hashmap (err... what was the python version again? dict? :) I'm a J2EE guy) and am using this to store a map of session cookies to the user's requests, so I have now a working session. I still don't know how long the data in these global variables will persist, or how much memory I have allocated to use, and I'm only working locally on the SDK so I haven't tested on GAE yet. I am planning to extend the session manager I created to work with memcache and datastore to automatically persist sessions if/when I do find out the real limits. The other question still open - is if GAE will be splitting multiple HTTP requests over multiple copies of my 'in memory globals' simultaneously... this will cause some more issues to work around... if anyone has experience with this yet do let me know!! Thanks, On Oct 31, 7:52 am, Nikola <[EMAIL PROTECTED]> wrote: > GAE has no "global memory" - every handler runs in it's own address > space. You can persist data in the datastore (perhaps caching with > memcache for performance), or in browser cookies. A lot of > applications can do with the Users API instead of full-blown sessions. > > -Nikola > > On Oct 30, 10:11 pm, sal <[EMAIL PROTECTED]> wrote: > > > I tried to RTFM a bit on the topic and wasn't able to find what was > > needed - if someone with more App Engine experience could point me in > > the right direction that would be great! > > > I understand that any cloud computing environment needs to allow for > > dynamic allocation of computing resources - I am just looking to > > discover what some of the operating parameters are so I can avoid > > errors with my apps. My questions: > > > - Regarding the behavior of regular python data structures: Assume I > > have an object (an instance of a class) in global memory (for > > performance reasons) which contains various fields holding values. > > How long will a single user's web requests hit the same instance of > > that object? Will multiple requests ever go to different 'cached' > > instances of that object or another instance of that in-memory data > > structure located on a different server? Do I need to anticipate > > requests coming from the same user going to different web servers that > > have independant cached versions of my application's data structures, > > simultaneously? > > > - How much 'memory' or 'ram' (Regular python memory') can I consume > > via my python code... in the form of lists, Strings, dicts, etc. > > before I encounter an exception? Is the limit per-user-session or per- > > app? > > > - I noticed from limited testing that there is at least a small > > semblance of a session... what is the token used to arrbitrate them? > > ie: does GAE use a browser cookie, the IP of the user - or a cookie > > containing the user id? (I want to make sure that I don't invalidate > > whatever the mechanic is, so I don't destroy the user's sessions) > > > I'll try to keep searching documentation for some answers... if anyone > > can point me to any info in the right direction I *greatly* > > appreciate, in advance, all your help, > > > - Sal --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Add a flag to the appcfg.py to zip/upload the sources
For me I don't need it really. But if you look at this group, how many dev lose their code because they think they could download it again. So download the source is an important feature even with a source control or for a small project. Regards. On 31 oct, 16:16, Mahmoud <[EMAIL PROTECTED]> wrote: > In our project, we pack the javascript and CSS and switch some flags > around before uploading the source to the server. So what gets > deployed is a bit different from what is on our SVN trunk (but we tag > it before uploading). I'm still not sure why you'd want to download > your source from GAE, if you can easily and securely check it out from > SVN. > > -Mahmoud > > On Oct 31, 5:36 am, Sylvain <[EMAIL PROTECTED]> wrote: > > > It seems that a lot of developers would like to download their full > > project (source + static). > > Currently, there is no solution (even a source constrol). > > > For my project, I use a script like this one (with 7zip): > > > --- START BAT --- > > 7za a -tzip src\source.zip src\ -xr!*.pyc > > appcfg.py update src/ > > --- END BAT --- > > > APP.YAML --- > > - url: /source.zip > > static_files: source.zip > > upload: source.zip > > login: admin > > -- > > > But I think all this script could be added to the "appcf.py update" > > layer with a flag like one "source=source.zip" > > > I've created a feature request > > :http://code.google.com/p/googleappengine/issues/detail?id=831 > > > Regards --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: will google think of to support app own unique IP
On Fri, Oct 31, 2008 at 2:36 AM, yu ping322 <[EMAIL PROTECTED]> wrote: > because "ghs.google.com" can not access in my country. If that's blocked in your country, assigning a static IP to an application is not going to help. Dave. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Help: Problem getting example: MuvMuv working on Google App Engine (simplejson dependancy)
Hi Alok, MuvMuv requires the simplejson library. The application is written assuming you've downloaded the library and included it in your app's root directory when deploying. It just so happens that Django actually includes simplejson as a library, so it wasn't necessary to do this. You can replace 'import simplejson' with 'from django.utils import simplejson' -Marzia On Fri, Oct 31, 2008 at 8:53 AM, Alok <[EMAIL PROTECTED]> wrote: > > Hi, > > I am a relatively new Python and web developer and am currently trying > to get Imaging working on Google App Engine. > I pulled down the MuvMuv sample application code to try and dissect it > and borrow some good working code. > The problem I am having is that the 'simplejson' module is not > importing properly on my local machine. I am not familiar with JSON > and was hoping for some sample code to learn from that didn't require > me to learn it just yet. I have downloaded and installed 'simplejson' > on my local machine using 'python setup.py install' and have confirmed > that it works in the Python command line, but for some reason it does > not work in GoogleAppEngineLauncher. > > Can anyone help me get this running or point me to a more basic > AppEngine application with source that uses the Imaging api? > > Here is the errror I get from the AppEngineLauncher server: > > File "/Users/alok/Desktop/appengine/muvmuv/main.py", line 8, in > >import simplejson > ImportError: No module named simplejson > > Thanks, > Alok > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Writing binary data in the HTTP response
Hi all,
I can't find a way to write arbitrary binary data in my response. I
have a big list of ints that I would like to serve in a response as
compact as possible. For this, I create an array of shorts (my ints
never take more than two bytes). I create my array like this:
from array import array
int_bytes = array('H')
for p in int_data:
int_bytes.append(p)
later on, when I want to write the response, I do this:
self.response.out.write(int_bytes.tostring())
but I get this nasty error:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xf1 in position
0: ordinal not in range(128)
I understand from this that StringIO, of which self.response.out is an
instance, does not accept bytes that cannot be encoded to ascii.
Does anyone know how I can do to write this kind of binary data?
Thank you,
Alfonso
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: Problem with ampersand in query string
I found the root of problem: The assignment setting up the link anchor.href = "javascript:window.location.href='delete?url=" + encodeURIComponent(url) + "'"; is incorrect, because it destroys the effects of encodeURIComponent (the %-sequences are stored as normal characters), so an additional escape is needed to prevent this: anchor.href = "javascript:window.location.href='delete?url=" + escape(encodeURIComponent(url)) + "'"; --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Hello Everyone
Kind of lurking a bit. I'm researching how to fully utilize Google Apps right now and I'm curious about how I can use and leverage App Engine for a site I'm working on. I've got Developing with Google App Engine (Firstpress) on pre-order from Amazon. Seems kind of small if the 100 pages is accurate information. I hope it's useful. http://www.amazon.com/gp/product/1430218312 Anyway, just saying hello. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: App Engine 'session' questions
GAE has no "global memory" - every handler runs in it's own address space. You can persist data in the datastore (perhaps caching with memcache for performance), or in browser cookies. A lot of applications can do with the Users API instead of full-blown sessions. -Nikola On Oct 30, 10:11 pm, sal <[EMAIL PROTECTED]> wrote: > I tried to RTFM a bit on the topic and wasn't able to find what was > needed - if someone with more App Engine experience could point me in > the right direction that would be great! > > I understand that any cloud computing environment needs to allow for > dynamic allocation of computing resources - I am just looking to > discover what some of the operating parameters are so I can avoid > errors with my apps. My questions: > > - Regarding the behavior of regular python data structures: Assume I > have an object (an instance of a class) in global memory (for > performance reasons) which contains various fields holding values. > How long will a single user's web requests hit the same instance of > that object? Will multiple requests ever go to different 'cached' > instances of that object or another instance of that in-memory data > structure located on a different server? Do I need to anticipate > requests coming from the same user going to different web servers that > have independant cached versions of my application's data structures, > simultaneously? > > - How much 'memory' or 'ram' (Regular python memory') can I consume > via my python code... in the form of lists, Strings, dicts, etc. > before I encounter an exception? Is the limit per-user-session or per- > app? > > - I noticed from limited testing that there is at least a small > semblance of a session... what is the token used to arrbitrate them? > ie: does GAE use a browser cookie, the IP of the user - or a cookie > containing the user id? (I want to make sure that I don't invalidate > whatever the mechanic is, so I don't destroy the user's sessions) > > I'll try to keep searching documentation for some answers... if anyone > can point me to any info in the right direction I *greatly* > appreciate, in advance, all your help, > > - Sal --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Help: Problem getting example: MuvMuv working on Google App Engine (simplejson dependancy)
Hi, I am a relatively new Python and web developer and am currently trying to get Imaging working on Google App Engine. I pulled down the MuvMuv sample application code to try and dissect it and borrow some good working code. The problem I am having is that the 'simplejson' module is not importing properly on my local machine. I am not familiar with JSON and was hoping for some sample code to learn from that didn't require me to learn it just yet. I have downloaded and installed 'simplejson' on my local machine using 'python setup.py install' and have confirmed that it works in the Python command line, but for some reason it does not work in GoogleAppEngineLauncher. Can anyone help me get this running or point me to a more basic AppEngine application with source that uses the Imaging api? Here is the errror I get from the AppEngineLauncher server: File "/Users/alok/Desktop/appengine/muvmuv/main.py", line 8, in import simplejson ImportError: No module named simplejson Thanks, Alok --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: could not read from datastore on WindowsXP
Hi guys, Everything seems to be working now. But I'm still having errors if I try to set my own datastore path via--datastore_path . I'm using the default datastore_path. Thanks. erick On Oct 25, 1:37 am, "Marzia Niccolai" <[EMAIL PROTECTED]> wrote: > This looks to be a permissions error, with the dev_appserver not being able > to read/write the datastore files where the dev_appserver stores the > development datastore. > > With what permissions is the datastore running? And what is the permissions > level of the directory to which it is trying to read/write? > > -Marzia > > On Fri, Oct 24, 2008 at 9:27 AM, erick <[EMAIL PROTECTED]> wrote: > > > Hi guys, > > > I can't read datastore via dev_appserver.py: > > WARNING 2008-10-24 16:07:29,250 datastore_file_stub.py] Could not > > read datastore data > > > The application I'm trying to run is the guestbook application which > > is installed in the \demos directory after installing appengine. I did > > not edit any of its content. > > > This is my dev machine: > > WindowsXP SP2 > > Python 2.5 > > appengine release 1.1.5 > > > I read this -http://code.google.com/appengine/docs/thedevwebserver.html > > > Tried most of the options like: > > --datastore_path \some_directory\ > > and I get something like this: > > WARNING 2008-10-24 16:07:29,250 datastore_file_stub.py] Could not > > read datastore data > > > --clear_datastore > > and I get something like this: > > WARNING 2008-10-24 16:14:17,687 dev_appserver.py] Removing file > > failed: [Error 5] Access is denied: > > > even tried changing port (although this may not seem necessary) > > -p > > and I get something like this: > > WARNING 2008-10-24 16:25:08,937 datastore_file_stub.py] Could not > > read datastore > > > I uninstalled appengine and reinstalled from a fresh download. Still > > the same error. > > I've ran into a few discussions here regarding the same issue and most > > of the advice their giving is to simply set the datastore_path. Not > > working in my case. > > > I am a python newbie. Any advice? Thanks. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Add a flag to the appcfg.py to zip/upload the sources
In our project, we pack the javascript and CSS and switch some flags around before uploading the source to the server. So what gets deployed is a bit different from what is on our SVN trunk (but we tag it before uploading). I'm still not sure why you'd want to download your source from GAE, if you can easily and securely check it out from SVN. -Mahmoud On Oct 31, 5:36 am, Sylvain <[EMAIL PROTECTED]> wrote: > It seems that a lot of developers would like to download their full > project (source + static). > Currently, there is no solution (even a source constrol). > > For my project, I use a script like this one (with 7zip): > > --- START BAT --- > 7za a -tzip src\source.zip src\ -xr!*.pyc > appcfg.py update src/ > --- END BAT --- > > APP.YAML --- > - url: /source.zip > static_files: source.zip > upload: source.zip > login: admin > -- > > But I think all this script could be added to the "appcf.py update" > layer with a flag like one "source=source.zip" > > I've created a feature request > :http://code.google.com/p/googleappengine/issues/detail?id=831 > > Regards --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Problem with ampersand in query string
what does the url that your javascript is generating look like? It should look like this: delete?url=http%3A%2F%2Fserver%2Fstuff%3Fa%3Db%26c%3Dd But you say your app is passing "delete?url=http://server/stuff? a=b&c=d" to app engine. If so, it sounds like you have a javascript problem, not a app engine problem. -peter On Oct 31, 5:49 am, gmalquestion <[EMAIL PROTECTED]> wrote: > I have javascript call which deletes an url from the user's > subscription. It looks like this: > > anchor.href = "javascript:window.location.href='delete?url=" + > encodeURIComponent(url) + "'"; > > The problem is the handler on appengine doesn't process the parameter > properly if it has an ampersand in it. For example, the url > > http://server/stuff?a=b&c=d > > is passed to appengine as > > delete?url=http://server/stuff?a=b&c=d > > so it thinks the url parameter ends at the ampersand: > > url=http://server/stuff?a=b > > That's why I used encodeURIComponent, but it didn't help. I also tried > escape and encode, but they didn't work either. > > Can it be a problem on the appengine part (incorrect query parameter > processing) or I should do something else in addition to > encodeURIComponent? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Chat room about Google App Engine...
the general rule on irc is to lurk, ask questions when you have them, and respond when you see someone else. i can only say that adding different channels or chatrooms wont help to increase the traffic in any of them. On Fri, Oct 31, 2008 at 1:59 AM, bvelasquez <[EMAIL PROTECTED]> wrote: > > I went to the channel after setting up Chatzilla, and saw no acivity. > Maybe it's a slow night. What times is more activity seen in the > channel so I know when to login. > > Thanks > > > > On Oct 30, 7:33 pm, "Calvin Spealman" <[EMAIL PROTECTED]> wrote: >> There is already an active channel, #appengine on freenode.net IRC. >> >> >> >> On Thu, Oct 30, 2008 at 8:22 PM, bvelasquez <[EMAIL PROTECTED]> wrote: >> >> > If anyone is interested, I put up a room on TaPingYa (which is my use >> > of Google App Engine), to chat live about Google App Engine >> > development. Of course, I'd love any feedback on the site also. >> >> >http://tapingya.appspot.com/room?room_id=3704 >> >> > If anyone has had issues with high CPU usage, or knows more about it, >> > I would be interested in chatting with you. No amount of optimization >> > seems to help and I got my first "google app is over its quota" >> > message today.. for a call that doesn't do that much. >> >> > Regards, >> >> > Barry >> >> -- >> Read my blog! I depend on your acceptance of my opinion! I am >> interesting!http://techblog.ironfroggy.com/ >> Follow me if you're into that sort of thing:http://www.twitter.com/ironfroggy > > > -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Add a flag to the appcfg.py to zip/upload the sources
It seems that a lot of developers would like to download their full project (source + static). Currently, there is no solution (even a source constrol). For my project, I use a script like this one (with 7zip): --- START BAT --- 7za a -tzip src\source.zip src\ -xr!*.pyc appcfg.py update src/ --- END BAT --- APP.YAML --- - url: /source.zip static_files: source.zip upload: source.zip login: admin -- But I think all this script could be added to the "appcf.py update" layer with a flag like one "source=source.zip" I've created a feature request : http://code.google.com/p/googleappengine/issues/detail?id=831 Regards --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Modeling Hierarchical Data
Store all the parent ids as a hierarchy in a string... h="id1/id2/" h="id1/id2/id3" h="id1/id2/id3/id4" h="id5/" h="id5/id6" You can then filter WHERE h > "id1/" and h < "id5/" to get all children of id1. Or you can use entity groups & ancestors. More details here... http://groups.google.com/group/google-appengine/browse_thread/thread/566d51bf807ae301/102a8f399d260d90#102a8f399d260d90 On Oct 31, 8:31 am, Chris Tan <[EMAIL PROTECTED]> wrote: > I'm wondering how other people have been modeling hierarchical > information > such as nested comments. > > My first thought was to do something like this: > > class Comment(db.Model): > reply_to = db.SelfReferenceProperty(collection_name="replies") > author = db.UserProperty() > content = db.TextProperty() > > However, this isn't scalable, as a query is made for every > comment in the thread. > > Any ideas? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Problem with ampersand in query string
I have javascript call which deletes an url from the user's subscription. It looks like this: anchor.href = "javascript:window.location.href='delete?url=" + encodeURIComponent(url) + "'"; The problem is the handler on appengine doesn't process the parameter properly if it has an ampersand in it. For example, the url http://server/stuff?a=b&c=d is passed to appengine as delete?url=http://server/stuff?a=b&c=d so it thinks the url parameter ends at the ampersand: url=http://server/stuff?a=b That's why I used encodeURIComponent, but it didn't help. I also tried escape and encode, but they didn't work either. Can it be a problem on the appengine part (incorrect query parameter processing) or I should do something else in addition to encodeURIComponent? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: will google think of to support app own unique IP
because "ghs.google.com" can not access in my country. 2008/10/31 David Symonds <[EMAIL PROTECTED]>: > > On Thu, Oct 30, 2008 at 10:03 PM, yu ping322 <[EMAIL PROTECTED]> wrote: > >> will google think of to support app own unique IP? > > I hope not. Why would you want something like that? > > > Dave. > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Modeling Hierarchical Data
I'm wondering how other people have been modeling hierarchical information such as nested comments. My first thought was to do something like this: class Comment(db.Model): reply_to = db.SelfReferenceProperty(collection_name="replies") author = db.UserProperty() content = db.TextProperty() However, this isn't scalable, as a query is made for every comment in the thread. Any ideas? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
