[google-appengine] Re: geopt query?
here's an interesting bit of a post from this very group that never
got answer almost a year ago. I've decided to 'bump it', so to speak,
as I judge these kinds of issues seem to have just been "swept under
the rug" every time someone cared to ask about it:
from 'john', on May 28th, 2008:
Hello,
GeoPt is a data type supported by the datastore. I want to query
it.
My first attempt is
class TestHandler(webapp.RequestHandler):
def get(self):
self.response.out.write("Since GeoPt is a data type for the
datastore, can a GAE datastore query be used for a bounding box geo
query? ")
loc1=db.GeoPt(25.1, -125.757)
# loc2=db.GeoPt(49.5,
-69.56)
loc2=db.GeoPt(49.5, -90)
q = kerncomputing.historicplace.HistoricPlace.all()
q.filter("loc > " , loc1 )
q.filter("loc < " , loc2 )
results = q.fetch(20)
i=1
for p in results:
self.response.out.write("%d) name = %s ( " %
(i,p.displayName))
self.response.out.write(p.loc)
self.response.out.write(")")
i += 1
loc1 and the version of loc2 which is commented out are a bounding
box
of the continently USA. If you run that query, all 11 points are
returned as they should be. If you invert the less than and
greater
than operations, the result is empty. So far, so good.
Now consider when loc2 isloc2=db.GeoPt(49.5, -90). I would
expect
all locations on the east coast to be omitted from the results but
all
are listed. Here is the surprising output with GeoPt's.
1) name = Sutter's Fort ( 38.57,-121.470833)
2) name = Boyd Theatre ( 39.951836,-75.167068)
3) name = Charity Hospital and the Adjacent Neighborhood
( 29.955595,-90.0771)
4) name = Great Falls Portage ( 47.503611,-111.286389)
5) name = Streetwall ( 41.884529,-87.624485)
6) name = Statler Hotel ( 32.77993,-96.795379)
7) name = Hanger One, Moffet Field ( 37.415,-122.048333)
8) name = Peace Bridge Neighborhood ( 42.90694,-78.90556)
9) name = The Lower East Side ( 40.7196006775,-73.989402771)
10) name = Sumner Elementary School ( 39.058056,-95.68)
11) name = Vizcaya and Bonnet House ( 25.743611,-80.210278)
Do you see an error or erroneous assumption in my code? Or pointer
to
an example of a bounding box geo query via the datastore would do
the
trick.
Thanks for your time and consideration,
Sincerely,
John
anyone ?? .
On Mar 24, 12:07 am, lock wrote:
> You can do equality type checks, but that's about it :-( .
>
> A quick search of this group for geohash should point you in
> the right direction though.
>
> On Mar 24, 10:41 am, pedepy wrote:
>
>
>
> > I just watched that talk about geo data on youtube (i think it was
> > from google IO?)... and it made me really think ... what queries can
> > be performed on geopt properties, if any ?
>
> > are any kinds of spatial query features planned for gql and the
> > datastore? it seems that right now, these properties are of no
> > worth whatsoever ... other than many saving us from writing 5 or 6
> > lines of code to define them ourselves.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] geopt query?
I just watched that talk about geo data on youtube (i think it was from google IO?)... and it made me really think ... what queries can be performed on geopt properties, if any ? are any kinds of spatial query features planned for gql and the datastore? it seems that right now, these properties are of no worth whatsoever ... other than many saving us from writing 5 or 6 lines of code to define them ourselves. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] jinja2 environment loading on each request to /
hey so i use jinja2 templates because they are better [no citation needed], but from what i understand, loading the jinja2 environment can be quite an expensive task. Through logging, I have found out that each request to / loads the environment over and over.. I have those routines in a seperate python script (not the one that app.yaml fowards requests to for /). anything im missing here ? I think this could be adding also an extra 1000 cpu ms on every requests ... (by the way, i have tried the cookbook's memcache technique and it only works locally, it fails on the google servers..) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: querying on lists...
somehow youre answering a question i had in a different post, which it
turns out was just a typo...
On Mar 10, 12:51 pm, Arun Shanker Prasad
wrote:
> Hi,
>
> Could you post at least a sample of your model?
>
> I have used the GQL query to find if a string is in the kinds list
> property and it's working.
>
> My model sample,
>
> Post(db.Model):
> tag = db.StringListProperty(required=True)
>
> Post.all().filter('tag =', some_tag).get()
>
> Thanks,
> Arun Shanker Prasad.
>
> On Mar 10, 12:41 pm, pedepy wrote:
>
> > ehy all .. it feels like ive already asked this before but i cant
> > remember ..
>
> > anyway, i have entities of a kind that have a stringlistproperty. Im
> > trying to query for entities who's list contain a particular item.
>
> > for instance, entity A's list is foo, bar, bob ... I want to query for
> > entities where the list property has 'bar'. So A should come up. A
> > does not.
>
> > help
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] querying on lists...
ehy all .. it feels like ive already asked this before but i cant remember .. anyway, i have entities of a kind that have a stringlistproperty. Im trying to query for entities who's list contain a particular item. for instance, entity A's list is foo, bar, bob ... I want to query for entities where the list property has 'bar'. So A should come up. A does not. help --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] query vs filter()
is there any difference in performance to use a single query call (like in, "WHERE prop = value AND prop = value, etc" vs running a loop of filter() calls and then executing the query ? seems to me that if you can build your query all in one shot, you can avoid one or many loop(s) in your code ... but everyone seems to be all about using all().filter() these days .. help? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Gettext and GAE/jijas2 integration
just as a side note, loading a jinja2 environment can be quite
expensive. You'll wanna seperate this code (that you will no doubt
reuse extensively) into it's own method. A little memcache love also
wouldnt hurt.
On Mar 4, 7:45 am, Jose wrote:
> I'm new in python and app engine. I'm developing a simple project
> without django, just webapp. Instead use django template system I'm
> using jijas2.
>
> To enable jijas2 into my project I downloaded the last version and
> after unpack it, I copied jijas2 folder into my project. I added this
> imports:
>
> from jinja2 import Template
> from jinja2 import FileSystemLoader, Environment
>
> An in my request handler I use templates in this way:
>
> template_dirs = os.path.join(os.path.dirname(__file__),
> 'templates')
> env = Environment(loader=FileSystemLoader(template_dirs))
> template = env.get_template("index.html")
> rendered = template.render(template_values)
> self.response.out.write(rendered)
>
> Everything works fine. Now, I'm trying to make use of i18n support in
> jijas2. But I have a lot of problems and I would apreciate your help.
>
> First of all I read jijas2 documentation and API, and the way to
> implement this feature is with this lines:
>
> import gettext
>
> (..and in my request handler)
>
> domain = "myproject"
> dirname = os.path.dirname(__file__)
> locales = "en_US"
>
> translations = gettext.find(domain, dirname, locales)
> env = Environment(loader=FileSystemLoader
> (template_dirs),extensions=['jinja2.ext.i18n'])
> env.install_gettext_translations(translations)
>
> Second question:
>
> I have experiencie with PHP, so it not so hard understand gettext and
> po/mo files operation, so I think I have to upload my po files into
> ROOT/locale/LC_MESSAGES/en_US/myproject.po
>
> Regards
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: Is there a handler for logout?
yes i was gonna say, keeping track of the original URL in a logout
operation is probably not important, as you most likely want to simply
redirect to home page. (this can also make the user more 'secure' that
his session has indeed been properly terminated if its important to
your app)..
here's my way, pretty similar:
class LogoutHandler(BaseHandler):
def get(self ):
self.redirect( users.create_logout_url( self.request.get
( 'uri', '/' ) ) )
On Mar 4, 1:45 pm, OliWeiD wrote:
> Hi Paul,
>
> after multiple trials I found a way which is working:
> 1) as link I'm using now /logout
> 2) the following handler is answering to it (typical stuff in app.yaml
> and wsgi application)
> class LogoutHelper(webapp.RequestHandler):
> def get(self):
> usr = users.get_current_user()
> if not usr:
> self.redirect("/")
> return
> # HERE I'M DOING THE CLEANUP STUFF
> url = users.create_logout_url("/")
> self.redirect(url)
>
> Thanks for the hints!
> Kind Regards
> Oliver W-D
>
> On 4 Mrz., 15:31, Paul Roy wrote:
>
> > yep well turns out this is exactly what im doing. but i dont see this
> > as "generating a blank page". i have to try out the code i suggested
> > yesterday.
>
> > and i did make it home, thx :)
>
> > Sent from my iPhone
>
> > On 09-03-04, at 01:28, OliWeiD
> > wrote:
>
> > > Well, I've got the idea.
>
> > > But I put the logout-uri into the html sitting on the page.
> > > With your idea in mind, don't I need a blank page,
> > > doing the del session stuff and then logout, passing the correct
> > > logout uri
> > > as an argument to that blank page??
>
> > > Now I have to go to the train, hopefully you get, where you want to
> > > go.
>
> > > Oliver W.D.
>
> > > On 4 Mrz., 06:26, Paul Roy wrote:
> > >> perhaps, instead of using the users' method directly, create a logout
> > >> handler where you perform the necessary operations before fowarding
> > >> to
> > >> the logout uri.
>
> > >> the tricky part is keeping track of the original uri. the way i do
> > >> it,
> > >> is to pass it as a request argument. you could maybe subclass the
> > >> request handler, with a logout method. such as:
>
> > >> def logout(self, uri=None):
> > >>
> > >> if uri is None: uri = self.request.uri #not sure this part
> > >> works..
>
> > >>
>
> > >> self.redirect(user.logout(uri))
>
> > >> ..
>
> > >> im not 100% sure about this since im just typing this on my phone
> > >> 50ft
> > >> underground in a speeding subway train with nothing to refer to, but
> > >> you should get the idea.
>
> > >> when i get home ill see what code i use exactly.. ;)
>
> > >> ps i actually just missed my stop! lol.. damn you google!! (shakes
> > >> fist)
>
> > >> Sent from my iPhone
>
> > >> On 09-03-04, at 00:01, OliWeiD
> > >> wrote:
>
> > >>> Hi everybody,
>
> > >>> due to security reasons and for efficiency some sort of logout
> > >>> handler
> > >>> is needed.
> > >>> That means, when the user is clicking on the created logout link
> > >>> (users.create_logout_url)
> > >>> During that logout I want to delete a session cookie and delete the
> > >>> memcache entries for that user.
>
> > >>> Any idea?
> > >>> Kind Regards
> > >>> Oliver W-D.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: using keys in an IN statement
uhah yea i believe i am
isnt the proper syntax WHERE property IN (foo, foo, foo) ?
On Mar 3, 2:38 am, ryan wrote:
> On Mar 1, 11:06 am, pedepy wrote:
>
> > I would like to fetch results only against a particular subset of
> > keys. I'm trying to use 'WHERE __key__ IN (:1), list_of_keys ... the
> > gql object accepts that as a valid query, but returns no result..
>
> hmm, __key__ IN ... works for me. fromhttp://shell.appspot.com/:
>
> >>> class Foo(db.Expando):
> pass
> >>> foos = Foo.all().filter('x >', 0).fetch(3)
> >>> keys = [f.key() for f in foos]
> >>> keys
>
> [datastore_types.Key.from_path('Foo', 32425L, _app=u'shell'),
> datastore_types.Key.from_path('Foo', 33376L, _app=u'shell'),
> datastore_types.Key.from_path('Foo', 33399L, _app=u'shell')]>>> got =
> Foo.gql('where __key__ in :1', keys).fetch(3)
> >>> [f.key() for f in got]
>
> [datastore_types.Key.from_path('Foo', 32425L, _app=u'shell'),
> datastore_types.Key.from_path('Foo', 33376L, _app=u'shell'),
> datastore_types.Key.from_path('Foo', 33399L, _app=u'shell')]
>
> out of curiosity, are you actually including those parentheses around :
> 1 in your GQL query?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] using keys in an IN statement
I would like to fetch results only against a particular subset of keys. I'm trying to use 'WHERE __key__ IN (:1), list_of_keys ... the gql object accepts that as a valid query, but returns no result.. If I try to pass a string represenation it doesn't accept that. I guess I could try to search by key name ? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] load test
hey all. Although my app is far from complete, I thought Id do a little early 'load testing' to see what it would look like. I had noticed awful big cpu usage times, usually anywhere between 1100ms to 5000ms on my / page ! ... After some memcaching (I had absolutely none before that), I was able to get it to reduce to the mid hundreds after a few loads. Anyway, here's what i did: My script sent gradual requests to my app, as I understand this is how to best test the load under gae. I dunno if these values are optimal, they were completely random: 10 requests 2 seconds appart, 20 more starting at 2 seconds and ending at 0.25 seconds apart, and then 15 more 'all at once'.I fetched / and then parsed the response for all a & img tags, and followed href's and src images. Then fetched those. I went 2 'levels' deep like that. I did not follow any forms. My first few 'simulated site hits' ran as slow as what I had experienced at first. But quickly enough, I saw cpu processing times drop dramatically. Now, this brought a few questions: first of all, I got an 'too much contention on entity' (or something like that) error, on my first attemps at a load test, which threw 100 requests for everything on my site all at once. I dont know what that error is, nor how do avoid it. second, I noticed that request times read like this /'url' 200'response code' and then a time mesure and then the cpu ms time. What is that first time mesure ? Is it the total amount of time spent ? Does it take into account any kind of latency on the client's side of things or what not or does this value indicate server ressources usage only? anyway, id like your input on that. in the mean time ill keep optimizing. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Entity Equality
here's what i had to add to get set of queries to intersect: def __hash__(self): return hash(self.key()) def __eq__(self, other): return self.key() == other.key() def __ne__(self, other): return not self == other i dunno if thats optimum but it works .. Note that this will cause a query to fetch the entire results from the datastore. You dont want that unless you have absolutely no other solution. (I ditched a huge portion of my initial code that used sets in favor of a few more queries, and was able to optimize the CPU time for my requests a whopping 600%! .. thats how expensive those operations were. unfortunately this has exposed another major flaw i am now in the process of trying to fix. but that story is for another time.) if youre simply trying to query for results that include all but ONE particular entry, wouldn't 'WHERE __key__ != :1', direct2.key() ... do the trick ? im no expert really. just my two cents. On Feb 22, 12:14 pm, Paul Roy wrote: > i believe this as to do with how object instances are evaluated for > equality in python. > > ive had a similar issue and if i recall correctly (im on my phone atm > and not close to my code), i needed to implement __eq__ & __ne__ (or > whatever non equal is, i cant remember) to my Model object subclasses. > > Sent from my iPhone > > On 09-02-21, at 20:51, MajorProgamming wrote: > > > > > I currently have a need to take a result from a query (which is a > > list) and remove one entity from it. For some reason though when I > > perform the remove, I get an error which says that the entity does not > > exist on the list. How can I do this correctly? > > > Code: > > #excerpt: > > #wx and wx2 are db.Query objects with filters... > > #I want to remove wx2 from wx > > direct=wx2.get() > > wx=wx.fetch(limit=50) > > wx.remove(direct) > > #this code returns an error. I know for a fact (through debugging) > > that wx2 entity exists in the wx result. > > > Thanks, --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: indexes for expando objects
This is not the main reason why i chose to go with expando objects, but it does a little like this: In expando's, I can store just information i need; that is, if I happen to have a 'color' attribute for a given entry, I can just store it's 'color' information even if others dont have it; Also, in expando's I can store list of lists by using setattr(self, property, listoflist). Model objects seem to not like the explicit setting of ListProperty(list). but maybe I got this backwards. any input would be appreciated. (ps, im rewritting the core portion of this anyway, as there is no obvious scalable workaround for my indexing problem. I will just have to choose a list of commonly used properties and build indexes against those, and dump everything else in a single predefined property (and index that one as well), as to never raise an index not found error in production). On Feb 21, 10:35 pm, bd_ wrote: > On Feb 20, 3:13 pm, pedepy wrote: > > > Hi .. I just stumbed upon a NeedIndexError. I understand why it > > happens, but I dont quite understand the 'motivations' behind it. > > > If the development server can just generate indexes for queries as I > > make them, why would that not be possible for the deployed app? My app > > makes extensive use of Expando objects who's properties I cannot fully > > predict. As such, the combination of all possible queries is also > > almost possible to determine before hand. > > The development server does not generate indexes, actually. Whenever > you do a query, it loops over every item in the datastore, filtering > for those that match your query. Obviously, this scales rather badly, > but on the plus side, dealing with a query that would otherwise use a > new index is trivial :) > > Actually generating a new index requires sorting every item of the > model in question and building an index out of that. As Dan points > out, that can be a very expensive operation, and moreover ongoing > costs for inserts/updates should be a concern. > > Since you've not given any details about /why/ you're using Expando > objects, all I can say is that you'll have to get clever with one or > two special properties to manufacture whatever queries you want. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] indexes for expando objects
Hi .. I just stumbed upon a NeedIndexError. I understand why it happens, but I dont quite understand the 'motivations' behind it. If the development server can just generate indexes for queries as I make them, why would that not be possible for the deployed app? My app makes extensive use of Expando objects who's properties I cannot fully predict. As such, the combination of all possible queries is also almost possible to determine before hand. What .. am I stuck here? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
