Thanks Barry/Nick. I was not aware of either projects and looks like the geomodel project will probably be a better fit for what I am trying to do. I will try to integrate it to see if it works for me.
Thanks again! On Jun 18, 2:51 am, "Nick Johnson (Google)" <nick.john...@google.com> wrote: > Hi ecognium, > > > > On Wed, Jun 17, 2009 at 9:02 PM, ecognium <ecogn...@gmail.com> wrote: > > > Thanks, Nick. Yes I am already using a similar approach to paging. I > > did not know about this module, which probably can replace what i have > > written so will definitely look into that. Btw, the link to the > > pager.py file is down so here the google cache link to the required > > class: > > >http://74.125.155.132/search?q=cache:23LPEeO2oHIJ:https://bitbucket.o... > > > I also grabbed the source and put it on drop.io:http://drop.io/pagerquery > > > Regarding the zip based filtering I was trying to do something like: > > assuming I know the lat/long of the zip code of interest then this > > simple query will get nearby zip codes (square region), which can be > > filtered down by using a great circle algo... > > > SELECT * FROM ZipData WHERE latitude >= x AND latitude <= y AND > > longitude >= r AND longitude <= s > > > As I understand the above query cannot be executed on Datastore due to > > the multiple inequality operator use. If you think there is another > > way to get the same result (that will work with datstore), please let > > me know. > > This query can't be executed efficiently with standard indexes on _any_ > database. A relational database using standard indexes will satisfy this > query by doing an index scan on one inequality (eg, the latitude), then > filtering by the other (eg, the longitude). In the worst case, this entails, > for example, retrieving and filtering a slice of the entire west cost of the > US, or everything near the equator, just to get a few points in one city! > You can, of course, do this in App Engine if you want, but there's a better > solution... > > The solution - both for relational databases and App Engine - is to use > spatial indexing. There are a number of Python libraries that provide > spatial indexing for App Engine. The best is undoubtedly Roman Nurik's > geomodel:http://code.google.com/p/geomodel/ > > -Nick Johnson > > > > > > > #2: > > Yup, I meant just keeping the session. I did not think of memcache > > and was just thinking about how the application itself is cached. It > > makes more sense just to use memcache so I can control the process. > > Thanks for your suggestion. > > > -e > > > On Jun 17, 3:09 am, "Nick Johnson (Google)" <nick.john...@google.com> > > wrote: > > > Hi ecognium, > > > > On Wed, Jun 17, 2009 at 9:17 AM, ecognium <ecogn...@gmail.com> wrote: > > > > > Hi All, > > > > My application requires certain types of query features that are > > > > not currently possible through Datastore API and so I am thinking of > > > > moving the query side of things to Anazon's SimpleDB (mainly use it to > > > > return keys). For example, I would like to return entities that are > > > > within a certain zip code range while supporting pagination. Since key > > > > based paging takes the only inequality operator allowed, it is not > > > > possible to do the traditional zip based retrieval. Even without > > > > paging there is no way to have two inequality operator. Hence the > > > > switch to SimpleDB -- let me know if there are any nice workarounds > > > > for zip-based data retrieval. > > > > You may want to check this out: > >http://appengine-cookbook.appspot.com/recipe/efficient-paging-for-any... > > > > > I have two questions for App Engine members: > > > > > 1. Have you noticed any major latency issues in accessing SimpleDB > > > > from App Engine (thinking of using Boto module)? If so, any tips on > > > > how to reduce it? > > > > I haven't personally used SimpleDB, but the same caveats apply as with > > any > > > other service accessed over HTTP - latency is dependent on the service > > and > > > its proximity. > > > > > 2. When I tested SimpleDB from my dev machine, I noticed SimpleDB > > > > takes up to 5 seconds to return results -- most of the time is > > > > actually spent in authorizing the request. So I would like to initiate > > > > the connection once in the app and reuse the object for all subsequent > > > > queries. Where should I do this initialization? I am not familiar with > > > > how App Engine caches the application. Should I create a > > > > amazon_login.py, include the logic for auth (two lines of code) and > > > > import the file in my code? or do I need to put it in a class and > > > > instantiate the class in the same file? > > > > When you say "initiate the connection once", are you referring to an > > actual > > > TCP connection, or to a 'session'? I presume the latter, since SimpleDB > > is > > > HTTP based. > > > > urlfetch doesn't let you control the lifetime of the underlying TCP > > > connection for HTTP requests. If you're obtaining an authentication > > token, > > > though, you can certainly do that on the first request to a given > > runtime, > > > and then cache the result in a global or class-level variable, or cache > > it > > > in memcache and reuse it across multiple instances. > > > > -Nick Johnson > > -- > Nick Johnson, App Engine Developer Programs Engineer > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: > 368047 --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
