[google-appengine] Re: My benchmarks say query is faster than get

2009-08-07 Thread Oliver Zheng
On Aug 7, 1:33 am, Nick Johnson (Google) nick.john...@google.com wrote: Queries fetch the entities in parallel - as do bulk gets. Thus, the time taken for the entity-fetching part of a query or a bulk get is proportional to the time required to fetch the slowest entity (plus some overhead

[google-appengine] Re: My benchmarks say query is faster than get

2009-08-07 Thread Oliver Zheng
On Aug 7, 1:33 pm, Alkis Evlogimenos ('Αλκης Ευλογημένος) evlogime...@gmail.com wrote: Oh I see you time only the last db.get() call. Nevertheless you are still not comparing apples to apples. In the db.get case you fetch entities and in the query case you fetch keys only. There is a

[google-appengine] Re: My benchmarks say query is faster than get

2009-08-07 Thread Oliver Zheng
On Aug 7, 1:33 pm, Alkis Evlogimenos ('Αλκης Ευλογημένος) evlogime...@gmail.com wrote: Oh I see you time only the last db.get() call. Nevertheless you are still not comparing apples to apples. In the db.get case you fetch entities and in the query case you fetch keys only. There is a

[google-appengine] My benchmarks say query is faster than get

2009-08-06 Thread Oliver Zheng
http://benchmaker.appspot.com/benchmark/bench/ See get 20 vs query 20. I must be doing something wrong here. The data in the datastore is about 1000 objects created with its ID autogenerated. There are no other objects, so these objects have sequentially incremental IDs. What is wrong with my

[google-appengine] Re: My benchmarks say query is faster than get

2009-08-06 Thread Oliver Zheng
On Aug 6, 5:27 pm, Alkis Evlogimenos ('Αλκης Ευλογημένος) evlogime...@gmail.com wrote: You are doing multiple db.get calls while you do a single Query.fetch. But isn't the point of db.get for multiples a direct real-time (i.e. constant time) grab? Where as a query searches through an index

[google-appengine] Re: Limit for db.get?

2009-08-04 Thread Oliver Zheng
it will not be a problem. For n=3 it should work. ( I might remember reading the number 300 sometime in this list for db.get() ) 2009/8/4 Oliver Zheng goo...@oliverzheng.com: I have run into a request too large for db.get (not db.put). It appears that sometimes I may need to request for thousands

[google-appengine] Limit for db.get?

2009-08-03 Thread Oliver Zheng
I have run into a request too large for db.get (not db.put). It appears that sometimes I may need to request for thousands of tiny objects, and this is hitting a barrier. What is the limit and what is it determined by? Thanks, Oliver --~--~-~--~~~---~--~~ You

[google-appengine] How to not dereference ReferenceProperties?

2009-07-06 Thread Oliver Zheng
For a model, I have a ListProperty of Keys, and whenever I access it, it's a list of keys. Great. How can I access a ReferenceProperty of a model and have it return a Key instead of the actual instance (which has to be retrieved from datastore)? I've setup my models by putting a lot of info into

[google-appengine] How does Django (on GAE) handle contrib libs such as User?

2009-06-24 Thread Oliver Zheng
I am currently using app-engine-patch, but now that Django 1.0 has been released on GAE, I'd like to port my app over. App-engine-patch zipimports Django and patches a lot of things to make it work out of the box, for example the User model is modified to use GAE models, and memcache is used for

[google-appengine] How is an entity key generated?

2009-06-12 Thread Oliver Zheng
Is it a hash function in that the resulting string is randomly distributed among its range? It seems like in a distributed hash table like BigTable, this would be how it's implemented. The reason I'm asking is, I want to do a random query. I want to select a random entity from an entity group. I

[google-appengine] Re: How is an entity key generated?

2009-06-12 Thread Oliver Zheng
I forgot to include this question: What is the overhead of querying like this for just the first result, vs just getting with the key? What is the magnitude of difference in time? On Jun 12, 4:22 pm, Oliver Zheng goo...@oliverzheng.com wrote: Is it a hash function in that the resulting string

[google-appengine] Re: PolyModel back references return additional unrequested references

2009-06-03 Thread Oliver Zheng
the specific members of a polymodel You probably need to put together a PolyModel aware version of ReferenceProperty T On Jun 2, 9:31 am, Oliver Zheng goo...@oliverzheng.com wrote: I am noticing this behaviour (which I think is a bug): class A(PolyModel):     pass class B

[google-appengine] Re: Is it a smart idea to group pretty much all of my classes in a single hierchy with PolyModel?

2009-06-03 Thread Oliver Zheng
using a regular db.Model parent class. db.PolyModel adds a listproperty of parent classes to each stored entity, which adds extra indexing overhead, and will in some cases require indexes for queries that otherwise wouldn't. -Nick Johnson On Mon, Jun 1, 2009 at 6:33 PM, Oliver Zheng goo

[google-appengine] Do not update auto_now on save()

2009-06-01 Thread Oliver Zheng
auto_now=True is very handy for most modified timestamp properties. Is it possible to override this behaviour on certain occasions? Thanks, Oliver --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine

[google-appengine] Re: Python: db.ReferenceProperty collection_name callable?

2009-06-01 Thread Oliver Zheng
= db.get(h1) h1.parents.fetch(5) h1.children.fetch(5) I'm guessing the above would be OK if it is acceptable that the Child's reference to House is called 'home' instead of the parent classes 'house'. Thank you, Jeff On May 22, 11:19 am, Oliver Zheng goo...@oliverzheng.com wrote: I'm

[google-appengine] PolyModel back references return additional unrequested references

2009-06-01 Thread Oliver Zheng
I am noticing this behaviour (which I think is a bug): class A(PolyModel): pass class B(B): a = ReferenceProperty(A, collection_name='b') class C(A): a = ReferenceProperty(A, collection_name='c') a = A() a.put() b = B() b.a = a b.put() c = C() c.a = a c.put() Now, a.b returns b

[google-appengine] Is it a smart idea to group pretty much all of my classes in a single hierchy with PolyModel?

2009-06-01 Thread Oliver Zheng
A lot of my models have things like modified_time and created_time. I currently have them setup so they inherit this from a PolyModel class. Is this a smart idea? I see that they are grouped together in the admin page. Performance wise, does this mean they will have to be located on the same

[google-appengine] Re: Calculating percentile

2009-05-26 Thread Oliver Zheng
Perfect. Thanks. I realized that for my needs, logarithmic scaling might suit better. In that case, to calculate frequency distribution, I can pretty much use a fixed number of buckets, since the maximum number is probably only 20 magnitudes larger than the smallest. Cheers, Oliver

[google-appengine] Calculating percentile

2009-05-25 Thread Oliver Zheng
For aggregate functions like sum or average, it has been recommended that they be calculated at the time of saving each new or old entry. But how would one go about calculating percentile? Is it mathematically possible to spread out the calculation of percentiles over new entries as they come

[google-appengine] Re: Calculating percentile

2009-05-25 Thread Oliver Zheng
Also, how would the calculation of frequency distribution be amortized as new values come in? On May 25, 3:25 pm, Oliver Zheng goo...@oliverzheng.com wrote: For aggregate functions like sum or average, it has been recommended that they be calculated at the time of saving each new or old entry

[google-appengine] Re: What's the difference among key, id, and key name?

2009-05-22 Thread Oliver Zheng
Hi Jason, On May 12, 5:12 pm, Jason (Google) apija...@google.com wrote: [...] thereby improving the performance of your application [...] What is the extent of this? I suppose you mean that if you know the key, you can construct an object and save it to datastore by overwriting it, without

[google-appengine] Python: db.ReferenceProperty collection_name callable?

2009-05-22 Thread Oliver Zheng
I'm trying to create a hierchy of kinds, the obvious reason of which is that kinds can share model fields. However, for ReferenceProperties, collection_name is fixed to when the model field is declared in the model class. If this is declared in the parent class, then all the children will have

[google-appengine] What's the difference among key, id, and key name?

2009-05-12 Thread Oliver Zheng
I have been looking at the stored data of some apps, and noticed those 3 columns. Key appears to be a hash/string of some sort. ID is usually empty. Key name looks like an actual readable identifier, but it's usually just key_ + username or something that already exists. What is the use for any

[google-appengine] What's the difference among key, id, and key name?

2009-05-12 Thread Oliver Zheng
I have been looking at the stored data of some apps, and noticed those 3 columns. Key appears to be a hash/string of some sort. ID is usually empty. Key name looks like an actual readable identifier, but it's usually just key_ + username or something that already exists. What is the use for any