[google-appengine] Re: About expando properties
On Jul 14, 9:36 am, someone1 wrote: > Using the simplejson class resulted in code significantly slower > (15-20x slower). However, using the built-in json class in Python 2.7 > resulted in code slightly faster than repr() (3-5x faster). > > With the upcoming support for 2.7, I'd suggest keeping json encoding. > It's also more portable "as-is" although it wouldn't be such a hard > task to switch between the two. Keep in mind that Python 2.7's json module might be using C speedups that might not be available on App Engine, and it's actually the same codebase as simplejson (except that the version in Python 2.7 lags behind simplejson a bit). -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: About expando properties
While this seems to be the case in Python 2.5 (without a built-in json
class), it isn't true in Python 2.7 (from my tests at least):
import string, random, profile, json
#import simplejson as json
def randomString(length):
return ''.join(random.choice(string.ascii_uppercase + string.digits)
for x in range(length))
t = {}
for x in range(500):
t[randomString(8)] = randomString(12)
profile.run("eval(repr(t))")
profile.run("json.loads(json.dumps(t))")
Using the simplejson class resulted in code significantly slower
(15-20x slower). However, using the built-in json class in Python 2.7
resulted in code slightly faster than repr() (3-5x faster).
With the upcoming support for 2.7, I'd suggest keeping json encoding.
It's also more portable "as-is" although it wouldn't be such a hard
task to switch between the two.
On Jul 13, 2:40 pm, Pol wrote:
> Very interesting, thanks for sharing.
>
> If you just need a simple dictionary-like storage, why not just repr()
> and eval() - wouldn't that be even faster?
>
> On Jul 13, 8:05 am, someone1 wrote:
>
>
>
>
>
>
>
> > I just looked into this myself. Best bet is to use a Text or Blob
> > property to store in a JSON format:
>
> >http://kovshenin.com/archives/pickle-vs-json-which-is-faster/
>
> > I did many tests with 250-500 expando properties and the results were
> > horrendous. From digging around I didn't find a way to disable
> > indexing expando properties, and this made writing them very
> > expensive.
>
> > On Jul 13, 1:35 am, Pol wrote:
>
> > > Hi,
>
> > > Regarding expando properties:
>
> > > 1) Can you prevent them from being indexed (I couldn't find anything
> > > about this, but I'd like official confirmation)?
>
> > > 2) What's the practical number of expando properties you shouldn't go
> > > above? If they are all indexed, then surely that can be quite
> > > expensive, so you should stay below 100 or something?
>
> > > 3) If you have to be above the limit defined at #2, what's the best
> > > alternative? Serializing into a blob property, possibly, but what
> > > would be the most optimal serialization format?
>
> > > Thanks!
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: About expando properties
There is an _unindexed_properties private property of model class: class A(db.Expando): pass A._unindexed_properties = frozenset(['prop']) a = A() a.prop = 1 # it's not indexed a.put() -- keakon My blog(Chinese): www.keakon.net Blog source code: https://bitbucket.org/keakon/doodle/ On Thu, Jul 14, 2011 at 2:40 AM, Pol wrote: > Very interesting, thanks for sharing. > > If you just need a simple dictionary-like storage, why not just repr() > and eval() - wouldn't that be even faster? > > On Jul 13, 8:05 am, someone1 wrote: > > I just looked into this myself. Best bet is to use a Text or Blob > > property to store in a JSON format: > > > > http://kovshenin.com/archives/pickle-vs-json-which-is-faster/ > > > > I did many tests with 250-500 expando properties and the results were > > horrendous. From digging around I didn't find a way to disable > > indexing expando properties, and this made writing them very > > expensive. > > > > On Jul 13, 1:35 am, Pol wrote: > > > > > > > > > > > > > > > > > Hi, > > > > > Regarding expando properties: > > > > > 1) Can you prevent them from being indexed (I couldn't find anything > > > about this, but I'd like official confirmation)? > > > > > 2) What's the practical number of expando properties you shouldn't go > > > above? If they are all indexed, then surely that can be quite > > > expensive, so you should stay below 100 or something? > > > > > 3) If you have to be above the limit defined at #2, what's the best > > > alternative? Serializing into a blob property, possibly, but what > > > would be the most optimal serialization format? > > > > > Thanks! > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: About expando properties
Very interesting, thanks for sharing. If you just need a simple dictionary-like storage, why not just repr() and eval() - wouldn't that be even faster? On Jul 13, 8:05 am, someone1 wrote: > I just looked into this myself. Best bet is to use a Text or Blob > property to store in a JSON format: > > http://kovshenin.com/archives/pickle-vs-json-which-is-faster/ > > I did many tests with 250-500 expando properties and the results were > horrendous. From digging around I didn't find a way to disable > indexing expando properties, and this made writing them very > expensive. > > On Jul 13, 1:35 am, Pol wrote: > > > > > > > > > Hi, > > > Regarding expando properties: > > > 1) Can you prevent them from being indexed (I couldn't find anything > > about this, but I'd like official confirmation)? > > > 2) What's the practical number of expando properties you shouldn't go > > above? If they are all indexed, then surely that can be quite > > expensive, so you should stay below 100 or something? > > > 3) If you have to be above the limit defined at #2, what's the best > > alternative? Serializing into a blob property, possibly, but what > > would be the most optimal serialization format? > > > Thanks! -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: About expando properties
Instead of setting an expando property to the property's value, can you set
it to an instance of a Property class? To modify the example in the
documentation:
class Song(db.Expando):
title = db.StringProperty()
crazy = Song(title='Crazy like a diamond',
author=db.StringProperty('Lucy Sky', indexed=False),
publish_date='yesterday',
rating=5.0)
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/2PaNX7b9L4AJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: About expando properties
I just looked into this myself. Best bet is to use a Text or Blob property to store in a JSON format: http://kovshenin.com/archives/pickle-vs-json-which-is-faster/ I did many tests with 250-500 expando properties and the results were horrendous. From digging around I didn't find a way to disable indexing expando properties, and this made writing them very expensive. On Jul 13, 1:35 am, Pol wrote: > Hi, > > Regarding expando properties: > > 1) Can you prevent them from being indexed (I couldn't find anything > about this, but I'd like official confirmation)? > > 2) What's the practical number of expando properties you shouldn't go > above? If they are all indexed, then surely that can be quite > expensive, so you should stay below 100 or something? > > 3) If you have to be above the limit defined at #2, what's the best > alternative? Serializing into a blob property, possibly, but what > would be the most optimal serialization format? > > Thanks! -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: About expando properties
1) text property? not familer with Python, I believe there is a better answer 2) max 5000 indexes are allowed to build on single entity 3) you can try to save all properties (those don't require index) into one single json / xml text property -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/vVSvFS7gP3IJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
