Started working on my first app-engine app and have a question about
how to organize my data.
Here's a highly simplified version of one part of my app:
for any url, the app generates a bunch of tags (about 100 - 300 tags
per url, although most urls will
have tags in the lower part of the range).
I am considering 2 options, but both have (I suspect) prohibitive
downsides.
Option 1, Relationship model:
create class TagUrl
to write:
TagUrl(tag=some_tag, url=some_url).put()
TagUrl(tag=some_other_tag, url=some_url).put()
etc., for each tag-url combo
That is a *lot* of writes to the datastore for each url and will
probably cause me to burn through my GAE quotas in a hurry. The reads
- to find all urls that match a tag - should be reasonably efficient.
So, high write costs, low read costs.
Option 2, use list properties (derived from Brett Slatkin's Building
Scalable, Complex app... talk):
create 2 classes, URL and Tags:
class URL(db.Model):
url = db.StringProperty()
class Tags(db.Model):
tags = db.StringListProperty()
to write:
store a list of tags with a url key as parent
to read:
indexes = GqlQuery("SELECT __key__ from Tags WHERE tags = :1" ,
target_tag)
keys = [k.parent for k in indexes]
urls = db.get(keys)
This option is better, but how efficient are the reads, really? For a
1000 urls, there will be a 1000 lists of tags which will have to be
traversed to find the target_tag. Assuming each list has 100 or so
elements, that's an awful lot of reading. The problem worsens,
obviously, for more urls (as is likely to be the case in my app) and
longer lists.
Anyone have a solution for this? I am leaning towards option 2, but
worry about how scalable it really is.
- Shailen Tuli
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~----------~----~----~----~------~----~------~--~---
