Hi,
This solution doesn't seemto work because
". in addition, the key_name of a keyword must be the actual
keyword" isn't possible as I will have duplicate keywords and the key
name is actually just used as part of the key so data gets converted
to a string such as "gZid29zZW1yGQsSBkRvbWFpbiINYnVpbHR3aXRoLmNvbQw"
as a key value.
Gary
On Aug 25, 9:18 pm, Philippe <philippe.cr...@gmail.com> wrote:
> another option could be:
> Record(db.Model):
> value = db.StringProperty()
>
> Keyword(db.Model):
> value = db.StringProperty() #this value is not necessary, but I do
> not know if you can have a Model without properties
>
> the idea is that for one Record, you input several keywords has record
> children. in addition, the key_name of a keyword must be the actual
> keyword
> then, you can simply get_by_key_name() the keyword. with a
> keys_only=true.
> and thus, ask the key.parent() to get the correct Record.key().
> finally, db.get(that reccord key).
>
> a db.get() takes less than 100ms. I read somewhere that a get(keys) is
> always faster and will not get slower if your number of entities
> increase.
>
> On Aug 25, 11:52 am, Gary <gbre...@gmail.com> wrote:
>
> > Great,
>
> > This is how I've done it, I've used the filter to do some testing and
> > with only 32,424 values and appoximately 500,000 keywords a search is
> > taking 3000ms CPU - will this stay the same as my datastore expands to
> > 4.5 million values and potentially 100 million keywords?
>
> > Gary
>
> > On Aug 18, 10:42 pm, Wooble <geoffsp...@gmail.com> wrote:
>
> > > You can do it like (in python):
>
> > > Record(db.Model):
> > > value = db.StringProperty()
> > > keywords = db.StringListProperty()
>
> > > No IN query is necessary to find values with a specific keyword or
> > > even multiple keywords, just do:
> > > filteredquery = Record.all().filter("keywords =", "mykeyword").filter
> > > ("keywords =", "myotherkeyword")...
>
> > > You can basically add as many .filter()s as you want thanks to merge
> > > join. Normalizing would require an exploding index if you ever want
> > > to search records having more than 1 given keyword. You might,
> > > however, want to optimize a bit for space by saving lists of integers
> > > and mapping those integers to keywords, although you'll end up with a
> > > performance hit when you search and storage is relatively cheap.
>
> > > On Aug 18, 2:26 am,garazy<gbre...@gmail.com> wrote:
>
> > > > Hi,
>
> > > > I want to store 4.5 million data records that have a string identifier
> > > > which each have 50 keywords associated with them.
>
> > > > For example,
>
> > > > value (string), keyword1 (string) ... keyword50 (string)
>
> > > > In SQL Server 2008 in a traditional database the best way, that I
> > > > found, was to store this is 3NF such as
>
> > > > value_id (int)
> > > > value (string)
> > > > |
> > > > |
> > > > value_id (int)
> > > > keyword_id (int)
> > > > |
> > > > |
> > > > keyword_id (int)
> > > > keyword_value (string)
>
> > > > The data is mostly static, I typically use queries where to find
> > > > values which use specific keywords, which in SQL uses IN queries,
> > > > however I'm not sure if this is the best approach for the app engine
> > > > datastore.
>
> > > > Do people recommend the data stored as-
>
> > > > value, keyword0, keyword1, keyword2 etc..
>
> > > > or in the same sort of way as I store it in SQL Server?
>
> > > > What would the performance be using the app engine be for this store
> > > > of datastore if I wanted to find values which have say 2 specific
> > > > keywords?
>
> > > > Thanks,
>
> > > > Gary
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~----------~----~----~----~------~----~------~--~---
