On Wed, Aug 12, 2009 at 2:47 PM, Markitecht<markite...@gmail.com> wrote:
>
> Wait a second, am I right in sort of realizing that BigTable doesn't
> support rather basic database functions like counting and grouping
> entities based on certain criteria?
That's correct, because they can't be implemented in a scalable manner.
> In my example, I can implement the suggestion, but it seems silly; if
> i am tracking events that have several pieces of metadata ('target' of
> the action the 'context' within which the user took the action, etc),
> i can't easily provide views of, for example, which 'targets' were
> most popular or which 'contexts' were most popular for a given target,
> right? I have to be maintaining indices in effect to measure all this
> extremely basic data?
Yes, if you want to show counts for large number of items, you have to
maintain those counts incrementally. This is the only efficient way of
doing so - "SELECT count(*) ..." takes time proportional to the number
of records being counted, which obviously doesn't scale well when
you're counting millions or tens of millions of records on every
request.
> As amazingly cool as App Engine is, is it not capable of performing
> very basic database query functions? It seems to me that making a data
> model less relational is a great idea for scaling, but why on Earth
> would simple count and group queries not be supported?
>
> Forgive my noob-ness ;-)
>
> On Aug 10, 9:10 am, Andy Freeman <ana...@earthlink.net> wrote:
>> > Finally, do look at the info on sharded counters if you expect
>> > individual counters to be updated more than about once a second.
>>
>> That is fairly disturbing advice as it seems to suggest that updating
>> a simple entity more than once a second is problematic.
>>
>> I would have thought that it would be safe to update a given entity
>> 10-15 times/second and maybe even 20x/second. (10x/second is 100ms per
>> update, 15 is over 65ms per update, 20 is 50ms per update.) Frankly,
>> I'm surprised that 5 times/second is too fast. (5x/second is 200ms per
>> update.)
>>
>> On Aug 10, 5:24 am, "Nick Johnson (Google)" <nick.john...@google.com>
>> wrote:
>>
>> > On Sun, Aug 9, 2009 at 4:58 PM, Markitecht<markite...@gmail.com> wrote:
>>
>> > > Thanks Nick, makes complete sense.
>>
>> > > I'll write it so that for each Interaction, it finds the appropriate
>> > > entity, increments the counter and stores it; if looking for the
>> > > entity turns nothing up, i make a new one with a counter set to 1.
>>
>> > > right?
>>
>> > Right. Just make sure to do it inside a transaction if you need exact
>> > counts. And use key naming to avoid the need to do queries.
>>
>> > Finally, do look at the info on sharded counters if you expect
>> > individual counters to be updated more than about once a second.
>>
>> > -Nick Johnson
>>
>> > > thanks again for the quick and kind attention.
>>
>> > > best,
>> > > Christopher
>>
>> > > On Aug 7, 6:02 am, "Nick Johnson (Google)" <nick.john...@google.com>
>> > > wrote:
>> > >> Hi Markitecht,
>>
>> > >> It sounds like your best option is to have a single Interaction entity
>> > >> for each unique string. You can use the key name to ensure uniqueness.
>> > >> Then, to record a new interaction, in a transaction fetch the existing
>> > >> one (if any), increment the count, and store it.
>>
>> > >> If you expect some interactions to be very popular (more than a few
>> > >> updates a second), you should probably look into sharded counters.
>>
>> > >> -Nick Johnson
>>
>> > >> On Wed, Aug 5, 2009 at 7:15 PM, Markitecht<markite...@gmail.com> wrote:
>>
>> > >> > I am writing a dirt-simple tracking API.
>>
>> > >> > For the sake of explanation, i will over-simplify my question even
>> > >> > further.
>>
>> > >> > I have an endpoint that accepts one item of string metadata, and saves
>> > >> > a new instance of an Interaction object.
>>
>> > >> > (the interaction object also saves the user and the date created)
>>
>> > >> > How do i query Interaction to return the most popular
>> > >> > 'interactions' (using those string metadata values), with a count for
>> > >> > each?
>>
>> > >> > This seems *so* simple, but i just can't figure out how to do it on
>> > >> > AE.
>>
>> > >> > Thanks,
>> > >> > Christopher
>>
>> > >> --
>> > >> Nick Johnson, Developer Programs Engineer, App Engine
>>
>> > --
>> > Nick Johnson, Developer Programs Engineer, App Engine- Hide quoted text -
>>
>> > - Show quoted text -
> >
>
--
Nick Johnson, Developer Programs Engineer, App Engine
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~----------~----~----~----~------~----~------~--~---
