I would feel a lot better about that if A) /all/ products were likely to get 100,000 ratings in a short period of time (some packages just, well, sound sufficiently lame that I bet few people even bother with them when they see the screenshot) and B) there were some mechanism by which I could occasionally (maybe every few months) run a sweep for each package and give it a completely accurate total (a query I'd expect to take no more than a minute). ;( You do make a good point with the significant figure thing, though. -J
-------------------------------------------------- From: "yejun" <[EMAIL PROTECTED]> Sent: Tuesday, November 04, 2008 10:13 AM To: "Google App Engine" <google-appengine@googlegroups.com> Subject: [google-appengine] Re: more complicated counters/ratings (sorting?) > > When someone changes the rating, you just need to calculate the delta > and update the total rating accordingly. > > Update individual record and average rating can not be atomically > synced. Entity group need to be small, because read or write on an > entity group will cause the entire group being serialized and > deserialized. > > Also the average will at most have 3 significant figures, one wrong > update on 1 of a million record won't affect your average value at > all. > > On Nov 4, 1:01 pm, "Jay Freeman \(saurik\)" <[EMAIL PROTECTED]> wrote: >> I'm sorry, I said "products in the same", I meant "individual ratings in >> the >> same", although I think you got it. >> >> I definitely need to have the individual ratings in the same entity group >> as >> the sharded counts or I lost safety updating them. Example: I might >> commit a >> rating (so the user sees his rating if he looks) but fail to update the >> count. If he later goes and changes the rating I would be removing the >> old >> total (which never got counted) and adding the new one, which is >> obviously >> wrong. >> >> This is the comment I was making earlier: these examples of using >> "sharded >> counters" seem to be completely missing on the idea that "the thing you >> are >> counting has to be in the same entity group as the sharded count or your >> risk double or zero counting it". The actual example from Brett is immune >> because he doesn't actually have anything he's counting (just button >> pushes), but if it were blog posts or comments then those items need to >> be >> in the same entity group so you can update the shard's count and add the >> post as a single, atomic transaction. >> >> -J >> >> -------------------------------------------------- >> From: "yejun" <[EMAIL PROTECTED]> >> Sent: Tuesday, November 04, 2008 9:52 AM >> To: "Google App Engine" <google-appengine@googlegroups.com> >> Subject: [google-appengine] Re: more complicated counters/ratings >> (sorting?) >> >> >> >> > You can save your individual rating anywhere, you just need it for the >> > record. >> > Put you update your total rating and total user record in a >> > transaction counter. For average rating you only need total rating and >> > total user counter, you don't need individual record. >> >> > On Nov 4, 12:15 pm, "Jay Freeman \(saurik\)" <[EMAIL PROTECTED]> >> > wrote: >> >> Ok, but this model still involves having the products in the same >> >> entity >> >> group as the sharded ratings, right? That isn't what I read from "use >> >> shards >> >> to track the ratings (but without using transactions)". I can easily >> >> see >> >> how >> >> you don't need a transaction to sum the shards into the total, but you >> >> definitely need a transaction to add the new rating and add its weight >> >> to >> >> the shard. -J >> >> >> -------------------------------------------------- >> >> From: "yejun" <[EMAIL PROTECTED]> >> >> Sent: Tuesday, November 04, 2008 9:11 AM >> >> To: "Google App Engine" <google-appengine@googlegroups.com> >> >> Subject: [google-appengine] Re: more complicated counters/ratings >> >> (sorting?) >> >> >> > You keep total rating values and number of users rated both in two >> >> > shard counters which are not belong to your product entity group. >> >> > Then >> >> > update your average rating for your product's entity group >> >> > periodically like once every 100 request or 1000 request depends on >> >> > your need. So the concurrency level is only determined by the number >> >> > of shards not by number of users or products. >> >> >> > Even though the transfer data from shard counter to your entity >> >> > group >> >> > is non-transactional, but the updating of raw records are >> >> > transactional, the data will be corrected next time when you update >> >> > average rate. Updating will only takes the time of a dozen of >> >> > datastore read and 1 put. >> >> >> > On Nov 4, 11:40 am, "Jay Freeman \(saurik\)" <[EMAIL PROTECTED]> >> >> > wrote: >> >> >> If I don't use transactions (with the ratings in the same entity >> >> >> groups >> >> >> as >> >> >> the shards they are being served by) then I can't be guaranteed I >> >> >> don't >> >> >> accidentally drop or double count ratings in the case of errors. >> >> >> :( Is >> >> >> the >> >> >> idea that you are recommending I just say "oh well, its a drop in >> >> >> the >> >> >> bucket"? -J >> >> >> >> -------------------------------------------------- >> >> >> From: "Alexander Kojevnikov" <[EMAIL PROTECTED]> >> >> >> Sent: Tuesday, November 04, 2008 1:12 AM >> >> >> To: "Google App Engine" <google-appengine@googlegroups.com> >> >> >> Subject: [google-appengine] Re: more complicated counters/ratings >> >> >> (sorting?) >> >> >> >> ...> I would use shards to track the ratings (but without using >> >> >> > transactions) and from time to time re-calculate the average from >> >> >> > the >> >> >> > shards and keep it with the product. This would allow indexing by >> >> >> > the >> >> >> > average rating, without many sacrifices. >> >> >> >> ... > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
