I was just thinking this last night when I wasn't at my computer.
Darn you for publishing my good idea first! :)
It doesn't bring the number of queries down to 1, but it does bring
them down to 2, at least on the request side.
Of course, it also means one extra transactions every time a new
entity is created, so I don't know if there's any net savings, but
this does seem like a reasonable way to address the issue at hand.
-Ben
On Jul 12, 11:08 pm, johnP <j...@thinkwave.com> wrote:
> > Dont know quite how would implement a fifo buffer on AppEngine, would
> > almost certainly use the datastore, so is going to be expensive...
>
> A possibility is to maintain an index, which is a pickled list of
> entity keys, stored as a blob somewhere. You'll need to maintain the
> index by adding/removing keys as you add/remove entities. But then,
> when you need to make a random choice, just unpickle the list, grab
> any five keys from the list, and db.get([keys]).
>
> johnP
>
> On Jul 11, 12:41 pm, Barry Hunter <barrybhun...@googlemail.com> wrote:
>
>
>
> > On 11/07/2009, Devel63 <danstic...@gmail.com> wrote:
>
> > > Barry, I understand your objections below, but do you have a better
> > > approach?
>
> > no, I dont. The way in my first reply is the best I can think off. Not
> > totally uniform, but dont think any 'random' solution is going to be,
> > unless you keep the data uptodate - which is not practical on
> > AppEngine.
>
> > A reasonably good solution, is to use a FIFO buffer, when the queue is
> > empty you get all keys, and shuffle them and add to the queue. then
> > getting random results is a matter of poping that many keys off the
> > queue, and fetching them.
>
> > Dont know quite how would implement a fifo buffer on AppEngine, would
> > almost certainly use the datastore, so is going to be expensive.
>
> > > Assigning random numbers to entities is guaranteed to be worse. If
> > > you are worried about an entity being deleted and opening a gap in the
> > > sequence, imagine the thousand-fold gaps you will see with random ID
> > > generation (e.g. 1, 10001, 10002, 20000, ...).
>
> > > See below.
>
> > > On Jul 10, 3:03 pm, Barry Hunter <barrybhun...@googlemail.com> wrote:
>
> > > > On 10/07/2009, Devel63 <danstic...@gmail.com> wrote:
>
> > > > > The best way is to assign a one-up counter to each record as you
> > > > > create it, then call random.randint(1,max_counter) to determine the
> > > > > desired record.
>
> > > > > To retrieve multiple random entities in a query, do a filter('IN ',
> > > > > [my random nums]).
>
> > > > doent work that well once records start getting deleted (get the same
> > > > issue, non uniform distribution)
>
> > > If an app needs to support entity deletion, you can still ensure
> > > uniformity by running a periodic cron job to compress the counter
> > > sequence.
>
> > > > nor does it work if you filtering at the same time :(
>
> > > Correct, in that the distribution is no longer uniform. But this is
> > > also true of the random ID approach. I admit that the random ID
> > > approach seems appealing at first, but when you actually look into it,
> > > you'll find that you are guaranteed that many results will be 3X more
> > > likely than others, or worse. It IS better in the case that you want
> > > to randomize based on time of entity creation, but there are other
> > > ways to deal with this.
>
> > > I would love it if someone could come up with a good way to do true
> > > random results of an arbitrary query set!!
>
> > > > > Note that behind the scenes this generates multiple queries, so
> > > you're
> > > > > not saving much time.
>
> > > > > On Jul 10, 7:34 am, Wooble <geoffsp...@gmail.com> wrote:
> > > > > > Highly non-optimal solution: have a cron job assign new random
> > > numbers
> > > > > > to your entities often enough to simulate randomness. Even just
> > > re-
> > > > > > assigning numbers to entities that have been previously selected
> > > might
> > > > > > work. This involves a lot more CPU as you'd be doing writes, but
> > > > > > shifts the work from request time to a background process so your
> > > > > > users don't see the added latency for doing N queries.
>
> > > > > > Another possible solution would be to fetch keys only for X*N
> > > entities
> > > > > > (where greater X's produce more apparent randomness) then choose
> > > N of
> > > > > > those keys to actually fetch entities.
>
> > > > > > On Jul 9, 12:33 pm, aloo <aleem.maw...@gmail.com> wrote:
>
> > > > > > > Hi all,
>
> > > > > > > I'm trying to write a GQL query that returns N random records
> > > of a
> > > > > > > specific kind. My current implementation works but requires N
> > > calls to
> > > > > > > the datastore. I'd like to make it 1 call to the datastore if
> > > > > > > possible.
>
> > > > > > > I currently assign a random number to every kind that I put
> > > into the
> > > > > > > datastore. When I query for a random record I generate another
> > > random
> > > > > > > number and query for records > rand ORDER BY asc LIMIT 1.
>
> > > > > > > This works, however, it only returns 1 record so I need to do N
> > > > > > > queries. Any ideas on how to make this one query? Thanks.
>
> > --
> > Barry
>
> > -www.nearby.org.uk-www.geograph.org.uk-
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~----------~----~----~----~------~----~------~--~---
