Re: [sqlite] Efficient random sampling in a large table using builtinfunctions.

2012-03-08 Thread Benoit Mortgat
On Thu, Mar 8, 2012 at 15:02, Igor Tandetnik wrote: > Benoit Mortgat wrote: >> * Generate N, random, row numbers between 1 and (SELECT COUNT(*) FROM >>   the_table_name). Maybe using remainder operator % and builtin ABS() >>   and RANDOM() functions can help (see below) * SELECT FROM the_table >>

Re: [sqlite] Efficient random sampling in a large table using builtinfunctions.

2012-03-08 Thread Nico Williams
On Thu, Mar 8, 2012 at 8:02 AM, Igor Tandetnik wrote: > You can implement this algorithm in your application's code, if you are so > inclined. Just prepare a query lile > > select * from the_table where rowid=?; > > then run it N times, binding different random numbers to the parameter. Of > cou

Re: [sqlite] Efficient random sampling in a large table using builtinfunctions.

2012-03-08 Thread Igor Tandetnik
Benoit Mortgat wrote: > I have a table with millions of records. > > When I run a query with > >ORDER BY random() LIMIT N; > > the RANDOM() function is evaluated against all rows of my result set, > then sorting occurs, and as a result the query is slow. > > In this case the query could be