Re: [sqlite] Selecting every nth row efficiently

2012-04-04 Thread Kees Nuyt
On Wed, 4 Apr 2012 10:59:20 -0500, "Jay A. Kreibich" wrote: > Not to mention avg(). > > You might want to have a look at how RRDtool deals with condensing > data. It is common pratice to plot average, min, and max to preserve > outliers, while still showing trends. > > http://oss.oetiker.c

Re: [sqlite] Selecting every nth row efficiently

2012-04-04 Thread Jay A. Kreibich
On Wed, Apr 04, 2012 at 12:05:44PM +0200, Steinar Midtskogen scratched on the wall: > [Jean-Christophe Deschamps] > > > You're going to have at most one random sample in every slice of 320 > > s. The GROUP BY clause will select only one for you and the query can > > be as simple as: > > > > sele

Re: [sqlite] Selecting every nth row efficiently

2012-04-04 Thread Steinar Midtskogen
Let's say that I want to plot the entire column and the plot is 1000 pixels wide. Then I only need 1000 samples, so I could do this: SELECT timestamp, sample FROM mytable GROUP BY timestamp * 1000 / ((SELECT max(timestamp) FROM mytable) - (SELECT min(timestamp) FROM mytable)); (timestamp is the

Re: [sqlite] Selecting every nth row efficiently

2012-04-04 Thread Steinar Midtskogen
[Jean-Christophe Deschamps] > You're going to have at most one random sample in every slice of 320 > s. The GROUP BY clause will select only one for you and the query can > be as simple as: > > select sample from from mytable group by timestamp / 320 order by > timestamp; Ah. I didn't think of

Re: [sqlite] Selecting every nth row efficiently

2012-04-04 Thread Jean-Christophe Deschamps
This is a good suggestion. A drawback is that the interval can't be too small otherwise there is a risk that a sample would be missed. So I will get more samples than I need. In you example, if there is a sample every second more or less, I would usually get 3-4 samples every hour instead of

Re: [sqlite] Selecting every nth row efficiently

2012-04-04 Thread Steinar Midtskogen
[Jean-Christophe Deschamps] >>If your sampling is essentially regular, why not make it >> >>select ... where timestamp % N between min_interval and max_interval >> >> N being the typical time delta of your n rows above and interval >> bounds reducing the possiblity of gross under- and over-samplin

Re: [sqlite] Selecting every nth row efficiently

2012-04-04 Thread Jean-Christophe Deschamps
Hit Send inadvertandly, sorry. My rowid isn't increasing with something predictable, so I can't do something like WHERE rowid % n = 0. I can use WHERE random() % n = 0 giving me sort of what I want (better than row % n, but I still need something better). If your sampling is essentially regul

Re: [sqlite] Selecting every nth row efficiently

2012-04-04 Thread Jean-Christophe Deschamps
I have a big database with timestamps and sensor readings, which I access with SELECT to have gnuplot draw graphs. However, sometimes I have readings every minute and want to plot several years of data, and feeding everything to gnuplot is overkill. In these cases it would be sufficient to sel

[sqlite] Selecting every nth row efficiently

2012-04-03 Thread Steinar Midtskogen
I have a big database with timestamps and sensor readings, which I access with SELECT to have gnuplot draw graphs. However, sometimes I have readings every minute and want to plot several years of data, and feeding everything to gnuplot is overkill. In these cases it would be sufficient to select