Re: [sqlite] Re: How to optimize a select that gets 17 rows from a 700k row DB (via perl using DBI)?

2007-01-16 Thread Nicolas Williams
On Sat, Jan 13, 2007 at 06:35:20PM -0500, [EMAIL PROTECTED] wrote: > I guess that's the trick, to have the "current" or at least "recent" > database and then the historical one. As of now, the process of polling > the 17 machines takes about 40 seconds or so (when I first started running > the

Re: [sqlite] Re: How to optimize a select that gets 17 rows from a 700k row DB (via perl using DBI)?

2007-01-14 Thread Kees Nuyt
Hi Jonathan, Splitting and automatic updating of history via triggers isn't very complicated. BEGIN TRANSACTION; CREATE TABLE StatsCurrent ( MachineVARCHAR(16) NOT NULL, Load REAL, ScratchCHAR(4), MemINTEGER, MemPctFree INTEGER,

Re: Re: [sqlite] Re: How to optimize a select that gets 17 rows from a 700k row DB (via perl using DBI)?

2007-01-14 Thread Miha Vrhovnik
I'm gonna cut all the content and say just one thing. If Sqlite supported table partitioning this would be piece of cake without any complications. http://www.postgresql.org/docs/8.2/static/ddl-partitioning.html -- It's time to get rid of your current e-mail client ... ... and start using

Re: [sqlite] Re: How to optimize a select that gets 17 rows from a 700k row DB (via perl using DBI)?

2007-01-14 Thread Jay Sprenkle
On 1/13/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: Jay: The closer to real-time, the better. The most often a cron job can run under Linux is minutely, and minutely is pretty good. I guess I could have the summary process occur at the end of the script that polls the machines. It

[sqlite] Re: How to optimize a select that gets 17 rows from a 700k row DB (via perl using DBI)?

2007-01-13 Thread Jonathan . W . Crane
First off, thanks for the help and sorry for the formatting of the message. I didn't know how it was going to turn out and I probably was overly optimistic as well as too verbose. Secondly, as I feared, seems like it was an XY question, so sorry for that as well. I'll address the two replies