Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-03-02 Thread Bernd Helmle
--On 28. Februar 2011 15:02:30 -0500 Tom Lane t...@sss.pgh.pa.us wrote: Because it's fifty times more mechanism than we need here? We don't want a SQL interface (not even a lightweight one) and it's unclear that we ever want the data to go to disk at all. I wonder wether a library like

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-03-01 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: The ideal solution would likely be for the stats collector to expose its data structures as shared memory, but I don't think we get to do that under SysV shmem --- it doesn't like variable-size shmem much. Maybe that's another argument for looking harder

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Josh Berkus
Well, what we have now is a bunch of counters in pg_stat_all_tables and pg_statio_all_tables. Right. What I'm saying is those aren't good enough, and have never been good enough. Counters without a time basis are pretty much useless for performance monitoring/management (Baron Schwartz has

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Martijn van Oosterhout
On Mon, Feb 28, 2011 at 10:04:54AM -0800, Josh Berkus wrote: Take, for example, a problem I was recently grappling with for Nagios. I'd like to do a check as to whether or not tables are getting autoanalyzed often enough. After all, autovac can fall behind, and we'd want to be alerted of

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Robert Haas
On Mon, Feb 28, 2011 at 1:04 PM, Josh Berkus j...@agliodbs.com wrote: On the other hand, anything which increases the size of pg_statistic would be a nightmare. Hmm? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Josh Berkus
On 2/28/11 10:24 AM, Robert Haas wrote: On Mon, Feb 28, 2011 at 1:04 PM, Josh Berkus j...@agliodbs.com wrote: On the other hand, anything which increases the size of pg_statistic would be a nightmare. Hmm? Like replacing each statistic with a series of time-based buckets, which would then

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: On 2/28/11 10:24 AM, Robert Haas wrote: On Mon, Feb 28, 2011 at 1:04 PM, Josh Berkus j...@agliodbs.com wrote: On the other hand, anything which increases the size of pg_statistic would be a nightmare. Hmm? Like replacing each statistic with a series of

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Robert Haas
On Mon, Feb 28, 2011 at 1:50 PM, Tom Lane t...@sss.pgh.pa.us wrote: Josh Berkus j...@agliodbs.com writes: On 2/28/11 10:24 AM, Robert Haas wrote: On Mon, Feb 28, 2011 at 1:04 PM, Josh Berkus j...@agliodbs.com wrote: On the other hand, anything which increases the size of pg_statistic would be

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Feb 28, 2011 at 1:50 PM, Tom Lane t...@sss.pgh.pa.us wrote: Ultimately we need to think of a reporting mechanism that's a bit smarter than rewrite the whole file for any update ... Well, we have these things called tables. Any chance of using

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Michael Glaesemann
On Feb 28, 2011, at 14:31, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Feb 28, 2011 at 1:50 PM, Tom Lane t...@sss.pgh.pa.us wrote: Ultimately we need to think of a reporting mechanism that's a bit smarter than rewrite the whole file for any update ... Well, we have

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Josh Berkus
Spitballing here, but could sqlite be an intermediate, compromise solution? For a core PostgreSQL component ?!?!? -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com --

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Joshua D. Drake
On Mon, 2011-02-28 at 11:39 -0800, Josh Berkus wrote: Spitballing here, but could sqlite be an intermediate, compromise solution? For a core PostgreSQL component ?!?!? Sure, why not? It is ACID compliant, has the right kind of license, has a standard API that we are all used to. It seems

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Euler Taveira de Oliveira
Em 28-02-2011 15:50, Tom Lane escreveu: Ultimately we need to think of a reporting mechanism that's a bit smarter than rewrite the whole file for any update ... What about splitting statistic file per database? -- Euler Taveira de Oliveira http://www.timbira.com/ -- Sent via

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Tom Lane
Joshua D. Drake j...@commandprompt.com writes: On Mon, 2011-02-28 at 11:39 -0800, Josh Berkus wrote: Spitballing here, but could sqlite be an intermediate, compromise solution? For a core PostgreSQL component ?!?!? Sure, why not? Because it's fifty times more mechanism than we need here?

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Tom Lane
Euler Taveira de Oliveira eu...@timbira.com writes: Em 28-02-2011 15:50, Tom Lane escreveu: Ultimately we need to think of a reporting mechanism that's a bit smarter than rewrite the whole file for any update ... What about splitting statistic file per database? That would improve matters

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Robert Haas
On Mon, Feb 28, 2011 at 2:31 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Feb 28, 2011 at 1:50 PM, Tom Lane t...@sss.pgh.pa.us wrote: Ultimately we need to think of a reporting mechanism that's a bit smarter than rewrite the whole file for any update