Re: [HACKERS] Reducing stats collection overhead

2007-07-30 Thread Arjen van der Meijden
On 31-7-2007 5:07 Alvaro Herrera wrote: Arjen van der Meijden wrote: Afaik Tom hadn't finished his patch when I was testing things, so I don't know. But we're in the process of benchmarking a new system (dual quad-core Xeon) and we'll have a look at how it performs in the postgres 8.2dev we us

Re: [HACKERS] Reducing stats collection overhead

2007-07-30 Thread Alvaro Herrera
Arjen van der Meijden wrote: > Afaik Tom hadn't finished his patch when I was testing things, so I don't > know. But we're in the process of benchmarking a new system (dual quad-core > Xeon) and we'll have a look at how it performs in the postgres 8.2dev we > used before, the stable 8.2.4 and a

Re: [HACKERS] Reducing stats collection overhead

2007-05-19 Thread Arjen van der Meijden
Afaik Tom hadn't finished his patch when I was testing things, so I don't know. But we're in the process of benchmarking a new system (dual quad-core Xeon) and we'll have a look at how it performs in the postgres 8.2dev we used before, the stable 8.2.4 and a fresh HEAD-checkout (which we'll cal

Re: [HACKERS] Reducing stats collection overhead

2007-05-18 Thread Alvaro Herrera
Tom Lane wrote: > Arjen van der Meijden told me that according to the tweakers.net > benchmark, HEAD is noticeably slower than 8.2.4, and I soon confirmed > here that for small SELECT queries issued as separate transactions, > there's a significant difference. I think much of the difference stems

Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Tom Lane
Gregory Stark <[EMAIL PROTECTED]> writes: > If we want to have an idle_in_statement_timeout then we'll need to introduce a > select loop instead of just directly blocking on recv anyways. Does that mean > we may as well bite the bullet now? If we wanted such a timeout (which I personally don't) we

Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> The first design that comes to mind is that at transaction end >> (pgstat_report_tabstat() time) we send a stats message only if at least >> X milliseconds have elapsed since we last sent one, where X is >> PGSTAT_STAT_INTERVAL or clos

Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Alvaro Herrera
Tom Lane wrote: > The first design that comes to mind is that at transaction end > (pgstat_report_tabstat() time) we send a stats message only if at least > X milliseconds have elapsed since we last sent one, where X is > PGSTAT_STAT_INTERVAL or closely related to it. We also make sure to > flush

Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Lukas Kahwe Smith
Tom Lane wrote: The first design that comes to mind is that at transaction end (pgstat_report_tabstat() time) we send a stats message only if at least X milliseconds have elapsed since we last sent one, where X is PGSTAT_STAT_INTERVAL or closely related to it. We also make sure to flush stats o

Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Gregory Stark
"Tom Lane" <[EMAIL PROTECTED]> writes: > So I think that complicating the design with, say, a timeout counter to > force out the stats after a sleep interval is not necessary. Doing so would > add a couple of kernel calls to every client interaction so I'd really > rather avoid that. > > Any thou

Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Simon Riggs
On Sun, 2007-04-29 at 00:44 -0400, Tom Lane wrote: > The first design that comes to mind is that at transaction end > (pgstat_report_tabstat() time) we send a stats message only if at least > X milliseconds have elapsed since we last sent one, where X is > PGSTAT_STAT_INTERVAL or closely related t

Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Yes, it seems we will have to do something for 8.3. > > Yeah, we've kind of ignored any possible overhead of the stats mechanism > for awhile, but I think we've got to face up to this if we're gonna have > it on by default. > > > I a

Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Yes, it seems we will have to do something for 8.3. Yeah, we've kind of ignored any possible overhead of the stats mechanism for awhile, but I think we've got to face up to this if we're gonna have it on by default. > I assume the method > below would r

Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Bruce Momjian
Yes, it seems we will have to do something for 8.3. I assume the method below would reduce frequent updates of the stats_command_string too. --- Tom Lane wrote: > Arjen van der Meijden told me that according to the tweakers

[HACKERS] Reducing stats collection overhead

2007-04-28 Thread Tom Lane
Arjen van der Meijden told me that according to the tweakers.net benchmark, HEAD is noticeably slower than 8.2.4, and I soon confirmed here that for small SELECT queries issued as separate transactions, there's a significant difference. I think much of the difference stems from the fact that we no