Re: [HACKERS] Problem with 8.4 stats collector high load

2010-02-16 Thread Jakub Ouhrabka
> This would guarantee that autovacuum is fired no later than > autovacuum_naptime after the condition for the run became true. Of course, this unfortunately not true... The guarantee is 1,5x autovacuum_naptime. But I'd be happy with it but I agree that's not what I'd as a user expect from this

Re: [HACKERS] Problem with 8.4 stats collector high load

2010-02-16 Thread Jakub Ouhrabka
> well, my current opinion is that we should spend some nonzero amount > of thought into figuring out what to do. I'd suggest to do it like this: Do autovac_refresh_stats() once per autovacuum_naptime/2 and share the result among all autovacuum workers. This would guarantee that autovacuum is

Re: [HACKERS] Problem with 8.4 stats collector high load

2010-02-16 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > Jakub Ouhrabka wrote: > >> Was autovacuum requesting to write this 20MB file 650x per minute? > > > Yes, exactly. > > > Ideally, autovacuum would only request a new copy of the file if the one > > it got was considerably out of date. Obviously a tent

Re: [HACKERS] Problem with 8.4 stats collector high load

2010-02-16 Thread Tom Lane
Alvaro Herrera writes: > Jakub Ouhrabka wrote: >> Was autovacuum requesting to write this 20MB file 650x per minute? > Yes, exactly. > Ideally, autovacuum would only request a new copy of the file if the one > it got was considerably out of date. Obviously a tenth of a second is > not old enoug

Re: [HACKERS] Problem with 8.4 stats collector high load

2010-02-16 Thread Alvaro Herrera
Jakub Ouhrabka wrote: > > Ideally, autovacuum would only request a new copy of the file if the > > one it got was considerably out of date. Obviously a tenth of a > > second is not old enough. > > I've tried to look at it and found that's already implemented - see > autovac_refresh_stats(). STATS

Re: [HACKERS] Problem with 8.4 stats collector high load

2010-02-16 Thread Jakub Ouhrabka
> Ideally, autovacuum would only request a new copy of the file if the > one it got was considerably out of date. Obviously a tenth of a > second is not old enough. I've tried to look at it and found that's already implemented - see autovac_refresh_stats(). STATS_READ_DELAY which is set to 1s.

Re: [HACKERS] Problem with 8.4 stats collector high load

2010-02-16 Thread Alvaro Herrera
Jakub Ouhrabka wrote: > > Maybe you should decrease naptime a bit. > > That did the trick, thanks! > > > Yes. There were some changes that needed to be done to autovacuum so > > that it didn't read the stats file too often, but I don't recall if I > > got around to it. > > I looked at the strac

Re: [HACKERS] Problem with 8.4 stats collector high load

2010-02-16 Thread Jakub Ouhrabka
> Maybe you should decrease naptime a bit. That did the trick, thanks! > Yes. There were some changes that needed to be done to autovacuum so > that it didn't read the stats file too often, but I don't recall if I > got around to it. I looked at the strace output and there are *writes* to the

Re: [HACKERS] Problem with 8.4 stats collector high load

2010-02-16 Thread Euler Taveira de Oliveira
Jakub Ouhrabka escreveu: > These databases are archive databases, so there is no user activity - no > connected users. But the stats collector generates load - 20-40% of > modern 2.8GHz core all the time. > Did you try to set stats_temp_directory in a RAM based filesystem? > Any clues what does i

Re: [HACKERS] Problem with 8.4 stats collector high load

2010-02-16 Thread Alvaro Herrera
Jakub Ouhrabka wrote: > > You might want to try setting log_autovacuum_min_duration=0 in the > > postgresql.conf > > Thanks, tried it. There is nothing in the log - the actual > vacuum/analyze commands are not run (as there is no query activity). > I suspect that autovacuum is checking each databa

Re: [HACKERS] Problem with 8.4 stats collector high load

2010-02-16 Thread Jakub Ouhrabka
> You might want to try setting log_autovacuum_min_duration=0 in the > postgresql.conf Thanks, tried it. There is nothing in the log - the actual vacuum/analyze commands are not run (as there is no query activity). I suspect that autovacuum is checking each database if it should run - and deci

Re: [HACKERS] Problem with 8.4 stats collector high load

2010-02-15 Thread Greg Smith
Jakub Ouhrabka wrote: I've found similar reports but with older versions of postgres: http://old.nabble.com/100--of-CPU-utilization-postgres-process-tt27302021.html Those all looked like a FreeBSD issue, doubt it's related to yours. The pgstat.stat is ~20MB. There are 650 databases, 140GB t

[HACKERS] Problem with 8.4 stats collector high load

2010-02-15 Thread Jakub Ouhrabka
Hi, sorry for repost but previous message didn't get through. So I'm trying another list and sending without attachment which I can send privately upon request (strace output mentioned below). We've migrated some of our databases to 8.4 cluster (from 8.2 and older versions). These database