Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-04-01 Thread Jim Nasby
On Mar 27, 2011, at 9:43 PM, Tom Lane wrote: >> 1) move the truncating to a new transaction just like we currently do >> toast tables in a separate transaction from the main vacuum. > > +1 if we are going to continue the behavior of allowing other > transactions to kick autovac off the exclusive l

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-04-01 Thread Jan Wieck
On 3/28/2011 12:35 PM, Jan Wieck wrote: On 3/27/2011 10:43 PM, Tom Lane wrote: In particular, I thought the direction Jan was headed was to release and reacquire the lock between truncating off limited-size chunks of the file. If we do that, we probably *don't* want or need to allow autovac

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-28 Thread Jan Wieck
On 3/27/2011 10:43 PM, Tom Lane wrote: In particular, I thought the direction Jan was headed was to release and reacquire the lock between truncating off limited-size chunks of the file. If we do that, we probably *don't* want or need to allow autovac to be booted off the lock more quickly. T

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-28 Thread Jan Wieck
On 3/27/2011 9:51 PM, Robert Haas wrote: On Sun, Mar 27, 2011 at 9:41 PM, Jan Wieck wrote: On 3/27/2011 6:21 PM, Robert Haas wrote: On Sun, Mar 27, 2011 at 3:25 PM, Jan Wieckwrote: Since we are talking about stable releases, I think just releasing and reacquiring the exclusive l

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-27 Thread Tom Lane
Greg Stark writes: > I like all of: > 1) move the truncating to a new transaction just like we currently do > toast tables in a separate transaction from the main vacuum. +1 if we are going to continue the behavior of allowing other transactions to kick autovac off the exclusive lock. However,

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-27 Thread Tom Lane
Robert Haas writes: > On Sun, Mar 27, 2011 at 3:25 PM, Jan Wieck wrote: >> Since we are talking about stable releases, I think just releasing and >> reacquiring the exclusive lock is enough. We can then try to further improve >> things for future releases. > That seems unsafe - things can change

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-27 Thread Robert Haas
On Sun, Mar 27, 2011 at 9:41 PM, Jan Wieck wrote: > On 3/27/2011 6:21 PM, Robert Haas wrote: >> >> On Sun, Mar 27, 2011 at 3:25 PM, Jan Wieck  wrote: >>> >>>  Since we are talking about stable releases, I think just releasing and >>>  reacquiring the exclusive lock is enough. We can then try to fu

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-27 Thread Jan Wieck
On 3/27/2011 6:21 PM, Robert Haas wrote: On Sun, Mar 27, 2011 at 3:25 PM, Jan Wieck wrote: Since we are talking about stable releases, I think just releasing and reacquiring the exclusive lock is enough. We can then try to further improve things for future releases. That seems unsafe - thi

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-27 Thread Robert Haas
On Sun, Mar 27, 2011 at 3:25 PM, Jan Wieck wrote: > Since we are talking about stable releases, I think just releasing and > reacquiring the exclusive lock is enough. We can then try to further improve > things for future releases. That seems unsafe - things can change under you while you don't h

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-27 Thread Greg Stark
On Sun, Mar 27, 2011 at 8:25 PM, Jan Wieck wrote: > > Since we are talking about stable releases, I think just releasing and > reacquiring the exclusive lock is enough. We can then try to further improve > things for future releases. I like all of: 1) move the truncating to a new transaction jus

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-27 Thread Jan Wieck
On 3/27/2011 1:24 PM, Simon Riggs wrote: So we should truncate in 16MB chunks also. On a second though, fiddling with the scan direction is probably too much of a change for back releases anyway. That 8.3/8.4 can get into a situation, where autovacuum causes a 12 minute freeze of a production

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-27 Thread Simon Riggs
On Sun, Mar 27, 2011 at 2:13 PM, Robert Haas wrote: > On Mar 26, 2011, at 4:16 PM, Jan Wieck wrote: >> That was what I meant. Go in steps of 16-64MB backwards and scan from there >> to the current end in forward direction to find a nondeletable block. In >> between these steps, release and reac

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-27 Thread Robert Haas
On Mar 26, 2011, at 4:16 PM, Jan Wieck wrote: > That was what I meant. Go in steps of 16-64MB backwards and scan from there > to the current end in forward direction to find a nondeletable block. In > between these steps, release and reacquire the exclusive lock so that client > transactions ca

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-27 Thread Simon Riggs
On Sat, Mar 26, 2011 at 8:05 PM, Jan Wieck wrote: > On 3/26/2011 12:12 PM, Simon Riggs wrote: >> >> On Sat, Mar 26, 2011 at 2:30 PM, Jan Wieck  wrote: >> >>>  My current idea for a fix is to modify lazy_truncate_heap(). It does >>> acquire >>>  and release the exclusive lock, so it should be possi

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-26 Thread Jan Wieck
On 3/26/2011 3:17 PM, Robert Haas wrote: On Mar 26, 2011, at 1:44 PM, Itagaki Takahiro wrote: On Sun, Mar 27, 2011 at 01:12, Simon Riggs wrote: At the same time I would change count_nondeletable_pages() so that it uses a forward scan direction (if that leads to a speedup). +1. Hmm.

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-26 Thread Jan Wieck
On 3/26/2011 12:12 PM, Simon Riggs wrote: On Sat, Mar 26, 2011 at 2:30 PM, Jan Wieck wrote: My current idea for a fix is to modify lazy_truncate_heap(). It does acquire and release the exclusive lock, so it should be possible to do this in smaller chunks, releasing and reacquiring the lock

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-26 Thread Robert Haas
On Mar 26, 2011, at 1:44 PM, Itagaki Takahiro wrote: > On Sun, Mar 27, 2011 at 01:12, Simon Riggs wrote: >>> At the same time I would >>> change count_nondeletable_pages() so that it uses a forward scan direction >>> (if that leads to a speedup). > > +1. Hmm. That would speed up truncations t

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-26 Thread Itagaki Takahiro
On Sun, Mar 27, 2011 at 01:12, Simon Riggs wrote: >> At the same time I would >> change count_nondeletable_pages() so that it uses a forward scan direction >> (if that leads to a speedup). +1. > Do we need that? Linux readahead works in both directions doesn't it? > Guess it wouldn't hurt too mu

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-26 Thread Simon Riggs
On Sat, Mar 26, 2011 at 2:30 PM, Jan Wieck wrote: > My current idea for a fix is to modify lazy_truncate_heap(). It does acquire > and release the exclusive lock, so it should be possible to do this in > smaller chunks, releasing and reacquiring the lock so that client > transactions can get thei

[HACKERS] Lock problem with autovacuum truncating heap

2011-03-26 Thread Jan Wieck
We have run across a problem with autovacuum that occurs when it can truncate off a large amount of empty blocks. It behaves different in version 9.0 than previous versions. Both behaviors are bad. Consider a relation receives constant inserts/updates that are satisfied using freespace at the