Re: [HACKERS] ExclusiveLock on PostgreSQL - Fabio Mendonça

2015-10-30 Thread Fabio Oliveira De Mendonca
...@mit.edu; pgsql-hackers@postgresql.org; fabio.mendonca@gmail.com Assunto: Re: [HACKERS] ExclusiveLock on PostgreSQL - Fabio Mendonça On Wed, Oct 28, 2015 at 5:59 PM, Fabio Oliveira De Mendonca wrote: > I 've a process with 600.000 rows, for insert on table "A" with 130 columns

[HACKERS] Re: [HACKERS] ExclusiveLock on PostgreSQL - Fabio Mendonça

2015-10-30 Thread Robert Haas
On Wed, Oct 28, 2015 at 5:59 PM, Fabio Oliveira De Mendonca wrote: > I 've a process with 600.000 rows, for insert on table "A" with 130 columns > and I'm received the "Exclusivelock" error message, making lost some > rows during transaction. The insert of transaction occurs on each 2 min. >

Re: [HACKERS] ExclusiveLock on extension of relation with huge shared_buffers

2014-12-28 Thread Borodin Vladimir
25 окт. 2014 г., в 4:31, Jim Nasby написал(а): > Please don't top-post. > > On 10/24/14, 3:40 AM, Borodin Vladimir wrote: >> I have taken some backtraces (they are attached to the letter) of two >> processes with such command: >> pid=17981; while true; do date; gdb -batch -e back >> /usr/pgsq

Re: [HACKERS] ExclusiveLock on extension of relation with huge shared_buffers

2014-10-24 Thread Jim Nasby
Please don't top-post. On 10/24/14, 3:40 AM, Borodin Vladimir wrote: I have taken some backtraces (they are attached to the letter) of two processes with such command: pid=17981; while true; do date; gdb -batch -e back /usr/pgsql-9.4/bin/postgres $pid; echo; echo; echo; echo; sleep 0.1; done

Re: [Testperf-general] Re: [HACKERS] ExclusiveLock

2004-11-24 Thread Kenneth Marshall
On Wed, Nov 24, 2004 at 11:00:30AM -0500, Bort, Paul wrote: > > From: Kenneth Marshall [mailto:[EMAIL PROTECTED] > [snip] > > The simplest idea I had was to pre-layout the WAL logs in a > > contiguous fashion > > on the disk. Solaris has this ability given appropriate FS > > parameters and we > >

Re: [Testperf-general] Re: [HACKERS] ExclusiveLock

2004-11-24 Thread Bort, Paul
Title: RE: [Testperf-general] Re: [HACKERS] ExclusiveLock > From: Kenneth Marshall [mailto:[EMAIL PROTECTED]] [snip] > The simplest idea I had was to pre-layout the WAL logs in a > contiguous fashion > on the disk. Solaris has this ability given appropriate FS > parameters

Re: [Testperf-general] Re: [HACKERS] ExclusiveLock

2004-11-23 Thread Kenneth Marshall
On Tue, Nov 23, 2004 at 12:04:17AM +, Simon Riggs wrote: > On Mon, 2004-11-22 at 23:37, Greg Stark wrote: > > Simon Riggs <[EMAIL PROTECTED]> writes: > > > > > - Find a way to reduce rotational delay when repeatedly writing last WAL > > > page > > > > > > Currently fsync of WAL requires the

Re: [Testperf-general] Re: [HACKERS] ExclusiveLock

2004-11-23 Thread Bort, Paul
Title: RE: [Testperf-general] Re: [HACKERS] ExclusiveLock > From: Doug McNaught [mailto:[EMAIL PROTECTED]] > > "Bort, Paul" <[EMAIL PROTECTED]> writes: > > >    One other thought: How does static RAM compare to disk > speed nowadays? > >    A 1Gb

Re: [Testperf-general] Re: [HACKERS] ExclusiveLock

2004-11-23 Thread Doug McNaught
"Bort, Paul" <[EMAIL PROTECTED]> writes: >One other thought: How does static RAM compare to disk speed nowadays? >A 1Gb flash drive might be reasonable for the WAL if it can keep up. Flash RAM "wears out"; it's not suitable for a continuously-updated application like WAL. -Doug

Re: [Testperf-general] Re: [HACKERS] ExclusiveLock

2004-11-23 Thread Bort, Paul
Title: RE: [Testperf-general] Re: [HACKERS] ExclusiveLock > The impression I had was that disk drives no longer pay the slightest > attention to interleave specs, because the logical model > implied by the > concept is too far removed from modern reality (on-disk buffering

Re: [Testperf-general] Re: [HACKERS] ExclusiveLock

2004-11-22 Thread Tom Lane
Greg Stark <[EMAIL PROTECTED]> writes: > Once upon a time when you formatted hard drives you actually gave them an > interleave factor for a similar reason. These days you invariably use an > interleave of 1, ie, store the blocks continuously. Whether that's because > controllers have become fast e

Re: [Testperf-general] Re: [HACKERS] ExclusiveLock

2004-11-22 Thread Bruce Momjian
Simon Riggs wrote: > On Mon, 2004-11-22 at 23:37, Greg Stark wrote: > > Simon Riggs <[EMAIL PROTECTED]> writes: > > > > > - Find a way to reduce rotational delay when repeatedly writing last WAL > > > page > > > > > > Currently fsync of WAL requires the disk platter to perform a full > > > rotat

Re: [Testperf-general] Re: [HACKERS] ExclusiveLock

2004-11-22 Thread Simon Riggs
On Mon, 2004-11-22 at 23:37, Greg Stark wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > > - Find a way to reduce rotational delay when repeatedly writing last WAL > > page > > > > Currently fsync of WAL requires the disk platter to perform a full > > rotation to fsync again. One idea is to

Re: [Testperf-general] Re: [HACKERS] ExclusiveLock

2004-11-22 Thread Greg Stark
Simon Riggs <[EMAIL PROTECTED]> writes: > - Find a way to reduce rotational delay when repeatedly writing last WAL > page > > Currently fsync of WAL requires the disk platter to perform a full > rotation to fsync again. One idea is to write the WAL to different > offsets that might reduce the r

Re: [Testperf-general] Re: [HACKERS] ExclusiveLock

2004-11-22 Thread Simon Riggs
On Thu, 2004-11-18 at 23:54, Tom Lane wrote: > I don't think so; WAL is inherently a linear log. (Awhile ago there was > some talk of nonlinear log writing to get around the one-commit-per- > disk-revolution syndrome, but the idea basically got rejected as > unworkably complicated.) ...this app

Re: [Testperf-general] Re: [HACKERS] ExclusiveLock

2004-11-21 Thread Simon Riggs
On Sat, 2004-11-20 at 16:14, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Thu, 2004-11-18 at 22:55, Tom Lane wrote: > >> If it is a problem, the LockBuffer calls in RelationGetBufferForTuple > >> would be the places showing contention delays. > > > You say this as if we can eas

Re: [Testperf-general] Re: [HACKERS] ExclusiveLock

2004-11-20 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Thu, 2004-11-18 at 22:55, Tom Lane wrote: >> If it is a problem, the LockBuffer calls in RelationGetBufferForTuple >> would be the places showing contention delays. > You say this as if we can easily check that. I think this can be done with oprofile .

Re: [Testperf-general] Re: [HACKERS] ExclusiveLock

2004-11-20 Thread Simon Riggs
On Thu, 2004-11-18 at 22:55, Tom Lane wrote: > Josh Berkus <[EMAIL PROTECTED]> writes: > >> The main problem on INSERTs is that it is usually the same few pages: > >> the lead data block and the lead index block. There are ways of > >> spreading the load out across an index, but I'm not sure what h

Re: [Testperf-general] Re: [HACKERS] ExclusiveLock

2004-11-18 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > Can we subdivide the WALInsertLock so there are multiple entry points to > wal_buffers, based upon hashing the xid? I don't think so; WAL is inherently a linear log. (Awhile ago there was some talk of nonlinear log writing to get around the one-commit-per

Re: [Testperf-general] Re: [HACKERS] ExclusiveLock

2004-11-18 Thread Simon Riggs
On Thu, 2004-11-18 at 23:19, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > Would it be possible to: when a new block is allocated from the relation > > file (rather than reused), we check the FSM - if it is empty, then we > > allocate 8 new blocks and add them all to the FSM. The ne

Re: [Testperf-general] Re: [HACKERS] ExclusiveLock

2004-11-18 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > Would it be possible to: when a new block is allocated from the relation > file (rather than reused), we check the FSM - if it is empty, then we > allocate 8 new blocks and add them all to the FSM. The next few > INSERTers will then use the FSM blocks norma

Re: [Testperf-general] Re: [HACKERS] ExclusiveLock

2004-11-18 Thread Simon Riggs
On Thu, 2004-11-18 at 22:51, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > The main problem on INSERTs is that it is usually the same few pages: > > the lead data block and the lead index block. There are ways of > > spreading the load out across an index, but I'm not sure what happ

Re: [Testperf-general] Re: [HACKERS] ExclusiveLock

2004-11-18 Thread Tom Lane
Josh Berkus <[EMAIL PROTECTED]> writes: >> The main problem on INSERTs is that it is usually the same few pages: >> the lead data block and the lead index block. There are ways of >> spreading the load out across an index, but I'm not sure what happens on >> the leading edge of the data relation, b

Re: [Testperf-general] Re: [HACKERS] ExclusiveLock

2004-11-18 Thread Josh Berkus
Simon, Tom, > The main problem on INSERTs is that it is usually the same few pages: > the lead data block and the lead index block. There are ways of > spreading the load out across an index, but I'm not sure what happens on > the leading edge of the data relation, but I think it hits the same > b

Re: [Testperf-general] Re: [HACKERS] ExclusiveLock

2004-11-18 Thread Simon Riggs
On Thu, 2004-11-18 at 22:12, Tom Lane wrote: > Josh Berkus <[EMAIL PROTECTED]> writes: > > Aside from foriegn keys, though, is there any way in which INSERT page > > locks > > could block other inserts? > > Not for longer than the time needed to physically add a tuple to a page. The main proble

Re: [Testperf-general] Re: [HACKERS] ExclusiveLock

2004-11-18 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > The main problem on INSERTs is that it is usually the same few pages: > the lead data block and the lead index block. There are ways of > spreading the load out across an index, but I'm not sure what happens on > the leading edge of the data relation, but I

Re: [Testperf-general] Re: [HACKERS] ExclusiveLock

2004-11-18 Thread Tom Lane
Josh Berkus <[EMAIL PROTECTED]> writes: > Aside from foriegn keys, though, is there any way in which INSERT page locks > could block other inserts? Not for longer than the time needed to physically add a tuple to a page. regards, tom lane ---(end

Re: [Testperf-general] Re: [HACKERS] ExclusiveLock

2004-11-18 Thread Josh Berkus
Tom, > I think you are right that these reflect heap or btree-index extension > operations. Those do not actually take locks on the *table* however, > but locks on a single page within it (which are completely orthogonal to > table locks and don't conflict). The pg_locks output leaves something

Re: [HACKERS] ExclusiveLock

2004-11-09 Thread Simon Riggs
On Mon, 2004-11-08 at 21:37, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > Recent runs of DBT-2 show very occasional ExclusiveLock (s) being held > > by transactions, sometimes waiting to be granted. > > I think you are right that these reflect heap or btree-index extension > opera

Re: [HACKERS] ExclusiveLock

2004-11-08 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > Recent runs of DBT-2 show very occasional ExclusiveLock (s) being held > by transactions, sometimes waiting to be granted. I think you are right that these reflect heap or btree-index extension operations. Those do not actually take locks on the *table* h