Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-12-07 Thread Michael Paquier
On Wed, Dec 3, 2014 at 4:03 PM, Michael Paquier wrote: > Ping? This patch is in a stale state for a couple of weeks and still > marked as waiting on author for this CF. Marked as returned with feedback. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make ch

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-12-02 Thread Michael Paquier
On Tue, Nov 18, 2014 at 12:33 AM, Robert Haas wrote: > On Mon, Nov 17, 2014 at 10:31 AM, Andres Freund > wrote: >> On 2014-11-17 10:21:04 -0500, Robert Haas wrote: >>> Andres, where are we with this patch? >>> >>> 1. You're going to commit it, but haven't gotten around to it yet. >>> >>> 2. You'

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-11-17 Thread Robert Haas
On Mon, Nov 17, 2014 at 10:31 AM, Andres Freund wrote: > On 2014-11-17 10:21:04 -0500, Robert Haas wrote: >> Andres, where are we with this patch? >> >> 1. You're going to commit it, but haven't gotten around to it yet. >> >> 2. You're going to modify it some more and repost, but haven't gotten >>

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-11-17 Thread Andres Freund
On 2014-11-17 10:21:04 -0500, Robert Haas wrote: > Andres, where are we with this patch? > > 1. You're going to commit it, but haven't gotten around to it yet. > > 2. You're going to modify it some more and repost, but haven't gotten > around to it yet. > > 3. You're willing to see it modified i

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-11-17 Thread Robert Haas
On Sat, Oct 25, 2014 at 1:50 AM, Amit Kapila wrote: > On Fri, Oct 24, 2014 at 4:05 PM, Andres Freund > wrote: >> >> On 2014-10-24 15:59:30 +0530, Amit Kapila wrote: >> > > > and w.r.t performance it can lead extra >> > > > function call, few checks and I think in some cases even can >> > > > acqu

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-24 Thread Amit Kapila
On Fri, Oct 24, 2014 at 4:05 PM, Andres Freund wrote: > > On 2014-10-24 15:59:30 +0530, Amit Kapila wrote: > > > > and w.r.t performance it can lead extra > > > > function call, few checks and I think in some cases even can > > > > acquire/release spinlock. > > > > > > I fail to see how that could

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-24 Thread Andres Freund
On 2014-10-24 15:59:30 +0530, Amit Kapila wrote: > > > and w.r.t performance it can lead extra > > > function call, few checks and I think in some cases even can > > > acquire/release spinlock. > > > > I fail to see how that could be the case. > > Won't it happen incase first backend sets releaseO

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-24 Thread Amit Kapila
On Wed, Oct 22, 2014 at 8:04 PM, Andres Freund wrote: > On 2014-10-21 19:56:05 +0530, Amit Kapila wrote: > > On Wed, Oct 8, 2014 at 6:17 PM, Andres Freund > > wrote: > > spin_delay_count gives > > how much delay has happened to acquire spinlock which when > > combined with other stats gives the c

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-24 Thread Amit Kapila
On Wed, Oct 22, 2014 at 7:12 PM, Andres Freund wrote: > > On 2014-10-22 13:32:07 +0530, Amit Kapila wrote: > > On Tue, Oct 21, 2014 at 7:56 PM, Amit Kapila > > wrote: > > > On Wed, Oct 8, 2014 at 6:17 PM, Andres Freund > > wrote: > > > > > > > > On 2014-06-25 19:06:32 +0530, Amit Kapila wrote: >

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-22 Thread Andres Freund
On 2014-10-21 19:56:05 +0530, Amit Kapila wrote: > On Wed, Oct 8, 2014 at 6:17 PM, Andres Freund > wrote: > > > > On 2014-06-25 19:06:32 +0530, Amit Kapila wrote: > > > 2. > > > LWLockWakeup() > > > { > > > .. > > > #ifdef LWLOCK_STATS > > > lwstats->spin_delay_count += SpinLockAcquire(&lock->mute

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-22 Thread Andres Freund
On 2014-10-22 13:32:07 +0530, Amit Kapila wrote: > On Tue, Oct 21, 2014 at 7:56 PM, Amit Kapila > wrote: > > On Wed, Oct 8, 2014 at 6:17 PM, Andres Freund > wrote: > > > > > > On 2014-06-25 19:06:32 +0530, Amit Kapila wrote: > > Today, I have verified all previous comments raised by > me and loo

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-22 Thread Amit Kapila
On Tue, Oct 21, 2014 at 7:56 PM, Amit Kapila wrote: > On Wed, Oct 8, 2014 at 6:17 PM, Andres Freund wrote: > > > > On 2014-06-25 19:06:32 +0530, Amit Kapila wrote: Today, I have verified all previous comments raised by me and looked at new code and below are my findings: >> >> 4. >> LWLockAcqui

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-21 Thread Amit Kapila
On Wed, Oct 8, 2014 at 6:17 PM, Andres Freund wrote: > > On 2014-06-25 19:06:32 +0530, Amit Kapila wrote: > > 2. > > LWLockWakeup() > > { > > .. > > #ifdef LWLOCK_STATS > > lwstats->spin_delay_count += SpinLockAcquire(&lock->mutex); > > #else > > SpinLockAcquire(&lock->mutex); > > #endif > > .. >

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-10 Thread Andres Freund
On 2014-10-08 20:07:35 -0400, Robert Haas wrote: > On Wed, Oct 8, 2014 at 2:04 PM, Andres Freund wrote: > > So, what makes it work for me (among other unrelated stuff) seems to be > > the following in .gdbinit, defineing away some things that gdb doesn't > > handle: > > macro define __builtin_offs

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-08 Thread Robert Haas
On Wed, Oct 8, 2014 at 2:04 PM, Andres Freund wrote: > So, what makes it work for me (among other unrelated stuff) seems to be > the following in .gdbinit, defineing away some things that gdb doesn't > handle: > macro define __builtin_offsetof(T, F) ((int) &(((T *) 0)->F)) > macro define __extensi

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-08 Thread Andres Freund
On 2014-10-08 14:23:44 -0300, Alvaro Herrera wrote: > Robert Haas wrote: > > On Wed, Oct 8, 2014 at 8:47 AM, Andres Freund > > wrote: > > > I don't see that as being relevant. The difference is an instruction or > > > two - in the slow path we'll enter the kernel and sleep. This doesn't > > > mat

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-08 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, Oct 8, 2014 at 8:47 AM, Andres Freund wrote: > > I don't see that as being relevant. The difference is an instruction or > > two - in the slow path we'll enter the kernel and sleep. This doesn't > > matter in comparison. > > And the code is *so* much more readable. >

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-08 Thread Andres Freund
On 2014-10-08 13:13:33 -0400, Robert Haas wrote: > On Wed, Oct 8, 2014 at 8:47 AM, Andres Freund wrote: > > I don't see that as being relevant. The difference is an instruction or > > two - in the slow path we'll enter the kernel and sleep. This doesn't > > matter in comparison. > > And the code i

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-08 Thread Robert Haas
On Wed, Oct 8, 2014 at 8:47 AM, Andres Freund wrote: > I don't see that as being relevant. The difference is an instruction or > two - in the slow path we'll enter the kernel and sleep. This doesn't > matter in comparison. > And the code is *so* much more readable. I find the slist/dlist stuff ac

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-08 Thread Andres Freund
On 2014-10-08 14:47:44 +0200, Andres Freund wrote: > On 2014-06-25 19:06:32 +0530, Amit Kapila wrote: > > 5. > > LWLockWakeup() > > { > > .. > > dlist_foreach_modify(iter, (dlist_head *) &wakeup) > > { > > PGPROC *waiter = dlist_container(PGPROC, lwWaitLink, iter.cur); > > LOG_LWDEBUG("LWLockReleas

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-08 Thread Andres Freund
On 2014-06-25 19:06:32 +0530, Amit Kapila wrote: > 2. > LWLockWakeup() > { > .. > #ifdef LWLOCK_STATS > lwstats->spin_delay_count += SpinLockAcquire(&lock->mutex); > #else > SpinLockAcquire(&lock->mutex); > #endif > .. > } > > Earlier while releasing lock, we don't count it towards LWLock stats >

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-06-25 Thread Amit Kapila
On Tue, Jun 24, 2014 at 9:33 AM, Amit Kapila wrote: > On Mon, Jun 23, 2014 at 9:12 PM, Andres Freund wrote: > > On 2014-06-23 19:59:10 +0530, Amit Kapila wrote: > > > 7. > > > LWLockWaitForVar() > > > { > > > .. > > > /* > > > * Add myself to wait queue. Note that this is racy, somebody else > >

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-06-23 Thread Amit Kapila
On Mon, Jun 23, 2014 at 9:12 PM, Andres Freund wrote: > On 2014-06-23 19:59:10 +0530, Amit Kapila wrote: > > 12. > > #ifdef LWLOCK_DEBUG > > lock->owner = MyProc; > > #endif > > > > Shouldn't it be reset in LWLockRelease? > > That's actually intentional. It's quite useful to know the last owner >

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-06-23 Thread Amit Kapila
On Mon, Jun 23, 2014 at 9:12 PM, Andres Freund wrote: > On 2014-06-23 19:59:10 +0530, Amit Kapila wrote: > > On Tue, Jun 17, 2014 at 8:56 PM, Andres Freund > > wrote: > > 2. > > LWLockAcquireCommon() > > { > > .. > > if (!LWLockDequeueSelf(l)) > > { > > /* > > * Somebody else dequeued us and has

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-06-23 Thread Andres Freund
On 2014-06-23 19:59:10 +0530, Amit Kapila wrote: > On Tue, Jun 17, 2014 at 8:56 PM, Andres Freund > wrote: > > On 2014-06-17 20:47:51 +0530, Amit Kapila wrote: > > > On Tue, Jun 17, 2014 at 6:35 PM, Andres Freund > > > wrote: > > > > > > You have followed it pretty well as far as I can understand

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-06-23 Thread Amit Kapila
On Tue, Jun 17, 2014 at 8:56 PM, Andres Freund wrote: > On 2014-06-17 20:47:51 +0530, Amit Kapila wrote: > > On Tue, Jun 17, 2014 at 6:35 PM, Andres Freund > > wrote: > > > > You have followed it pretty well as far as I can understand from your > > replies, as there is no reproducible test (which

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-06-17 Thread Andres Freund
On 2014-06-17 20:47:51 +0530, Amit Kapila wrote: > On Tue, Jun 17, 2014 at 6:35 PM, Andres Freund > wrote: > > On 2014-06-17 18:01:58 +0530, Amit Kapila wrote: > > > On Tue, Jun 17, 2014 at 3:56 PM, Andres Freund > > > > On 2014-06-17 12:41:26 +0530, Amit Kapila wrote: > > I unfortunately still c

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-06-17 Thread Amit Kapila
On Tue, Jun 17, 2014 at 6:35 PM, Andres Freund wrote: > > On 2014-06-17 18:01:58 +0530, Amit Kapila wrote: > > On Tue, Jun 17, 2014 at 3:56 PM, Andres Freund > > > On 2014-06-17 12:41:26 +0530, Amit Kapila wrote: > > > > 2. > > > > Handling of potentialy_spurious case seems to be pending > > > >

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-06-17 Thread Andres Freund
On 2014-06-17 18:01:58 +0530, Amit Kapila wrote: > On Tue, Jun 17, 2014 at 3:56 PM, Andres Freund > > On 2014-06-17 12:41:26 +0530, Amit Kapila wrote: > > > 2. > > > Handling of potentialy_spurious case seems to be pending > > > in LWLock functions like LWLockAcquireCommon(). > > > > > > LWLockAcq

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-06-17 Thread Amit Kapila
On Tue, Jun 17, 2014 at 3:56 PM, Andres Freund wrote: > > On 2014-06-17 12:41:26 +0530, Amit Kapila wrote: > > On Fri, May 23, 2014 at 10:01 PM, Amit Kapila > > wrote: > > > On Fri, Jan 31, 2014 at 3:24 PM, Andres Freund > > wrote: > > > > I've pushed a rebased version of the patchset to > > > >

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-06-17 Thread Andres Freund
On 2014-06-17 12:41:26 +0530, Amit Kapila wrote: > On Fri, May 23, 2014 at 10:01 PM, Amit Kapila > wrote: > > On Fri, Jan 31, 2014 at 3:24 PM, Andres Freund > wrote: > > > I've pushed a rebased version of the patchset to > > > http://git.postgresql.org/gitweb/?p=users/andresfreund/postgres.git >

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-06-17 Thread Amit Kapila
On Fri, May 23, 2014 at 10:01 PM, Amit Kapila wrote: > On Fri, Jan 31, 2014 at 3:24 PM, Andres Freund wrote: > > I've pushed a rebased version of the patchset to > > http://git.postgresql.org/gitweb/?p=users/andresfreund/postgres.git > > branch rwlock contention. > > 220b34331f77effdb46798ddd7cca

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-05-23 Thread Amit Kapila
On Fri, Jan 31, 2014 at 3:24 PM, Andres Freund wrote: > I've pushed a rebased version of the patchset to > http://git.postgresql.org/gitweb/?p=users/andresfreund/postgres.git > branch rwlock contention. > 220b34331f77effdb46798ddd7cca0cffc1b2858 actually was the small problem, > ea9df812d8502fff74

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-02-10 Thread Heikki Linnakangas
On 01/31/2014 11:54 AM, Andres Freund wrote: Hi, On 2014-01-28 21:27:29 -0800, Peter Geoghegan wrote: On Fri, Nov 15, 2013 at 11:47 AM, Andres Freund wrote: 1) I've added an abstracted atomic ops implementation. Needs a fair amount of work, also submitted as a separate CF entry. (Patch 1

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-02-04 Thread Andres Freund
On 2014-02-03 17:51:20 -0800, Peter Geoghegan wrote: > On Sun, Feb 2, 2014 at 6:00 AM, Andres Freund wrote: > > On 2014-02-01 19:47:29 -0800, Peter Geoghegan wrote: > >> Here are the results of a benchmark on Nathan Boley's 64-core, 4 > >> socket server: > >> http://postgres-benchmarks.s3-website

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-02-03 Thread Peter Geoghegan
On Sun, Feb 2, 2014 at 6:00 AM, Andres Freund wrote: > On 2014-02-01 19:47:29 -0800, Peter Geoghegan wrote: >> Here are the results of a benchmark on Nathan Boley's 64-core, 4 >> socket server: >> http://postgres-benchmarks.s3-website-us-east-1.amazonaws.com/amd-4-socket-rwlocks/ > > That's inter

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-02-03 Thread Peter Geoghegan
On Sun, Feb 2, 2014 at 6:00 AM, Andres Freund wrote: > The changed algorithm for lwlock imo is an *algorithmic* improvement, > not one for a particular architecture. The advantage being that locking > a lwlock which is primarily taken in shared mode will never need need to > wait or loop. I agree

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-02-03 Thread Jeff Janes
On Sun, Feb 2, 2014 at 6:00 AM, Andres Freund wrote: > > Some background: > The setups that triggered me into working on the patchset didn't really > have a pgbench like workload, the individual queries were/are more > complicated even though it's still an high throughput OLTP workload. And > the

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-02-02 Thread Andres Freund
Hi, On 2014-02-01 19:47:29 -0800, Peter Geoghegan wrote: > Here are the results of a benchmark on Nathan Boley's 64-core, 4 > socket server: > http://postgres-benchmarks.s3-website-us-east-1.amazonaws.com/amd-4-socket-rwlocks/ That's interesting. The maximum number of what you see here (~293125)

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-02-01 Thread Peter Geoghegan
On Sat, Feb 1, 2014 at 1:41 PM, Andres Freund wrote: >> However, I tested the >> most recent revision from your git remote on the AWS instance. > But that was before my fix, right. Except you managed to timetravel :) Heh, okay. So Nathan Boley has generously made available a machine with 4 AMD O

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-02-01 Thread Andres Freund
On 2014-02-01 13:40:20 -0800, Peter Geoghegan wrote: > On Sat, Feb 1, 2014 at 4:57 AM, Andres Freund wrote: > >> I'm looking at alternative options, because this is not terribly > >> helpful. With those big caveats in mind, consider the results of the > >> benchmark, which show the patch performin

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-02-01 Thread Peter Geoghegan
On Sat, Feb 1, 2014 at 4:57 AM, Andres Freund wrote: >> I'm looking at alternative options, because this is not terribly >> helpful. With those big caveats in mind, consider the results of the >> benchmark, which show the patch performing somewhat worse than the >> master baseline at higher client

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-02-01 Thread Andres Freund
On 2014-01-31 17:52:58 -0800, Peter Geoghegan wrote: > On Fri, Jan 31, 2014 at 1:54 AM, Andres Freund wrote: > > I plan to split the atomics patch into smaller chunks before > > reposting. Imo the "Convert the PGPROC->lwWaitLink list into a dlist > > instead of open coding it." is worth being appl

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-02-01 Thread Peter Geoghegan
I thought I'd try out what I was in an immediate position to do without having access to dedicated multi-socket hardware: A benchmark on AWS. This was a "c3.8xlarge" instance, which are reportedly backed by Intel Xeon E5-2680 processors. Since the Intel ARK website reports that these CPUs have 16 "

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-01-31 Thread Peter Geoghegan
On Fri, Jan 31, 2014 at 1:54 AM, Andres Freund wrote: > I plan to split the atomics patch into smaller chunks before > reposting. Imo the "Convert the PGPROC->lwWaitLink list into a dlist > instead of open coding it." is worth being applied independently from > the rest of the series, it simplies

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-01-31 Thread Andres Freund
Hi, On 2014-01-28 21:27:29 -0800, Peter Geoghegan wrote: > On Fri, Nov 15, 2013 at 11:47 AM, Andres Freund > wrote: > > 1) I've added an abstracted atomic ops implementation. Needs a fair > >amount of work, also submitted as a separate CF entry. (Patch 1 & 2) > > Commit 220b34331f77effdb467

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-01-28 Thread Peter Geoghegan
On Fri, Nov 15, 2013 at 11:47 AM, Andres Freund wrote: > 1) I've added an abstracted atomic ops implementation. Needs a fair >amount of work, also submitted as a separate CF entry. (Patch 1 & 2) Commit 220b34331f77effdb46798ddd7cca0cffc1b2858 caused bitrot when applying 0002-Very-basic-atomic

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2013-12-21 Thread Peter Eisentraut
This patch didn't make it out of the 2013-11 commit fest. You should move it to the next commit fest (probably with an updated patch) before January 15th. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailp

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2013-11-16 Thread Peter Eisentraut
On Fri, 2013-11-15 at 20:47 +0100, Andres Freund wrote: > So, here's the next version of this patchset: The 0002 patch contains non-ASCII, non-UTF8 characters: 0002-Very-basic-atomic-ops-implementation.patch: line 609, char 1, byte offset 43: invalid UTF-8 code Please change that to ASCII, or U

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2013-11-15 Thread Andres Freund
On 2013-11-15 20:47:26 +0100, Andres Freund wrote: > Hi, > > On 2013-09-27 00:55:45 +0200, Andres Freund wrote: > > So what's todo? The file header tells us: > > * - revive pure-spinlock implementation > > * - abstract away atomic ops, we really only need a few. > > * - CAS > > * - LOCK XA