Re: [COMMITTERS] pgsql: Provide errno-translation wrappers around bind() and listen() on

2016-04-13 Thread David Rowley
On 13 April 2016 at 11:52, Tom Lane wrote: > Provide errno-translation wrappers around bind() and listen() on Windows. > > I've seen one too many "could not bind IPv4 socket: No error" log entries > from the Windows buildfarm members. Per previous discussion, this is > likely caused by the fact t

[COMMITTERS] pgsql: Fix broken dependency-mongering for index operator classes/famil

2016-04-13 Thread Tom Lane
Fix broken dependency-mongering for index operator classes/families. For a long time, opclasscmds.c explained that "we do not create a dependency link to the AM [for an opclass or opfamily], because we don't currently support DROP ACCESS METHOD". Commit 473b93287040b200 invented DROP ACCESS METHO

Re: [COMMITTERS] pgsql: Fix pg_dump so pg_upgrade'ing an extension with simple opfamilie

2016-04-13 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> I see this has broken the buildfarm's pg_upgrade checks, though only in >> HEAD which makes me suspicious that it's exposing a pre-existing bug. >> Odd, because I definitely tested pg_upgrade here. Will dig into it >> later this ev

[COMMITTERS] pgsql: Fix duplicated index entry in doc.

2016-04-13 Thread Fujii Masao
Fix duplicated index entry in doc. Commit cfe96ae corrected the name of pg_logical_emit_message() in its index entry. But this typo fix caused duplicated index entry because there was another index entry for the function. Spotted by Tom Lane. Branch -- master Details --- http://git.post

Re: [COMMITTERS] pgsql: Fix pg_dump so pg_upgrade'ing an extension with simple opfamilie

2016-04-13 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Tom Lane writes: > > Fix pg_dump so pg_upgrade'ing an extension with simple opfamilies works. > > I see this has broken the buildfarm's pg_upgrade checks, though only in > HEAD which makes me suspicious that it's exposing a pre-existing bug. > Odd, because

[COMMITTERS] pgsql: Disallow SET SESSION AUTHORIZATION pg_*

2016-04-13 Thread Stephen Frost
Disallow SET SESSION AUTHORIZATION pg_* As part of reserving the pg_* namespace for default roles and in line with SET ROLE and other previous efforts, disallow settings the role to a default/reserved role using SET SESSION AUTHORIZATION. These checks and restrictions on what is allowed regarding

[COMMITTERS] pgsql: Add required database and origin filtering for logical messages.

2016-04-13 Thread Andres Freund
Add required database and origin filtering for logical messages. Logical messages, added in 3fe3511d05, during decoding failed to filter messages emitted in other databases and messages emitted "under" a replication origin the output plugin isn't interested in. Add tests to verify that both types

Re: [COMMITTERS] pgsql: Fix pg_dump so pg_upgrade'ing an extension with simple opfamilie

2016-04-13 Thread Tom Lane
Tom Lane writes: > Fix pg_dump so pg_upgrade'ing an extension with simple opfamilies works. I see this has broken the buildfarm's pg_upgrade checks, though only in HEAD which makes me suspicious that it's exposing a pre-existing bug. Odd, because I definitely tested pg_upgrade here. Will dig int

[COMMITTERS] pgsql: Make init_spin_delay() C89 compliant and change stuck spinlock r

2016-04-13 Thread Andres Freund
Make init_spin_delay() C89 compliant and change stuck spinlock reporting. The current definition of init_spin_delay (introduced recently in 48354581a) wasn't C89 compliant. It's not legal to refer to refer to non-constant expressions, and the ptr argument was one. This, as reported by Tom, lead t

[COMMITTERS] pgsql: Fix pg_dump so pg_upgrade'ing an extension with simple opfamilie

2016-04-13 Thread Tom Lane
Fix pg_dump so pg_upgrade'ing an extension with simple opfamilies works. As reported by Michael Feld, pg_upgrade'ing an installation having extensions with operator families that contain just a single operator class failed to reproduce the extension membership of those operator families. This caus

[COMMITTERS] pgsql: Fix pg_dump so pg_upgrade'ing an extension with simple opfamilie

2016-04-13 Thread Tom Lane
Fix pg_dump so pg_upgrade'ing an extension with simple opfamilies works. As reported by Michael Feld, pg_upgrade'ing an installation having extensions with operator families that contain just a single operator class failed to reproduce the extension membership of those operator families. This caus

[COMMITTERS] pgsql: Fix pg_dump so pg_upgrade'ing an extension with simple opfamilie

2016-04-13 Thread Tom Lane
Fix pg_dump so pg_upgrade'ing an extension with simple opfamilies works. As reported by Michael Feld, pg_upgrade'ing an installation having extensions with operator families that contain just a single operator class failed to reproduce the extension membership of those operator families. This caus

[COMMITTERS] pgsql: Fix pg_dump so pg_upgrade'ing an extension with simple opfamilie

2016-04-13 Thread Tom Lane
Fix pg_dump so pg_upgrade'ing an extension with simple opfamilies works. As reported by Michael Feld, pg_upgrade'ing an installation having extensions with operator families that contain just a single operator class failed to reproduce the extension membership of those operator families. This caus

[COMMITTERS] pgsql: Fix pg_dump so pg_upgrade'ing an extension with simple opfamilie

2016-04-13 Thread Tom Lane
Fix pg_dump so pg_upgrade'ing an extension with simple opfamilies works. As reported by Michael Feld, pg_upgrade'ing an installation having extensions with operator families that contain just a single operator class failed to reproduce the extension membership of those operator families. This caus

[COMMITTERS] pgsql: Fix pg_dump so pg_upgrade'ing an extension with simple opfamilie

2016-04-13 Thread Tom Lane
Fix pg_dump so pg_upgrade'ing an extension with simple opfamilies works. As reported by Michael Feld, pg_upgrade'ing an installation having extensions with operator families that contain just a single operator class failed to reproduce the extension membership of those operator families. This caus

[COMMITTERS] pgsql: Avoid atomic operation in MarkLocalBufferDirty().

2016-04-13 Thread Andres Freund
Avoid atomic operation in MarkLocalBufferDirty(). The recent patch to make Pin/UnpinBuffer lockfree in the hot path (48354581a), accidentally used pg_atomic_fetch_or_u32() in MarkLocalBufferDirty(). Other code operating on local buffers was careful to only use pg_atomic_read/write_u32 which just r

[COMMITTERS] pgsql: Widen amount-to-flush arguments of FileWriteback and callers.

2016-04-13 Thread Tom Lane
Widen amount-to-flush arguments of FileWriteback and callers. It's silly to define these counts as narrower than they might someday need to be. Also, I believe that the BLCKSZ * nflush calculation in mdwriteback was capable of overflowing an int. Branch -- master Details --- http://git.

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Andres Freund
On 2016-04-13 16:05:25 -0500, Kevin Grittner wrote: > OK, thanks. I can't think of anything else to ask for at this > point. If you feel that you have enough to press for some > particular course of action, go for it. I think we, at the very least, need a clear proposal how to resolve the scalab

[COMMITTERS] pgsql: Fix assorted portability issues with using msync() for data flus

2016-04-13 Thread Tom Lane
Fix assorted portability issues with using msync() for data flushing. Commit 428b1d6b29ca599c5700d4bc4f4ce4c5880369bf introduced the use of msync() for flushing dirty data from the kernel's file buffers. Several portability issues were overlooked, though: * Not all implementations of mmap() thin

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Kevin Grittner
On Wed, Apr 13, 2016 at 3:47 PM, Andres Freund wrote: > On 2016-04-13 15:21:31 -0500, Kevin Grittner wrote: >> What is the kernel on which these tests were run? > > 3.16. I can upgrade to 4.4 if necessary. No, I'm not aware of any problems from 3.8 on. > But I still believe very strongly that t

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Andres Freund
On 2016-04-13 15:21:31 -0500, Kevin Grittner wrote: > On Wed, Apr 13, 2016 at 3:01 PM, Andres Freund wrote: > > > If you want me to rn some other tests I can, but ISTM we have the > > data we need? > > Thanks for the additional detail on how this was run. I think I > still need a little more co

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Kevin Grittner
On Wed, Apr 13, 2016 at 3:01 PM, Andres Freund wrote: > If you want me to rn some other tests I can, but ISTM we have the > data we need? Thanks for the additional detail on how this was run. I think I still need a little more context, though: What is the kernel on which these tests were run?

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Andres Freund
Hi Kevin, On 2016-04-13 12:21:10 -0700, Andres Freund wrote: > 0: > progress: 100.0 s, 593351.0 tps, lat 0.215 ms stddev 0.118 > progress: 200.0 s, 594035.9 tps, lat 0.215 ms stddev 0.118 > progress: 300.0 s, 594013.3 tps, lat 0.215 ms stddev 0.117 > > -1: > progress: 100.0 s, 600835.3 tps, lat 0

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Andres Freund
On 2016-04-13 14:08:49 -0500, Kevin Grittner wrote: > On Wed, Apr 13, 2016 at 1:56 PM, Andres Freund wrote: > > > I'll run with -1 once the current (longer) run has finished. > > Just for the record, were any of the other results purporting to be > with the feature "off" also actually running wi

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Andres Freund
On 2016-04-13 13:52:15 -0500, Kevin Grittner wrote: > On Wed, Apr 13, 2016 at 12:25 PM, Robert Haas wrote: > > > [test results with old_snapshot_threshold = 0 and 10] > > From the docs: > > | A value of -1 disables this feature, and is the default. > > > Yuck. Aside from the fact that performance

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Robert Haas
On Wed, Apr 13, 2016 at 3:08 PM, Kevin Grittner wrote: > On Wed, Apr 13, 2016 at 1:56 PM, Andres Freund wrote: > >> I'll run with -1 once the current (longer) run has finished. > > Just for the record, were any of the other results purporting to be > with the feature "off" also actually running w

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Kevin Grittner
On Wed, Apr 13, 2016 at 1:56 PM, Andres Freund wrote: > I'll run with -1 once the current (longer) run has finished. Just for the record, were any of the other results purporting to be with the feature "off" also actually running with the feature set for its fastest possible timeout? -- Kevin G

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Andres Freund
On 2016-04-13 13:52:15 -0500, Kevin Grittner wrote: > On Wed, Apr 13, 2016 at 12:25 PM, Robert Haas wrote: > > > [test results with old_snapshot_threshold = 0 and 10] > > From the docs: > > | A value of -1 disables this feature, and is the default. Hm, ok, let me run that as well then. The rea

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Kevin Grittner
On Wed, Apr 13, 2016 at 12:25 PM, Robert Haas wrote: > [test results with old_snapshot_threshold = 0 and 10] >From the docs: | A value of -1 disables this feature, and is the default. > Yuck. Aside from the fact that performance tanks when the feature is > turned on, it seems that there is a

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Andres Freund
On 2016-04-13 13:25:14 -0400, Robert Haas wrote: > > With -c old_snapshot_threshold=0: > > > > latency average = 0.218 ms > > latency stddev = 0.154 ms > > tps = 584666.289753 (including connections establishing) > > tps = 584867.785569 (excluding connections establishing) > > > > > > With -c old_s

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Robert Haas
On Wed, Apr 13, 2016 at 1:19 PM, Andres Freund wrote: > On an EC2 m4.10xlarge (dedicated, but still a VM) - sorry I don't have > anything better at hand right now, and it was already running. > > postgres config: > postgres -D /srv/data/dev/ > -c shared_buffers=64GB \ > -c max_wa

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Andres Freund
On 2016-04-12 14:53:57 -0500, Kevin Grittner wrote: > On Tue, Apr 12, 2016 at 2:28 PM, Andres Freund wrote: > > On 2016-04-12 14:17:12 -0500, Kevin Grittner wrote: > >> Well, something is different between your environment and mine, > >> since I saw no difference at scale 100 and 2.2% at scale 200

[COMMITTERS] pgsql: Improve documentation for \crosstabview.

2016-04-13 Thread Tom Lane
Improve documentation for \crosstabview. Fix misleading syntax summary (there cannot be a space between colH and scolH). Provide a link from the existing crosstab() function's documentation to \crosstabview. Copy-edit the command's description. Christoph Berg and Tom Lane Branch -- master

[COMMITTERS] pgsql: Use PG_INT32_MIN instead of reiterating the constant.

2016-04-13 Thread Robert Haas
Use PG_INT32_MIN instead of reiterating the constant. Makes no difference, but it's cleaner this way. Michael Paquier Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/cbb2a812d710dd58e68088b334f8c492346a0d0f Modified Files -- src/backend/utils/adt/numuti