Re: Missing wal_receiver_status_interval in Subscribers section

2018-01-23 Thread Masahiko Sawada
On Wed, Jan 24, 2018 at 9:34 AM, Bruce Momjian wrote: > On Tue, Jan 23, 2018 at 06:36:04PM -0500, Bruce Momjian wrote: >> >> Can someone confirm this so I can apply this patch? > > Never mind. I see this was applied: > Thank you for your kind response. Regards, -- Masahiko Sawada NIPPON TELEGR

Re: Remove utils/dsa.h from autovacuum.c

2018-01-23 Thread Masahiko Sawada
On Wed, Jan 24, 2018 at 3:39 AM, Alvaro Herrera wrote: > Masahiko Sawada wrote: >> Hi, >> >> Attached a patch for $subject. The implementation of autovacuum >> work-item has been changed by commit >> 31ae1638ce35c23979f9bcbb92c6bb51744dbccb but the loading of dsa.h >> header file is remained. > >

Re: pgsql: Add parallel-aware hash joins.

2018-01-23 Thread Thomas Munro
On Wed, Jan 24, 2018 at 12:10 PM, Tom Lane wrote: > There is a very clear secular trend up in the longer data series, > which indicates that we're testing more stuff, +1 > which doesn't bother > me in itself as long as the time is well spent. However, the trend > over the last two months is ver

Re: Failed to request an autovacuum work-item in silence

2018-01-23 Thread Masahiko Sawada
On Tue, Jan 23, 2018 at 8:03 PM, Fabrízio de Royes Mello wrote: > > Em ter, 23 de jan de 2018 às 03:36, Masahiko Sawada > escreveu: >> >> Hi all, >> >> While reading the code, I realized that the requesting an autovacuum >> work-item could fail in silence if work-item array is full. So the >> use

Re: Possible performance regression in version 10.1 with pgbench read-write tests.

2018-01-23 Thread Amit Kapila
On Wed, Jan 24, 2018 at 12:06 AM, Mithun Cy wrote: > Hi all, > > When I was trying to do read-write pgbench bench-marking of PostgreSQL > 9.6.6 vs 10.1 I found PostgreSQL 10.1 regresses against 9.6.6 in some > cases. > > Non Default settings and test > == > Server: > ./postgres

Re: [PATCH] fix for C4141 warning on MSVC

2018-01-23 Thread Thomas Munro
On Wed, Jan 24, 2018 at 1:42 PM, Tom Lane wrote: > Thomas Munro writes: >> On Wed, Jan 24, 2018 at 1:16 PM, Michail Nikolaev >> wrote: >>> Just very small fix for C4141 warning > >> Thanks. This is similar to the fix I proposed over here: >> https://www.postgresql.org/message-id/CAEepm%3D2iTKvb

Re: PATCH: Configurable file mode mask

2018-01-23 Thread Michael Paquier
On Tue, Jan 23, 2018 at 09:18:51AM -0500, David Steele wrote: > On 1/20/18 5:47 PM, Michael Paquier wrote: >> Making this possible would require first some >> refactoring of PostgresNode.pm so as a node is aware of the binary paths >> it uses to be able to manipulate multiple instances with differe

Re: Handling better supported channel binding types for SSL implementations

2018-01-23 Thread Michael Paquier
On Tue, Jan 23, 2018 at 12:08:37PM -0500, Peter Eisentraut wrote: > On 1/22/18 02:29, Michael Paquier wrote: >> However there is as well the argument that this list's contents are not >> directly used now, and based on what I saw from the MacOS SSL and GnuTLS >> patches that would not be the case a

Re: pg_rewind and replication slots

2018-01-23 Thread Michael Paquier
On Tue, Jan 23, 2018 at 08:49:53PM +0100, Laurenz Albe wrote: > When a primary with replication slots gets reset with pg_rewind, > it keeps the replication slots. > > This does no harm per se, but when it gets promoted again, > the replication slots are still there and are in the way. > Won't they

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Amit Kapila
On Wed, Jan 24, 2018 at 2:35 AM, Robert Haas wrote: > On Tue, Jan 23, 2018 at 11:15 AM, Robert Haas wrote: >> On Mon, Jan 22, 2018 at 10:56 PM, Amit Kapila >> wrote: >> >> OK. I've committed this patch and back-patched it to 9.6. >> Back-patching to 9.5 didn't looks simple because nworkers_lau

Re: pgsql: Allow UPDATE to move rows between partitions.

2018-01-23 Thread Amit Kapila
On Tue, Jan 23, 2018 at 9:59 PM, Robert Haas wrote: > On Tue, Jan 23, 2018 at 8:44 AM, Amit Kapila wrote: >> On Sat, Jan 20, 2018 at 2:03 AM, Robert Haas wrote: >>> Allow UPDATE to move rows between partitions. >> >> +If an UPDATE on a partitioned table causes a row to >> move >> +to an

Re: copy.c allocation constant

2018-01-23 Thread Bruce Momjian
On Tue, Nov 28, 2017 at 11:51:28AM -0500, Andrew Dunstan wrote: > > > While reading copy.c I noticed this line: > > > #define RAW_BUF_SIZE 65536        /* we palloc RAW_BUF_SIZE+1 bytes */ > > > Doesn't that seem rather odd? If we're adding 1 wouldn't it be better as > 65535 so we palloc a po

Re: pgindent run?

2018-01-23 Thread Bruce Momjian
On Tue, Nov 28, 2017 at 04:38:12PM -0500, Tom Lane wrote: > Thomas Munro writes: > > On Wed, Nov 29, 2017 at 9:47 AM, Tom Lane wrote: > >> I think that'd be taking it too far, especially given that the dependency > >> on a typedefs list means that the git hook might have a different idea > >> of

Re: documentation is now XML

2018-01-23 Thread Tom Lane
Bruce Momjian writes: > On Thu, Nov 23, 2017 at 03:39:24PM -0500, Tom Lane wrote: >> Also, we're way overdue for getting out from under the creaky TeX-based >> toolchain for producing PDFs. > I am coming in late here, but I am not aware of any open source > professional typesetting software that

Re: documentation is now XML

2018-01-23 Thread Bruce Momjian
On Tue, Jan 23, 2018 at 10:22:53PM -0500, Tom Lane wrote: > Bruce Momjian writes: > > On Thu, Nov 23, 2017 at 03:39:24PM -0500, Tom Lane wrote: > >> Also, we're way overdue for getting out from under the creaky TeX-based > >> toolchain for producing PDFs. > > > I am coming in late here, but I am

Re: documentation is now XML

2018-01-23 Thread David G. Johnston
On Tuesday, January 23, 2018, Bruce Momjian wrote: > On Tue, Jan 23, 2018 at 10:22:53PM -0500, Tom Lane wrote: > (a) it's got hard > > limits we're approaching, > All agreed, but what alternatives are being developed? > > I seem to recall a proposal a while back to gain margin on some of the l

Re: PATCH: Configurable file mode mask

2018-01-23 Thread David Steele
On 1/23/18 9:22 PM, Michael Paquier wrote: > On Tue, Jan 23, 2018 at 09:18:51AM -0500, David Steele wrote: >> On 1/20/18 5:47 PM, Michael Paquier wrote: >>> Making this possible would require first some >>> refactoring of PostgresNode.pm so as a node is aware of the binary paths >>> it uses to be a

Re: Incorrect comments in partition.c

2018-01-23 Thread Amit Langote
On 2018/01/23 20:43, Etsuro Fujita wrote: > Here is a comment for get_qual_for_list in partition.c: > > * get_qual_for_list > * > * Returns an implicit-AND list of expressions to use as a list partition's > - * constraint, given the partition key and bound structures. > > I don't think the

Re: [PATCH] fix for C4141 warning on MSVC

2018-01-23 Thread Tom Lane
Thomas Munro writes: > Here's one like that. Pushed; we'll soon see if the buildfarm likes it. I added a tweak to prevent forced inlining at -O0, as discussed in the other thread; and worked on the comments a bit. regards, tom lane

Re: [HACKERS] MERGE SQL Statement for PG11

2018-01-23 Thread Simon Riggs
On 24 January 2018 at 01:35, Peter Geoghegan wrote: > On Tue, Jan 23, 2018 at 5:51 AM, Simon Riggs wrote: >>> Not yet working >>> * Partitioning >>> * RLS >>> >>> Based on this successful progress I imagine I'll be looking to commit >>> this by the end of the CF, allowing us 2 further months to b

Re: documentation is now XML

2018-01-23 Thread Tom Lane
"David G. Johnston" writes: > On Tuesday, January 23, 2018, Bruce Momjian wrote: >> All agreed, but what alternatives are being developed? > I seem to recall a proposal a while back to gain margin on some of the > limits by pruning the release notes section down to at least this century > and ar

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Thomas Munro
On Wed, Jan 24, 2018 at 3:45 PM, Amit Kapila wrote: > On Wed, Jan 24, 2018 at 2:35 AM, Robert Haas wrote: >> In the case of Gather, for example, we won't >> WaitForParallelWorkersToFinish() until we've read all the tuples. If >> there's a tuple queue that does not yet have a sender, then we'll j

Re: PATCH: Configurable file mode mask

2018-01-23 Thread Michael Paquier
On Tue, Jan 23, 2018 at 10:48:07PM -0500, David Steele wrote: > Sorry - it means "level of effort". I was trying to get an idea if it > is something that could be available in the PG11 development cycle, or > if I should be looking at other ways to get the testing for this patch > completed. I do

Regarding ambulkdelete, amvacuumcleanup index methods

2018-01-23 Thread Abinaya k
Hai all, We are building In-memory index extension for postgres. We would capture table inserts, updates, deletes using triggers. During vacuum operation, postgres would give calls to ambulkdelete, amvacuumcleanup (as part of index cleanup). As we handle all updates, deletes using triggers, w

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Peter Geoghegan
On Tue, Jan 23, 2018 at 8:25 PM, Thomas Munro wrote: >> Hmm, I think that case will be addressed because tuple queues can >> detect if the leader is not attached. It does in code path >> shm_mq_receive->shm_mq_counterparty_gone. In >> shm_mq_counterparty_gone, it can detect if the worker is gone

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Amit Kapila
On Wed, Jan 24, 2018 at 10:03 AM, Peter Geoghegan wrote: > On Tue, Jan 23, 2018 at 8:25 PM, Thomas Munro > wrote: >>> Hmm, I think that case will be addressed because tuple queues can >>> detect if the leader is not attached. It does in code path >>> shm_mq_receive->shm_mq_counterparty_gone. In

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Amit Kapila
On Wed, Jan 24, 2018 at 9:55 AM, Thomas Munro wrote: > On Wed, Jan 24, 2018 at 3:45 PM, Amit Kapila wrote: >> On Wed, Jan 24, 2018 at 2:35 AM, Robert Haas wrote: >>> In the case of Gather, for example, we won't >>> WaitForParallelWorkersToFinish() until we've read all the tuples. If >>> there's

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-23 Thread Amit Kapila
On Wed, Jan 24, 2018 at 12:20 AM, Peter Geoghegan wrote: > On Tue, Jan 23, 2018 at 10:36 AM, Robert Haas wrote: >> As Amit says, what remains is the case where fork() fails or the >> worker dies before it reaches the line in ParallelWorkerMain that >> reads shm_mq_set_sender(mq, MyProc). In thos

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Thomas Munro
On Wed, Jan 24, 2018 at 5:43 PM, Amit Kapila wrote: >> Hmm. Yeah. I can't seem to reach a stuck case and was probably just >> confused and managed to confuse Robert too. If you make >> fork_process() fail randomly (see attached), I see that there are a >> couple of easily reachable failure mode

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-23 Thread Thomas Munro
On Wed, Jan 24, 2018 at 5:59 PM, Amit Kapila wrote: >>> I am going to repeat my previous suggest that we use a Barrier here. >>> Given the discussion subsequent to my original proposal, this can be a >>> lot simpler than what I suggested originally. Each worker does >>> BarrierAttach() before beg

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Peter Geoghegan
On Tue, Jan 23, 2018 at 9:02 PM, Thomas Munro wrote: >> Yes, this is what I am trying to explain on parallel create index >> thread. I think there we need to either use >> WaitForParallelWorkersToFinish or WaitForParallelWorkersToAttach (a >> new API as proposed in that thread) if we don't want t

Re: Possible performance regression in version 10.1 with pgbench read-write tests.

2018-01-23 Thread Mithun Cy
On Wed, Jan 24, 2018 at 7:36 AM, Amit Kapila wrote: > Both the cases look identical, but from the document attached, it > seems the case-1 is for scale factor 300. Oops sorry it was a typo. CASE 1 is scale factor 300 which will fit in shared buffer =8GB. -- Thanks and Regards Mithun C Y Ente

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Amit Kapila
On Wed, Jan 24, 2018 at 10:38 AM, Peter Geoghegan wrote: > On Tue, Jan 23, 2018 at 9:02 PM, Thomas Munro > wrote: >>> Yes, this is what I am trying to explain on parallel create index >>> thread. I think there we need to either use >>> WaitForParallelWorkersToFinish or WaitForParallelWorkersToAt

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-23 Thread Amit Kapila
On Wed, Jan 24, 2018 at 10:36 AM, Thomas Munro wrote: > On Wed, Jan 24, 2018 at 5:59 PM, Amit Kapila wrote: I am going to repeat my previous suggest that we use a Barrier here. Given the discussion subsequent to my original proposal, this can be a lot simpler than what I suggested

Re: Incorrect comments in partition.c

2018-01-23 Thread Etsuro Fujita
(2018/01/24 13:06), Amit Langote wrote: > On 2018/01/23 20:43, Etsuro Fujita wrote: >> Here is a comment for get_qual_for_list in partition.c: >> >>* get_qual_for_list >>* >>* Returns an implicit-AND list of expressions to use as a list partition's >> - * constraint, given the partition

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Peter Geoghegan
On Tue, Jan 23, 2018 at 9:38 PM, Amit Kapila wrote: > On Wed, Jan 24, 2018 at 10:38 AM, Peter Geoghegan wrote: >> The leader can go ahead and sort before calling something like a new >> WaitForParallelWorkersToAttach() function (or even >> WaitForParallelWorkersToFinish()). If we did add a >> Wai

Re: Index-only scan returns incorrect results when using a composite GIST index with a gist_trgm_ops column.

2018-01-23 Thread Andrey Borodin
Hi! > 24 янв. 2018 г., в 2:13, Sergei Kornilov написал(а): > > Should we also make backport to older versions? I test on REL_10_STABLE - > patch builds and works ok, but "make check" fails on new testcase with error: >> CREATE INDEX ON t USING gist (a test_inet_ops, a inet_ops); >> + ERROR: mis

Re: [HACKERS] taking stdbool.h into use

2018-01-23 Thread Michael Paquier
On Tue, Jan 23, 2018 at 11:33:56AM -0500, Peter Eisentraut wrote: > Here is a proposed patch set to clean this up. First, add some test > coverage for record_image_cmp. (There wasn't any, only for > record_image_eq as part of MV testing.) Then, remove the GET_ macros > from record_image_{eq,cmp}

Re: Doc tweak for huge_pages?

2018-01-23 Thread Catalin Iacob
On Tue, Jan 23, 2018 at 7:13 PM, Catalin Iacob wrote: > By the way, Fedora 27 does disable THP by default, they deviate from > upstream in this regard: > When I have some time I'll try to do some digging into history of the > Fedora kernel package to see if they provide a rationale for changing >

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-23 Thread Thomas Munro
On Wed, Jan 24, 2018 at 6:43 PM, Amit Kapila wrote: > On Wed, Jan 24, 2018 at 10:36 AM, Thomas Munro > wrote: >> On Wed, Jan 24, 2018 at 5:59 PM, Amit Kapila wrote: > I am going to repeat my previous suggest that we use a Barrier here. > Given the discussion subsequent to my original pro

Re: Index-only scan returns incorrect results when using a composite GIST index with a gist_trgm_ops column.

2018-01-23 Thread Kyotaro HORIGUCHI
Thank you for looking this. At Wed, 24 Jan 2018 00:13:51 +0300, Sergei Kornilov wrote in <348951516742...@web54j.yandex.ru> > Hello > I tested this patch and think it can be commited to master. Is there a CF > record? I can not find one. Not yet. I'm thinking of creating an entry in the next C

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-01-23 Thread amul sul
On Tue, Jan 23, 2018 at 7:01 PM, Amit Kapila wrote: > On Fri, Jan 12, 2018 at 11:43 AM, amul sul wrote: [] > I have asked to change the message "tuple to be updated .." after > heap_lock_tuple call not in nodeModifyTable.c, so please revert the > message in nodeModifyTable.c. > Understood, f

Re: Doc tweak for huge_pages?

2018-01-23 Thread Justin Pryzby
On Wed, Jan 24, 2018 at 07:46:41AM +0100, Catalin Iacob wrote: > I see Peter assigned himself as committer, some more information below > for him to decide on the strength of the anti THP message. Thanks for digging this up! > And it would be good if somebody could run benchmarks on pre 4.6 and >

JIT compiling with LLVM v9.0

2018-01-23 Thread Andres Freund
Hi, I've spent the last weeks working on my LLVM compilation patchset. In the course of that I *heavily* revised it. While still a good bit away from committable, it's IMO definitely not a prototype anymore. There's too many small changes, so I'm only going to list the major things. A good bit of

Re: [HACKERS] Subscription code improvements

2018-01-23 Thread Masahiko Sawada
On Tue, Jan 23, 2018 at 7:58 AM, Stephen Frost wrote: > Greetings, > > * Michael Paquier (michael.paqu...@gmail.com) wrote: >> On Fri, Aug 18, 2017 at 2:09 PM, Masahiko Sawada >> wrote: >> > On Thu, Aug 17, 2017 at 8:17 PM, Masahiko Sawada >> > wrote: >> >> On Thu, Aug 17, 2017 at 9:10 AM, Pet

Help needed in using 'on_dsm_detach' callback

2018-01-23 Thread Gaddam Sai Ram
Hello people, We are trying to build an in-memory index in postgres using dsa. Here is how we implemented dsa part. We have PROC_DSA_AREA global variable(Process specific DSA Pointer) We have a piece of traditional postgres shared memory to store dsa_handle Each process t

Re: Regarding ambulkdelete, amvacuumcleanup index methods

2018-01-23 Thread Masahiko Sawada
On Wed, Jan 24, 2018 at 1:27 PM, Abinaya k wrote: > Hai all, > We are building In-memory index extension for postgres. We would > capture table inserts, updates, deletes using triggers. During vacuum > operation, postgres would give calls to ambulkdelete, amvacuumcleanup (as > part of index

<    1   2