Re: 64-bit XIDs in deleted nbtree pages

2021-02-12 Thread Victor Yegorov
сб, 13 февр. 2021 г. в 05:39, Masahiko Sawada : > > (BTW, I've been using txid_current() for my own "laptop testing", as a > > way to work around this issue.) > > > > * More generally, if you really can't do recycling of pages that you > > deleted during the last VACUUM during this VACUUM

Re: Support ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION ... syntax

2021-02-12 Thread japin
Thanks for your review again. On Wed, 10 Feb 2021 at 21:49, Bharath Rupireddy wrote: > On Fri, Feb 5, 2021 at 6:51 PM japin wrote: >> On Fri, 05 Feb 2021 at 17:50, Bharath Rupireddy >> wrote: >> We will get cell == NULL when we iterate all items in publist. I use it >> to check whether the

Re: Why do we have MakeSingleTupleTableSlot instead of not using MakeTupleTableSlot?

2021-02-12 Thread Tom Lane
Bharath Rupireddy writes: > Right, we could as well have an inline function. My point was that why > do we need to wrap MakeTupleTableSlot inside MakeSingleTupleTableSlot > which just does nothing. As I said upthread, how about renaming > MakeTupleTableSlot to > MakeSingleTupleTableSlot which

Re: 64-bit XIDs in deleted nbtree pages

2021-02-12 Thread Peter Geoghegan
On Fri, Feb 12, 2021 at 8:38 PM Masahiko Sawada wrote: > I agree that there already are huge problems in that case. But I think > we need to consider an append-only case as well; after bulk deletion > on an append-only table, vacuum deletes heap tuples and index tuples, > marking some index pages

Re: 64-bit XIDs in deleted nbtree pages

2021-02-12 Thread Masahiko Sawada
On Thu, Feb 11, 2021 at 12:10 PM Peter Geoghegan wrote: > > On Wed, Feb 10, 2021 at 2:20 AM Masahiko Sawada wrote: > > Thank you for working on this! > > I'm glad that I finally found time for it! It seems like it'll make > things easier elsewhere. > > Attached is v3 of the index. I'll describe

Re: Why do we have MakeSingleTupleTableSlot instead of not using MakeTupleTableSlot?

2021-02-12 Thread Bharath Rupireddy
On Fri, Feb 12, 2021 at 9:37 PM Zhihong Yu wrote: > On Fri, Feb 12, 2021 at 5:44 AM Bharath Rupireddy > wrote: >> >> Hi, >> >> I wonder, is there a specific reason that MakeTupleTableSlot is >> wrapped up in MakeSingleTupleTableSlot without doing anything than >> just returning the slot created

Re: Possible dereference null return (src/backend/replication/logical/reorderbuffer.c)

2021-02-12 Thread Zhihong Yu
Hi, How about the following patch ? ReorderBufferSetBaseSnapshot() can return a bool to indicate whether the base snapshot is set up. For the call by SnapBuildCommitTxn(), it seems xid is top transaction. So the return value doesn't need to be checked. Cheers On Fri, Feb 12, 2021 at 6:40 PM

Re: [PoC] Non-volatile WAL buffer

2021-02-12 Thread Masahiko Sawada
On Thu, Jan 28, 2021 at 1:41 AM Tomas Vondra wrote: > > On 1/25/21 3:56 AM, Masahiko Sawada wrote: > >> > >> ... > >> > >> On 1/21/21 3:17 AM, Masahiko Sawada wrote: > >>> ... > >>> > >>> While looking at the two methods: NTT and simple-no-buffer, I realized > >>> that in XLogFlush(), NTT patch

Re: [DOC] add missing "[ NO ]" to various "DEPENDS ON" synopses

2021-02-12 Thread Michael Paquier
On Fri, Feb 12, 2021 at 10:32:14AM +0900, Ian Lawrence Barwick wrote: > In the documentation, the "[ NO ]" option is listed in the synopsis for > ALTER TRIGGER and ALTER FUNCTION, but not the others. > Trivial patch attached. There are two flavors to cover for 6 commands per gram.y, and you are

Re: Possible dereference null return (src/backend/replication/logical/reorderbuffer.c)

2021-02-12 Thread Michael Paquier
On Fri, Feb 12, 2021 at 03:56:02PM +0900, Kyotaro Horiguchi wrote: > If the return from the first call is a subtransaction, the second call > always obtain the top transaction. If the top transaction actualy did > not exist, it's rather the correct behavior to cause SEGV, than > creating a bogus

Re: Tightening up allowed custom GUC names

2021-02-12 Thread Michael Paquier
On Thu, Feb 11, 2021 at 02:50:13PM -0500, Robert Haas wrote: > +1 for not back-patching whatever we do here. +1. -- Michael signature.asc Description: PGP signature

Re: some pointless HeapTupleHeaderIndicatesMovedPartitions calls

2021-02-12 Thread Michael Paquier
On Fri, Feb 12, 2021 at 04:42:26PM +, Anastasia Lubennikova wrote: > I wonder, why this patch hangs on commitfest for so long. > The idea of the fix is clear, the code is correct and all tests pass, so, I > move it to ReadyForCommitter status. > > The new status of this patch is: Ready for

Re: pg_cryptohash_final possible out-of-bounds access (per Coverity)

2021-02-12 Thread Michael Paquier
On Fri, Feb 12, 2021 at 03:21:40PM +0900, Kyotaro Horiguchi wrote: > The v3 drops the changes of the uuid_ossp contrib. I'm not sure the > change of scram_HMAC_final is needed. Meaning that v3 would fail to compile uuid-ossp. v3 also produces compilation warnings in auth-scram.c. > In v2,

Re: [POC] verifying UTF-8 using SIMD instructions

2021-02-12 Thread John Naylor
On Mon, Feb 8, 2021 at 6:17 AM Heikki Linnakangas wrote: > > I also tested the fallback implementation from the simdjson library > (included in the patch, if you uncomment it in simdjson-glue.c): > > mixed | ascii > ---+--- > 447 |46 > (1 row) > > I think we should at least try

Re: Improvements and additions to COPY progress reporting

2021-02-12 Thread Justin Pryzby
--- a/doc/src/sgml/ref/analyze.sgml +++ b/doc/src/sgml/ref/analyze.sgml

Re: Detecting pointer misalignment (was Re: pgsql: Implementation of subscripting for jsonb)

2021-02-12 Thread Tom Lane
Alexander Korotkov writes: > On Fri, Feb 12, 2021 at 8:19 PM Tom Lane wrote: >> So maybe it'd better be __GNUC__ >= 6 not __GNUC__ >= 5. I think >> we can wait a little bit for more reports before messing with that, >> though. > I've rechecked this in the documentation. no_sanitize attribute

Re: Detecting pointer misalignment (was Re: pgsql: Implementation of subscripting for jsonb)

2021-02-12 Thread Tom Lane
I wrote: > Looking around at other recent reports, it looks like we'll need to tweak > the compiler version cutoffs a bit. I see for instance that spurfowl, > with gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, is whining: > ... > So maybe it'd better be __GNUC__ >= 6 not __GNUC__ >= 5. I

Re: PostgreSQL <-> Babelfish integration

2021-02-12 Thread Adam Brusselback
Just wanted to link to the discussion on this from HN for anyone intersted: https://news.ycombinator.com/item?id=26114281

Re: Detecting pointer misalignment (was Re: pgsql: Implementation of subscripting for jsonb)

2021-02-12 Thread Alexander Korotkov
On Fri, Feb 12, 2021 at 8:19 PM Tom Lane wrote: > I've updated buildfarm member longfin to use "-fsanitize=alignment > -fsanitize-trap=alignment", and it just got through a run successfully > with that. It'd be good perhaps if some other buildfarm owners > followed suit (mumble JIT coverage

Re: Preventing free space from being reused

2021-02-12 Thread Noah Bergbauer
> (My reaction to your previous thread was that it was simply a question > of blindly insisting on using BRIN indexes for a case that they're quite > badly adapted to. The better answer is to not use BRIN.) Apologies, perhaps I am completely misunderstanding the motivation for BRIN? >From the

Re: Preventing free space from being reused

2021-02-12 Thread Tom Lane
Noah Bergbauer writes: > I am working on a project where I do not want Postgres to reuse free space > in old pages (see > https://www.postgresql.org/message-id/flat/CABjy%2BRhbFu_Hs8ZEiOzaPaJSGB9jqFF0gDU5gtwCLiurG3NLjQ%40mail.gmail.com > for details). I found that the HEAP_INSERT_SKIP_FSM flag

Experimenting with redo batching

2021-02-12 Thread Thomas Munro
Hello, As a very simple exploration of the possible gains from batching redo records during replay, I tried to avoid acquiring and releasing buffers pins and locks while replaying records that touch the same page as the previous record. The attached experiment-grade patch works by trying to give

Preventing free space from being reused

2021-02-12 Thread Noah Bergbauer
Hello, I am working on a project where I do not want Postgres to reuse free space in old pages (see https://www.postgresql.org/message-id/flat/CABjy%2BRhbFu_Hs8ZEiOzaPaJSGB9jqFF0gDU5gtwCLiurG3NLjQ%40mail.gmail.com for details). I found that the HEAP_INSERT_SKIP_FSM flag accomplishes this. For a

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2021-02-12 Thread Joel Jacobson
Hi Mark, On Fri, Feb 12, 2021, at 20:56, Mark Rofail wrote: >Indeed you are right, to support the correct behaviour we have to use >@>>(anycompatiblearray, anycompatiblenonarry) and >this throws a sanity error >in opr_santiy since the left operand doesn't equal the gin opclass which is

Re: [BUG] Autovacuum not dynamically decreasing cost_limit and cost_delay

2021-02-12 Thread David Zhang
Thanks for the patch, Mead. For 'MAXVACUUMCOSTLIMIT", it would be nice to follow the current GUC pattern to do define a constant. For example, the constant "MAX_KILOBYTES" is defined in guc.h, with a pattern like, "MAX_" to make it easy to read. Best regards, David On 2021-02-08 6:48

Re: PostgreSQL <-> Babelfish integration

2021-02-12 Thread Peter Geoghegan
On Fri, Feb 12, 2021 at 11:13 AM Matthias van de Meent wrote: > I agree. I believe that Babelfish's efforts can be compared with the > zedstore and zheap efforts: they require work in core before they can > be integrated or added as an extension that could replace the normal > heap tableam, and

Re: PostgreSQL <-> Babelfish integration

2021-02-12 Thread Peter Geoghegan
On Fri, Feb 12, 2021 at 11:04 AM Álvaro Hernández wrote: > I'm sure if we embrace an open and honest conversation, we will be > able to figure out what the integration costs are even before the source > code gets published. As I said, this goes beyond the very technical > detail of source

Re: PostgreSQL <-> Babelfish integration

2021-02-12 Thread Matthias van de Meent
On Fri, 12 Feb 2021 at 19:44, Peter Geoghegan wrote: > > On Fri, Feb 12, 2021 at 10:26 AM Álvaro Hernández wrote: > > As I stated in the mentioned post, I believe Babelfish is a very > > welcomed addition to the PostgreSQL ecosystem. It allows PostgreSQL to > > reach other users, other use

Re: PostgreSQL <-> Babelfish integration

2021-02-12 Thread Álvaro Hernández
On 12/2/21 19:44, Peter Geoghegan wrote: > On Fri, Feb 12, 2021 at 10:26 AM Álvaro Hernández wrote: >> As I stated in the mentioned post, I believe Babelfish is a very >> welcomed addition to the PostgreSQL ecosystem. It allows PostgreSQL to >> reach other users, other use cases, other

Re: PostgreSQL <-> Babelfish integration

2021-02-12 Thread Peter Geoghegan
On Fri, Feb 12, 2021 at 10:26 AM Álvaro Hernández wrote: > As I stated in the mentioned post, I believe Babelfish is a very > welcomed addition to the PostgreSQL ecosystem. It allows PostgreSQL to > reach other users, other use cases, other markets; something which in my > opinion PostgreSQL

PostgreSQL <-> Babelfish integration

2021-02-12 Thread Álvaro Hernández
    I would like to share my thoughts in the list about the potential PostgreSQL <-> Babelfish integration. There is already a thread about protocol hooks [1], but I'd like to offer my PoV from a higher level perspective and keep that thread for the technical aspects of the protocol hooks. This

Re: pg13.2: invalid memory alloc request size NNNN

2021-02-12 Thread Justin Pryzby
On Fri, Feb 12, 2021 at 06:44:54PM +0100, Tomas Vondra wrote: > > (gdb) p len > > $1 = -4 > > > > This VM had some issue early today and I killed the VM, causing PG to > > execute > > recovery. I'm tentatively blaming that on zfs, so this could conceivably > > be a > > data error (although

Re: Trigger execution role

2021-02-12 Thread Tom Lane
Isaac Morland writes: > I was trying to use triggers, and ran into something I hadn't realized > until now: triggers run, not as the owner of the table, but as the user who > is doing the insert/update/delete. If you don't want that, you can make the trigger function SECURITY DEFINER. If we

Re: WIP: WAL prefetch (another approach)

2021-02-12 Thread Tomas Vondra
On 2/12/21 5:46 AM, Andres Freund wrote: Hi, On 2021-02-12 00:42:04 +0100, Tomas Vondra wrote: Yeah, that's a good point. I think it'd make sense to keep track of recent FPIs and skip prefetching such blocks. But how exactly should we implement that, how many blocks do we need to track? If

Re: Improvements and additions to COPY progress reporting

2021-02-12 Thread Matthias van de Meent
On Fri, 12 Feb 2021 at 13:40, Bharath Rupireddy wrote: > > On Fri, Feb 12, 2021 at 5:40 PM Matthias van de Meent > wrote: > > > > On Fri, 12 Feb 2021 at 12:23, Matthias van de Meent > > wrote: > > > > > > On Thu, 11 Feb 2021 at 15:44, Bharath Rupireddy > > > wrote: > > > > > > > > > > > > On

Re: pg13.2: invalid memory alloc request size NNNN

2021-02-12 Thread Tomas Vondra
On 2/12/21 2:48 AM, Justin Pryzby wrote: ts=# \errverbose ERROR: XX000: invalid memory alloc request size 18446744073709551613 #0 pg_re_throw () at elog.c:1716 #1 0x00a33b12 in errfinish (filename=0xbff20e "mcxt.c", lineno=959, funcname=0xbff2db <__func__.6684> "palloc") at

Trigger execution role

2021-02-12 Thread Isaac Morland
I was trying to use triggers, and ran into something I hadn't realized until now: triggers run, not as the owner of the table, but as the user who is doing the insert/update/delete. It seems to me that for a lot of the suggested uses of triggers this is not the desired behaviour. For example, in

Re: Detecting pointer misalignment (was Re: pgsql: Implementation of subscripting for jsonb)

2021-02-12 Thread Tom Lane
I've updated buildfarm member longfin to use "-fsanitize=alignment -fsanitize-trap=alignment", and it just got through a run successfully with that. It'd be good perhaps if some other buildfarm owners followed suit (mumble JIT coverage mumble). Looking around at other recent reports, it looks

Re: WIP: document the hook system

2021-02-12 Thread Anastasia Lubennikova
On 17.01.2021 16:53, Magnus Hagander wrote: On Fri, Jan 15, 2021 at 8:28 AM Peter Eisentraut wrote: On 2020-12-31 04:28, David Fetter wrote: This could probably use a lot of filling in, but having it in the actual documentation beats needing to know folklore even to know that the capability

Re: some pointless HeapTupleHeaderIndicatesMovedPartitions calls

2021-02-12 Thread Anastasia Lubennikova
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:not tested I wonder, why this patch hangs on commitfest for so long. The idea

Re: logical replication seems broken

2021-02-12 Thread er
> On 02/12/2021 1:51 PM Amit Kapila wrote: > > > On Fri, Feb 12, 2021 at 6:04 PM Erik Rijkers wrote: > > > > Hello, > > > > I am seeing errors in replication in a test program that I've been running > > for years with very little change (since 2017, really [1]). Hi, Here is a test program.

Re: Possible dereference after null check (src/backend/executor/ExecUtils.c)

2021-02-12 Thread Ranier Vilela
Em sex., 12 de fev. de 2021 às 03:28, Kyotaro Horiguchi < horikyota@gmail.com> escreveu: > At Wed, 10 Feb 2021 19:54:46 -0300, Ranier Vilela > wrote in > > Hi, > > > > Per Coverity. > > > > The functions ExecGetInsertedCols and ExecGetUpdatedCols at ExecUtils.c > > only are safe to call if

Re: Possible dereference null return (src/backend/replication/logical/reorderbuffer.c)

2021-02-12 Thread Ranier Vilela
Em sex., 12 de fev. de 2021 às 03:56, Kyotaro Horiguchi < horikyota@gmail.com> escreveu: > At Wed, 10 Feb 2021 20:12:38 -0300, Ranier Vilela > wrote in > > Hi, > > > > Per Coverity. > > > > If xid is a subtransaction, the setup of base snapshot on the top-level > > transaction, > > can be

Re: Why do we have MakeSingleTupleTableSlot instead of not using MakeTupleTableSlot?

2021-02-12 Thread Zhihong Yu
Hi, MakeSingleTupleTableSlot can be defined as a macro, calling MakeTupleTableSlot(). Cheers On Fri, Feb 12, 2021 at 5:44 AM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > Hi, > > I wonder, is there a specific reason that MakeTupleTableSlot is > wrapped up in

Re: Detecting pointer misalignment (was Re: pgsql: Implementation of subscripting for jsonb)

2021-02-12 Thread Alexander Korotkov
Hi, Thomas! On Fri, Feb 12, 2021 at 12:04 AM Thomas Munro wrote: > On Tue, Feb 9, 2021 at 1:34 PM Alexander Korotkov > wrote: > > Could we have both cfbot + buildfarm animals? > For cfbot, yeah it does seem like a good idea to throw whatever code > sanitiser stuff we can into the automated

Re: Dump public schema ownership & seclabels

2021-02-12 Thread Noah Misch
On Thu, Feb 11, 2021 at 04:08:34AM -0800, Noah Misch wrote: > On Sun, Jan 17, 2021 at 12:00:06PM +0100, Vik Fearing wrote: > > On 1/17/21 10:41 AM, Noah Misch wrote: > > > On Sat, Jan 16, 2021 at 02:05:43PM +0100, Vik Fearing wrote: > > >> On 12/30/20 12:59 PM, Noah Misch wrote: > > >>> On Tue,

Re: Extensibility of the PostgreSQL wire protocol

2021-02-12 Thread Fabrízio de Royes Mello
On Thu, Feb 11, 2021 at 12:07 PM Tom Lane wrote: > > Robert Haas writes: > > On Thu, Feb 11, 2021 at 9:42 AM Jonah H. Harris wrote: > >> As Jan said in his last email, they're not proposing all the different > >> aspects needed. In fact, nothing has actually been proposed yet. This > >> is an

Why do we have MakeSingleTupleTableSlot instead of not using MakeTupleTableSlot?

2021-02-12 Thread Bharath Rupireddy
Hi, I wonder, is there a specific reason that MakeTupleTableSlot is wrapped up in MakeSingleTupleTableSlot without doing anything than just returning the slot created by MakeTupleTableSlot? Do we really need MakeSingleTupleTableSlot? Can't we just use MakeTupleTableSlot directly? Am I missing

Re: Parallel INSERT (INTO ... SELECT ...)

2021-02-12 Thread Amit Langote
On Thu, Feb 11, 2021 at 4:43 PM Greg Nancarrow wrote: > On Thu, Feb 11, 2021 at 5:33 PM Greg Nancarrow wrote: > > On Tue, Feb 9, 2021 at 1:04 AM Amit Langote wrote: > > > > > > * I think that the concerns raised by Tsunakawa-san in: > > > > > >

Re: PostgreSQL and Flashback Database

2021-02-12 Thread Philippe Beaudoin
Hi Didier, Have you ever had a look at the E-Maj extension. Depending on the features you are really looking for, it may fit the needs. Here are some pointers : - github repo for the extension : https://github.com/dalibo/emaj - github repo for the web client :

Re: logical replication seems broken

2021-02-12 Thread Amit Kapila
On Fri, Feb 12, 2021 at 6:04 PM Erik Rijkers wrote: > > Hello, > > I am seeing errors in replication in a test program that I've been running > for years with very little change (since 2017, really [1]). > > The symptom: > HEAD-replication fails (most of the time) when cascading 3 instances >

Re: Parallel INSERT (INTO ... SELECT ...)

2021-02-12 Thread Greg Nancarrow
On Fri, Feb 12, 2021 at 5:30 PM tsunakawa.ta...@fujitsu.com wrote: > > > If I disable parallel_leader_participation. > > > > For max_parallel_workers_per_gather = 4, It still have performance > > degradation. > > > > For max_parallel_workers_per_gather = 2, the performance degradation will > >

Re: Improvements and additions to COPY progress reporting

2021-02-12 Thread Bharath Rupireddy
On Fri, Feb 12, 2021 at 5:40 PM Matthias van de Meent wrote: > > On Fri, 12 Feb 2021 at 12:23, Matthias van de Meent > wrote: > > > > On Thu, 11 Feb 2021 at 15:44, Bharath Rupireddy > > wrote: > > > > > > > > > On Thu, Feb 11, 2021, 8:08 PM Josef Šimánek > > > wrote: > > >> I have split it

logical replication seems broken

2021-02-12 Thread Erik Rijkers
Hello, I am seeing errors in replication in a test program that I've been running for years with very little change (since 2017, really [1]). The symptom: HEAD-replication fails (most of the time) when cascading 3 instances (master+2 replicas). HEAD-replication works with 2 instances

Re: Improvements and additions to COPY progress reporting

2021-02-12 Thread Matthias van de Meent
On Fri, 12 Feb 2021 at 12:23, Matthias van de Meent wrote: > > On Thu, 11 Feb 2021 at 15:44, Bharath Rupireddy > wrote: > > > > > > On Thu, Feb 11, 2021, 8:08 PM Josef Šimánek wrote: > >> I have split it since it should be the start of progress reporting > >> testing at all. If you better

Re: Parallel INSERT (INTO ... SELECT ...)

2021-02-12 Thread Zhihong Yu
Greg: Thanks for more debugging. Cheers On Thu, Feb 11, 2021 at 9:43 PM Greg Nancarrow wrote: > On Fri, Feb 12, 2021 at 3:21 PM Zhihong Yu wrote: > > > > Greg: > > bq. we should just return parsetree->hasModifyingCTE at this point, > > > > Maybe you can clarify a bit. > > The if

Re: Improvements and additions to COPY progress reporting

2021-02-12 Thread Matthias van de Meent
On Thu, 11 Feb 2021 at 15:44, Bharath Rupireddy wrote: > > > On Thu, Feb 11, 2021, 8:08 PM Josef Šimánek wrote: >> I have split it since it should be the start of progress reporting >> testing at all. If you better consider this as part of COPY testing, >> feel free to move it to already

Re: [HACKERS] logical decoding of two-phase transactions

2021-02-12 Thread Amit Kapila
On Fri, Feb 12, 2021 at 12:29 PM osumi.takami...@fujitsu.com wrote: > > On Thursday, February 11, 2021 5:10 PM Peter Smith > wrote: > > Please find attached the new 2PC patch set v39* > I started to review the patchset > so, let me give some comments I have at this moment. > > (1) > > File :

Re: repeated decoding of prepared transactions

2021-02-12 Thread Amit Kapila
On Fri, Feb 12, 2021 at 1:10 AM Robert Haas wrote: > > On Thu, Feb 11, 2021 at 5:37 AM Amit Kapila wrote: > > to explain the exact case to you which is not very apparent. The basic > > idea is that we ship/replay all transactions where commit happens > > after the snapshot has a consistent state

Re: Asynchronous Append on postgres_fdw nodes.

2021-02-12 Thread Kyotaro Horiguchi
At Wed, 10 Feb 2021 21:31:15 +0900, Etsuro Fujita wrote in > On Wed, Feb 10, 2021 at 7:31 PM Etsuro Fujita wrote: > > Attached is an updated version of the patch. Sorry for the delay. > > I noticed that I forgot to add new files. :-(. Please find attached > an updated patch. Thanks for