Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2021-02-13 Thread Joel Jacobson
Hi Mark, >On Fri, Feb 12, 2021, at 20:56, Mark Rofail wrote: >Attachments: >anyarray_anyelement_operators-v2.patch Some more code review comments: Comparing the v1 and v2 patch, I noticed this change in array_contains_elem(): + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); -+ if (oprr

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2021-02-13 Thread Joel Jacobson
Hi again Mark, On Fri, Feb 12, 2021, at 20:56, Mark Rofail wrote: >Attachments: >anyarray_anyelement_operators-v2.patch One regression test fails on my machine: make installcheck test opr_sanity ... FAILED 3994 ms 1 of 202 tests failed.

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2021-02-13 Thread Mark Rofail
Hey Joel, + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); > -+ if (oprresult) > ++ if (!locfcinfo->isnull && oprresult) > + return true; > + } > > Is checking !locfcinfo->isnull due to something new in v2, > or what is just a correction for a problem also in v1? > The “!locfcinfo->isnul

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2021-02-13 Thread Mark Rofail
Hey Joel, test opr_sanity ... FAILED > > AND binary_coercible(p2.opcintype, p1.amoplefttype)); > amopfamily | amopstrategy | amopopr > +--+- > -(0 rows) > + 2745 |5 |6105 > +(1 row) > > -- Operators that

Re: logical replication seems broken

2021-02-13 Thread Amit Kapila
On Fri, Feb 12, 2021 at 10:00 PM wrote: > > > On 02/12/2021 1:51 PM Amit Kapila wrote: > > > > > > On Fri, Feb 12, 2021 at 6:04 PM Erik Rijkers wrote: > > > > > > Hello, > > > > > > I am seeing errors in replication in a test program that I've been > > > running for years with very little chang

Re: pg13.2: invalid memory alloc request size NNNN

2021-02-13 Thread Greg Stark
I think to get a size of -4 you would be trying to read a varlena pointer pointing to four nul bytes. I bet if you run dd on the corresponding block you'll find a chunk of nuls in the page. That perhaps makes sense with ZFS where if a new page was linked to the tree but never written it would be an

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2021-02-13 Thread Joel Jacobson
On Fri, Feb 12, 2021, at 20:56, Mark Rofail wrote: >Attachments: >anyarray_anyelement_operators-v2.patch I've created a quite sophisticated test in PL/pgSQL, that takes advantage of all the test data produced by the official PostgreSQL regression test suite. It goes through all tables and columns

Re: logical replication seems broken

2021-02-13 Thread Erik Rijkers
> On 02/13/2021 11:49 AM Amit Kapila wrote: > > On Fri, Feb 12, 2021 at 10:00 PM wrote: > > > > > On 02/12/2021 1:51 PM Amit Kapila wrote: > > > > > > On Fri, Feb 12, 2021 at 6:04 PM Erik Rijkers wrote: > > > > > > > > I am seeing errors in replication in a test program that I've been > > > >

Re: Preventing free space from being reused

2021-02-13 Thread John Naylor
On Fri, Feb 12, 2021 at 6:21 PM Noah Bergbauer wrote: > > A btree index on the same column is 700x the size of BRIN, or 10% of relation itself. It does not perform significantly better than BRIN. The issue here is twofold: not only does slotting these tuples into older pages significantly reduce t

Re: public schema default ACL

2021-02-13 Thread Noah Misch
I'm attaching the patch for $SUBJECT, which applies atop the four patches from the two other threads below. For convenience of testing, I've included a rollup patch, equivalent to applying all five patches. On Sat, Oct 31, 2020 at 09:35:18AM -0700, Noah Misch wrote: > More details on the semantic

Re: [HACKERS] Custom compression methods

2021-02-13 Thread Dilip Kumar
On Thu, Feb 11, 2021 at 8:17 PM Robert Haas wrote: > > On Thu, Feb 11, 2021 at 7:36 AM Dilip Kumar wrote: > > W.R.T the attached patch, In HeapTupleHeaderGetDatum, we don't even > > attempt to detoast if there is no external field in the tuple, in POC > > I have got rid of that check, but I think

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2021-02-13 Thread Joel Jacobson
On Sat, Feb 13, 2021, at 12:35, Joel Jacobson wrote: >psql:type-test.sql:165: WARNING: >SQL queries produced different results: >SELECT '285970053'::pg_catalog."numeric" = >ANY(ARRAY['285970053']::pg_catalog.float4[]) >false >SELECT '285970053'::pg_catalog."numeric" <<@ >ARRAY['285970053']::pg_ca

Re: partial heap only tuples

2021-02-13 Thread Andres Freund
Hi, On 2021-02-09 18:48:21 +, Bossart, Nathan wrote: > In order to be eligible for cleanup, the final tuple in the > corresponding PHOT/HOT chain must also be eligible for cleanup, or all > indexes must have been updated later in the chain before any visible > tuples. This sounds like it migh

Re: repeated decoding of prepared transactions

2021-02-13 Thread Andres Freund
Hi, On 2021-02-10 08:02:17 +0530, Amit Kapila wrote: > On Wed, Feb 10, 2021 at 12:08 AM Robert Haas wrote: > > > > On Tue, Feb 9, 2021 at 6:57 AM Amit Kapila wrote: > > > I think similar happens without any of the work done in PG-14 as well > > > if we restart the apply worker before the commit

Re: repeated decoding of prepared transactions

2021-02-13 Thread Petr Jelinek
> > On 13 Feb 2021, at 17:32, Andres Freund wrote: > > Hi, > > On 2021-02-10 08:02:17 +0530, Amit Kapila wrote: >> On Wed, Feb 10, 2021 at 12:08 AM Robert Haas wrote: >>> >> If by successfully confirmed, you mean that once the subscriber node >> has received, it won't be sent again then as fa

Re: repeated decoding of prepared transactions

2021-02-13 Thread Andres Freund
Hi, On 2021-02-13 17:37:29 +0100, Petr Jelinek wrote: > Agreed, if we relied purely on flush location of a slot, there would > be no need for origins to track the lsn. And we would be latency bound replicating transactions, which'd not be fun for single-insert ones for example... > AFAIK this i

Re: snowball update

2021-02-13 Thread Oleg Bartunov
On Fri, Feb 12, 2021 at 1:14 PM Peter Eisentraut wrote: > > There is another snowball release out, and I have prepared a patch to > integrate it. It's very big and mostly boring, so I'm not attaching it > here, but you can see it at > > https://github.com/petere/postgresql/commit/d0aa6c2148bcef10

Re: Some regular-expression performance hacking

2021-02-13 Thread Joel Jacobson
Hi Tom, On Thu, Feb 11, 2021, at 05:39, Tom Lane wrote: >0001-invent-rainbow-arcs.patch >0002-recognize-matchall-NFAs.patch Many thanks for working on the regex engine, this looks like an awesome optimization. To test the correctness of the patches, I thought it would be nice with some real-life

Re: Some regular-expression performance hacking

2021-02-13 Thread Tom Lane
"Joel Jacobson" writes: > In total, I scraped the first-page of some ~50k websites, > which produced 45M test rows to import, > which when GROUP BY pattern and flags was reduced > down to 235k different regex patterns, > and 1.5M different text string subjects. This seems like an incredibly usefu

How to get Relation tuples in C function

2021-02-13 Thread Patrick Handja
Greetings! I would like to know if there is a better way to pass a relation or if the relation name (CString) as a parameter in a C function and thus be able to manipulate its tuples. The documentation is available here: https://www.postgresql.org/docs/13/xfunc-c.html#id-1.8.3.13.11. But it is not

Re: Possible dereference null return (src/backend/replication/logical/reorderbuffer.c)

2021-02-13 Thread Ranier Vilela
Em sáb., 13 de fev. de 2021 às 01:07, Zhihong Yu escreveu: > Hi, > How about the following patch ? > > ReorderBufferSetBaseSnapshot() can return a bool to indicate whether the > base snapshot is set up. > > For the call by SnapBuildCommitTxn(), it seems xid is top transaction. So > the return val

Re: pg_cryptohash_final possible out-of-bounds access (per Coverity)

2021-02-13 Thread Ranier Vilela
Em sex., 12 de fev. de 2021 às 22:47, Michael Paquier escreveu: > On Fri, Feb 12, 2021 at 03:21:40PM +0900, Kyotaro Horiguchi wrote: > > The v3 drops the changes of the uuid_ossp contrib. I'm not sure the > > change of scram_HMAC_final is needed. > > Meaning that v3 would fail to compile uuid-os

Re: Possible dereference null return (src/backend/replication/logical/reorderbuffer.c)

2021-02-13 Thread Ranier Vilela
Em sáb., 13 de fev. de 2021 às 17:35, Ranier Vilela escreveu: > > Em sáb., 13 de fev. de 2021 às 01:07, Zhihong Yu > escreveu: > >> Hi, >> How about the following patch ? >> >> ReorderBufferSetBaseSnapshot() can return a bool to indicate whether the >> base snapshot is set up. >> >> For the call

Re: Possible dereference null return (src/backend/replication/logical/reorderbuffer.c)

2021-02-13 Thread Zhihong Yu
Hi, + (errmsg("BaseSnapshot cant't be setup at point %X/%X", + (uint32) (lsn >> 32), (uint32) lsn), +errdetail("Top transaction is running."))); Did you mean this errdetail: Top transaction is not running. Cheers On Sat, Feb 13, 2021 at 12:34

Re: Possible dereference null return (src/backend/replication/logical/reorderbuffer.c)

2021-02-13 Thread Ranier Vilela
Em sáb., 13 de fev. de 2021 às 17:48, Zhihong Yu escreveu: > Hi, > + (errmsg("BaseSnapshot cant't be setup at point %X/%X", > + (uint32) (lsn >> 32), (uint32) lsn), > +errdetail("Top transaction is running."))); > > Did you mean this errdetail:

Re: Possible dereference null return (src/backend/replication/logical/reorderbuffer.c)

2021-02-13 Thread Zhihong Yu
Hi, Patch v4 corrects a small typo: + (errmsg("BaseSnapshot cant't be setup at point %X/%X", Cheers On Sat, Feb 13, 2021 at 12:58 PM Ranier Vilela wrote: > Em sáb., 13 de fev. de 2021 às 17:48, Zhihong Yu > escreveu: > >> Hi, >> + (errmsg("BaseSnapshot cant't be set

Re: Some regular-expression performance hacking

2021-02-13 Thread Tom Lane
"Joel Jacobson" writes: > No is_match differences were detected, good! > However, there were 23 cases where what got captured differed: These all stem from the same oversight: checkmatchall() was being too cavalier by ignoring "pseudocolor" arcs, which are arcs that match start-of-string or end-o

Re: How to get Relation tuples in C function

2021-02-13 Thread Tom Lane
Patrick Handja writes: > I would like to know if there is a better way to pass a relation or if the > relation name (CString) as a parameter in a C function and thus be able to > manipulate its tuples. The documentation is available here: > https://www.postgresql.org/docs/13/xfunc-c.html#id-1.8.3.

Re: WIP: WAL prefetch (another approach)

2021-02-13 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2021-02-12 00:42:04 +0100, Tomas Vondra wrote: > > Yeah, that's a good point. I think it'd make sense to keep track of recent > > FPIs and skip prefetching such blocks. But how exactly should we implement > > that, how many blocks do we n

Re: Detecting pointer misalignment (was Re: pgsql: Implementation of subscripting for jsonb)

2021-02-13 Thread Tom Lane
I wrote: > Alexander Korotkov writes: >> On Fri, Feb 12, 2021 at 8:19 PM Tom Lane wrote: >>> Once this does settle, should we consider back-patching so that it's >>> possible to run alignment checks in the back branches too? >> +1 > Let's make sure we have a clean set of builds and then do that

Re: pg_cryptohash_final possible out-of-bounds access (per Coverity)

2021-02-13 Thread Michael Paquier
On Sat, Feb 13, 2021 at 05:37:32PM -0300, Ranier Vilela wrote: > IMO there is no necessity in back-patching. You are missing the point here. What you are proposing here would not be backpatched. However, reusing the same words as upthread, this has a cost in terms of *future* maintenance. In sh

Re: pg_cryptohash_final possible out-of-bounds access (per Coverity)

2021-02-13 Thread Ranier Vilela
Em sáb., 13 de fev. de 2021 às 20:32, Michael Paquier escreveu: > On Sat, Feb 13, 2021 at 05:37:32PM -0300, Ranier Vilela wrote: > > IMO there is no necessity in back-patching. > > You are missing the point here. What you are proposing here would not > be backpatched. However, reusing the same

Re: Detecting pointer misalignment (was Re: pgsql: Implementation of subscripting for jsonb)

2021-02-13 Thread Alexander Korotkov
On Sun, Feb 14, 2021 at 1:59 AM Tom Lane wrote: > I wrote: > > Alexander Korotkov writes: > >> On Fri, Feb 12, 2021 at 8:19 PM Tom Lane wrote: > >>> Once this does settle, should we consider back-patching so that it's > >>> possible to run alignment checks in the back branches too? > > >> +1 > >

Re: How to get Relation tuples in C function

2021-02-13 Thread Andy Fan
On Sun, Feb 14, 2021 at 5:23 AM Tom Lane wrote: > Patrick Handja writes: > > I would like to know if there is a better way to pass a relation or if > the > > relation name (CString) as a parameter in a C function and thus be able > to > > manipulate its tuples. The documentation is available her

Re: 64-bit XIDs in deleted nbtree pages

2021-02-13 Thread Peter Geoghegan
On Fri, Feb 12, 2021 at 10:27 PM Victor Yegorov wrote: > I'd like to outline one relevant case. > > Quite often bulk deletes are done on a time series data (oldest) and > effectively > removes a continuous chunk of data at the (physical) beginning of the table, > this is especially true for the a

Re: 64-bit XIDs in deleted nbtree pages

2021-02-13 Thread Peter Geoghegan
On Fri, Feb 12, 2021 at 9:04 PM Peter Geoghegan wrote: > On Fri, Feb 12, 2021 at 8:38 PM Masahiko Sawada wrote: > > I agree that there already are huge problems in that case. But I think > > we need to consider an append-only case as well; after bulk deletion > > on an append-only table, vacuum d