Re: pg_replication_origin_session_setup and superuser

On Tue, Feb 16, 2021 at 07:54:32AM +, Zohar Gofer wrote: > Thanks. This seems to be the fix we need. > Would it be possible to push it to previous versions? 12 or 13? New features don't go into stable branches, only bug fixes do. And this is not a bug fix, but a feature. -- Michael signatur

Re: [HACKERS] Custom compression methods

On Mon, Feb 15, 2021 at 1:58 AM Justin Pryzby wrote: > > On Sun, Feb 14, 2021 at 12:49:40PM -0600, Justin Pryzby wrote: > > On Wed, Feb 10, 2021 at 04:56:17PM -0500, Robert Haas wrote: > > > Small delta patch with a few other suggested changes attached. > > > > Robert's fixup patch caused the CI t

RE: [PoC] Non-volatile WAL buffer

From: Takashi Menjo > I made a new page at PostgreSQL Wiki to gather and summarize information and > discussion about PMEM-backed WAL designs and implementations. Some parts of > the page are TBD. I will continue to maintain the page. Requests are welcome. > > Persistent Memory for WAL > https

Re: [POC] Fast COPY FROM command for the table with foreign partitions

On 2/15/21 1:31 PM, Amit Langote wrote: Tsunakawa-san, Andrey, +static void +postgresBeginForeignCopy(ModifyTableState *mtstate, + ResultRelInfo *resultRelInfo) +{ ... + if (resultRelInfo->ri_RangeTableIndex == 0) + { + ResultRelInfo *rootResultRelInfo = resultR

Re: Parallel INSERT (INTO ... SELECT ...)

On Mon, Feb 8, 2021 at 8:13 PM Hou, Zhijie wrote: > > > > Did it actually use a parallel plan in your testing? > > > When I ran these tests with the Parallel INSERT patch applied, it did > > > not naturally choose a parallel plan for any of these cases. > > > > Yes, these cases pick parallel plan

Re: [PoC] Non-volatile WAL buffer

Hi Takayuki, Thank you for your helpful comments. In "Allocates WAL buffers on shared buffers", "shared buffers" should be > DRAM because shared buffers in Postgres means the buffer cache for database > data. > That's true. Fixed. > I haven't tracked the whole thread, but could you collect inf

ERROR: "ft1" is of the wrong type.

Hello, If I invoked a wrong ALTER TABLE command like this, I would see an unexpected error. =# ALTER TABLE ATTACH PARTITION ERROR: "ft1" is of the wrong type The cause is that ATWrongRelkidError doesn't handle ATT_TABLE | ATT_ATT_PARTITIONED_INDEX. After checking all callers of ATSimpleP

Re: Tid scan improvements

On Thu, 4 Feb 2021 at 23:51, David Rowley wrote: > Updated patch attached. I made another pass over this patch and did a bit of renaming work around the heap_* functions and the tableam functions. I think the new names are a bit more aligned to the existing names. I don't really see anything els

Re: POC: postgres_fdw insert batching

On Tue, Feb 16, 2021 at 1:36 AM Tomas Vondra wrote: > On 2/5/21 3:52 AM, Amit Langote wrote: > > Tsunakwa-san, > > > > On Mon, Jan 25, 2021 at 1:21 PM tsunakawa.ta...@fujitsu.com > > wrote: > >> From: Amit Langote > >>> Yes, it can be simplified by using a local join to prevent the update of >

RE: pg_replication_origin_session_setup and superuser

Thanks. This seems to be the fix we need. Would it be possible to push it to previous versions? 12 or 13? Zohar -Original Message- From: Michael Paquier Sent: Tuesday, February 16, 2021 2:52 AM To: Zohar Gofer Cc: pgsql-hackers@lists.postgresql.org Subject: Re: pg_replication_origin_se

Re: [PATCH] pg_hba.conf error messages for logical replication connections

On Tue, Feb 2, 2021 at 1:43 AM Paul Martinez wrote: > > On Fri, Jan 29, 2021 at 8:41 PM Amit Kapila wrote: > > > > Yeah, hints or more details might improve the situation but I am not > > sure we want to add more branching here. Can we write something > > similar to HOSTNAME_LOOKUP_DETAIL for hin

Re: progress reporting for partitioned REINDEX

On Tue, 16 Feb 2021, 07:42 Justin Pryzby, wrote: > > It looks like we missed this in a6642b3ae. > > I think it's an odd behavior of pg_stat_progress_create_index to > simultaneously > show the global progress as well as the progress for the current partition ... > > It seems like for partitioned

Re: 64-bit XIDs in deleted nbtree pages

On Tue, Feb 16, 2021 at 3:52 PM Peter Geoghegan wrote: > > On Mon, Feb 15, 2021 at 7:26 PM Peter Geoghegan wrote: > > Actually, there is one more reason why I bring up idea 1 now: I want > > to hear your thoughts on the index AM API questions now, which idea 1 > > touches on. Ideally all of the d

Re: Parallel INSERT (INTO ... SELECT ...)

On Mon, Feb 15, 2021 at 4:39 PM Greg Nancarrow wrote: > On Sat, Feb 13, 2021 at 12:17 AM Amit Langote wrote: > > On Thu, Feb 11, 2021 at 4:43 PM Greg Nancarrow wrote: > > > Actually, I tried adding the following in the loop that checks the > > > parallel-safety of each partition and it seemed to

Re: A reloption for partitioned tables - parallel_workers

On Tue, 2021-02-16 at 16:29 +0900, Amit Langote wrote: > > I am +1 on allowing to override the degree of parallelism on a parallel > > append. If "parallel_workers" on the partitioned table is an option for > > that, it might be a simple solution. On the other hand, perhaps it would > > be less c

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

On Tue, Feb 16, 2021 at 12:01 PM David Rowley wrote: > On Fri, 12 Feb 2021 at 15:18, Andy Fan wrote: > > > > On Fri, Feb 12, 2021 at 9:02 AM David Rowley > wrote: > >> The reason I don't really like this is that it really depends where > >> you want to use RelOptInfo.notnullattrs. If someone w

Re: Performing partition pruning using row value

On 21.07.2020 11:24, kato-...@fujitsu.com wrote: So, after looking at these functions and modifying this patch, I would like to add this patch to the next I updated this patch and registered for the next CF . https://commitfest.postgresql.org/29/2654/ regards, sho kato Thank you for working

Re: [HACKERS] Custom compression methods

On Sat, Feb 13, 2021 at 8:14 PM Dilip Kumar wrote: > > On Thu, Feb 11, 2021 at 8:17 PM Robert Haas wrote: > > > > On Thu, Feb 11, 2021 at 7:36 AM Dilip Kumar wrote: > > > W.R.T the attached patch, In HeapTupleHeaderGetDatum, we don't even > > > attempt to detoast if there is no external field in

Re: [Proposal] Page Compression for OLTP

Hi, hackers I want to know whether this patch can be accepted by the community, that is, whether it is necessary for me to continue working for this Patch. If you have any suggestions, please feedback to me. Best Regards Chen Huajun

Re: ERROR: invalid spinlock number: 0

On 2021/02/16 15:50, Michael Paquier wrote: On Tue, Feb 16, 2021 at 12:43:42PM +0900, Fujii Masao wrote: On 2021/02/16 6:28, Andres Freund wrote: So what? It's just about free to initialize a spinlock, whether it's using the fallback implementation or not. Initializing upon walsender startup

Re: [Proposal] Page Compression for OLTP

> On 16 Feb 2021, at 15:45, chenhj wrote: > I want to know whether this patch can be accepted by the community, that is, > whether it is necessary for me to continue working for this Patch. > If you have any suggestions, please feedback to me. It doesn't seem like the patch has been registered

How to customize postgres for sharing read-only tables in multiple data-dirs between servers

Hi, Problem statement: I have to develop a solution in which a single source populates a table. Once the table is populated, it is considered as read-only and then we run many read-only queries on it. Such read-only tables are generated by multiple simulation runs: each simulation populates an

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

On Tue, Feb 16, 2021 at 10:03 PM Andy Fan wrote: > > > On Tue, Feb 16, 2021 at 12:01 PM David Rowley > wrote: > >> On Fri, 12 Feb 2021 at 15:18, Andy Fan wrote: >> > >> > On Fri, Feb 12, 2021 at 9:02 AM David Rowley >> wrote: >> >> The reason I don't really like this is that it really depends

Re: POC: postgres_fdw insert batching

On 2/16/21 10:25 AM, Amit Langote wrote: On Tue, Feb 16, 2021 at 1:36 AM Tomas Vondra wrote: On 2/5/21 3:52 AM, Amit Langote wrote: Tsunakwa-san, On Mon, Jan 25, 2021 at 1:21 PM tsunakawa.ta...@fujitsu.com wrote: From: Amit Langote Yes, it can be simplified by using a local join to pre

Re: libpq PQresultErrorMessage vs PQerrorMessage API issue

Craig Ringer writes: > This morning for the the umpteenth time I saw: > some error message: [blank here] > output from a libpq program. > That's because passing a NULL PGresult to PQgetResultErrorMessage() returns > "". But a NULL PGresult is a normal result from PQexec when it fails to > submit

Re: [Proposal] Page Compression for OLTP

At 2021-02-16 21:51:14, "Daniel Gustafsson" wrote: >> On 16 Feb 2021, at 15:45, chenhj wrote: > >> I want to know whether this patch can be accepted by the community, that is, >> whether it is necessary for me to continue working for this Patch. >> If you have any suggestions, please feedback

Re: PATCH: Batch/pipelining support for libpq

On 2021-Feb-16, Craig Ringer wrote: > FWIW I'm also thinking of revising the docs to mostly use the term > "pipeline" instead of "batch". Use "pipelining and batching" in the chapter > topic, and mention "batch" in the index, and add a para that explains how > to run batches on top of pipelining,

Re: Tid scan improvements

On Tue, Feb 16, 2021 at 10:22:41PM +1300, David Rowley wrote: > On Thu, 4 Feb 2021 at 23:51, David Rowley wrote: > > Updated patch attached. > > I made another pass over this patch and did a bit of renaming work > around the heap_* functions and the tableam functions. I think the new > names are

Re: proposal: schema variables

Hi út 2. 2. 2021 v 9:43 odesílatel Pavel Stehule napsal: > Hi > > rebase and set PK for pg_variable table > rebase Pavel > Regards > > Pavel > schema-variables-20200216.patch.gz Description: application/gzip

TRIM_ARRAY

The SQL standard defines a function called TRIM_ARRAY that surprisingly has syntax that looks like a function! So I implemented it using a thin wrapper around our array slice syntax. It is literally just ($1)[1:$2]. An interesting case that I decided to handle by explaining it in the docs is tha

Re: TRIM_ARRAY

On Tue, 16 Feb 2021 at 12:54, Vik Fearing wrote: > The SQL standard defines a function called TRIM_ARRAY that surprisingly > has syntax that looks like a function! So I implemented it using a thin > wrapper around our array slice syntax. It is literally just ($1)[1:$2]. > > An interesting case

Re: PoC/WIP: Extended statistics on expressions

On 1/27/21 12:02 PM, Dean Rasheed wrote: > On Fri, 22 Jan 2021 at 03:49, Tomas Vondra > wrote: >> >> Whooops. A fixed version attached. >> > > The change to pg_stats_ext_exprs isn't quite right, because now it > cross joins expressions and their stats, which leads to too many rows, > with the

Re: proposal: possibility to read dumped table's name from file

Hi rebase Regards Pavel diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml index bcbb7a25fb..f24b3b5262 100644 --- a/doc/src/sgml/ref/pg_dump.sgml +++ b/doc/src/sgml/ref/pg_dump.sgml @@ -956,6 +956,42 @@ PostgreSQL documentation + + --options-fi

Re: 64-bit XIDs in deleted nbtree pages

On Tue, Feb 16, 2021 at 4:17 AM Masahiko Sawada wrote: > Ugh, yes, I think it's a bug. I was actually thinking of a similar bug in nbtree deduplication when I spotted this one -- see commit 48e12913. The index AM API stuff is tricky. > When developing this feature, in an old version patch, we us

Re: [PATCH] pg_hba.conf error messages for logical replication connections

On Tue, Feb 16, 2021 at 2:22 AM Amit Kapila wrote: > > I don't think we need to update the error messages, it makes the code > a bit difficult to parse without much benefit. How about just adding > errdetail? See attached and let me know what you think? > Yeah, I think that looks good. Thanks! -

Re: 64-bit XIDs in deleted nbtree pages

On Tue, Feb 16, 2021 at 11:35 AM Peter Geoghegan wrote: > Isn't btvacuumcleanup() (or any other amvacuumcleanup() routine) > entitled to rely on the bulk delete stats being set in the way I've > described? I assumed that that was okay in general, but I haven't > tested parallel VACUUM specifically

Re: Trigger execution role

On Fri, 12 Feb 2021 at 12:58, Tom Lane wrote: > Isaac Morland writes: > > I was trying to use triggers, and ran into something I hadn't realized > > until now: triggers run, not as the owner of the table, but as the user > who > > is doing the insert/update/delete. > > If you don't want that, yo

Re: Tid scan improvements

Hi, On 2021-02-04 23:51:39 +1300, David Rowley wrote: > I ended up adding just two new API functions to table AM. > > void (*scan_set_tid_range) (TableScanDesc sscan, >ItemPointer mintid, >ItemPointer maxtid); > > and > bool (*scan_tid_range_nextslot) (TableScanDesc sscan, > ScanDirecti

Re: [Patch] ALTER SYSTEM READ ONLY

On Thu, Jan 28, 2021 at 7:17 AM Amul Sul wrote: > I am still on this. The things that worried me here are the wal records > sequence > being written in the startup process -- UpdateFullPageWrites() generate record > just before the recovery check-point record and XLogReportParameters() just > aft

[PATCH] Note effect of max_replication_slots on subscriber side in documentation.

Hey, all, The configuration parameter max_replication_slots is most notably used to control how many replication slots can be created on a server, but it also controls how many replication origins can be tracked on the subscriber side. This is noted in the Configuration Settings section in the Lo

Re: TRIM_ARRAY

On 2/16/21 7:32 PM, Isaac Morland wrote: > On Tue, 16 Feb 2021 at 12:54, Vik Fearing wrote: > >> The SQL standard defines a function called TRIM_ARRAY that surprisingly >> has syntax that looks like a function! So I implemented it using a thin >> wrapper around our array slice syntax. It is lit

Re: Trigger execution role

On 2/16/21 3:59 PM, Isaac Morland wrote: > On Fri, 12 Feb 2021 at 12:58, Tom Lane > wrote: > > Isaac Morland > writes: > > I was trying to use triggers, and ran into something I hadn't > realized > > until now: trigg

Re: SSL SNI

On Mon, 2021-02-15 at 15:09 +0100, Peter Eisentraut wrote: > The question I had was whether this should be an optional behavior, or > conversely a behavior that can be turned off, or whether it should just > be turned on all the time. Personally I think there should be a toggle, so that any user

Re: PATCH: Batch/pipelining support for libpq

Here's a new version, where I've renamed everything to "pipeline". I think the docs could use some additional tweaks now in order to make a coherent story on pipeline mode, how it can be used in a batched fashion, etc. Here's the renames I applied. It's mostly mechanical, except PQbatchSendQueue

Re: PATCH: Batch/pipelining support for libpq

Hi, + if (querymode == QUERY_SIMPLE) + { + commandFailed(st, "startpipeline", "cannot use pipeline mode with the simple query protocol"); + st->state = CSTATE_ABORTED; + return CSTATE_ABORTED; I wonder why the st->state is only assigned for this if block.

Re: TRIM_ARRAY

On 2/16/21 11:38 PM, Vik Fearing wrote: > On 2/16/21 7:32 PM, Isaac Morland wrote: >> On Tue, 16 Feb 2021 at 12:54, Vik Fearing wrote: >> >>> The SQL standard defines a function called TRIM_ARRAY that surprisingly >>> has syntax that looks like a function! So I implemented it using a thin >>> wra

Re: Support for NSS as a libpq TLS backend

On Mon, 2020-07-20 at 15:35 +0200, Daniel Gustafsson wrote: > This version adds support for sslinfo on NSS for most the functions. I've poked around to see what can be done about the unimplemented ssl_client_dn_field/ssl_issuer_field functions. There's a nasty soup of specs to wade around in, and

Re: How to get Relation tuples in C function

On Sun, Feb 14, 2021 at 7:56 PM Michael Paquier wrote: > On Sun, Feb 14, 2021 at 09:29:08AM +0800, Andy Fan wrote: > > Thank you tom for the reply. What would be the difference between the > > SPI and "write a pure SQL UDF" and call it with DirectFunctionCall1? I > > just ran into a similar situ

Re: Parallel INSERT (INTO ... SELECT ...)

On Wed, Feb 17, 2021 at 12:19 AM Amit Langote wrote: > > On Mon, Feb 15, 2021 at 4:39 PM Greg Nancarrow wrote: > > On Sat, Feb 13, 2021 at 12:17 AM Amit Langote > > wrote: > > > On Thu, Feb 11, 2021 at 4:43 PM Greg Nancarrow > > > wrote: > > > > Actually, I tried adding the following in the l

Re: pg_collation_actual_version() ERROR: cache lookup failed for collation 123

On Mon, Jan 18, 2021 at 11:22 AM Thomas Munro wrote: > On Mon, Jan 18, 2021 at 10:59 AM Justin Pryzby > |postgres=# SELECT > pg_collation_actual_version(123); > > |ERROR: cache lookup failed for collation 123 > > Yeah, not a great user experience. Will fix next week; perhaps > get_collation_vers

Re: [Patch] ALTER SYSTEM READ ONLY

Hi, On 2021-02-16 17:11:06 -0500, Robert Haas wrote: > I can't promise that what I'm about to write is an entirely faithful > representation of what he said, but hopefully it's not so far off that > he gets mad at me or something. Seems accurate - and also I'm way too tired that I'd be mad ;) >

Re: [DOC] add missing "[ NO ]" to various "DEPENDS ON" synopses

On Tue, Feb 16, 2021 at 11:18:47AM +0900, Ian Lawrence Barwick wrote: > Hmm, with the current implementation "alter index my_index no " > doesn't work > anyway; you'd need to add this before the above lines: > > + else if (Matches("ALTER", "INDEX", MatchAny, "NO")) > + COMPLETE

Re: Is it worth accepting multiple CRLs?

The commit fe61df7f82 shot down this. This patch allows a new GUC ssl_crl_dir and a new libpq connection option sslcrldir to specify CRL directory, which stores multiple files that contains one CRL. With that method server loads only CRLs for the CA of the certificate being validated. Along with

Re: ERROR: invalid spinlock number: 0

On Tue, Feb 16, 2021 at 11:47:52PM +0900, Fujii Masao wrote: > On 2021/02/16 15:50, Michael Paquier wrote: >> + /* >> +* Read "writtenUpto" without holding a spinlock. So it may not be >> +* consistent with other WAL receiver's shared variables protected by a >> +* spinlock. This is O

Re: [HACKERS][PATCH] Applying PMDK to WAL operations for persistent memory

Rebased to make patchset v5. I also found that my past replies have separated the thread in the pgsql-hackers archive. I try to connect this mail to the original thread [1], and let this point to the separated portions [2][3][4]. Note that the patchset v3 is in [3] and v4 is in [4]. Regards, [1]

Re: [POC] verifying UTF-8 using SIMD instructions

I wrote: > [v3] > - It's not smart enough to stop at the last valid character boundary -- it's either all-valid or it must start over with the fallback. That will have to change in order to work with the proposed noError conversions. It shouldn't be very hard, but needs thought as to the clearest

Re: progress reporting for partitioned REINDEX

On Tue, Feb 16, 2021 at 12:39:08PM +0100, Matthias van de Meent wrote: > These were added to report the index and table that are currently > being worked on in concurrent reindexes of tables, schemas and > databases. Before that commit, it would only report up to the last > index being prepared in

Re: progress reporting for partitioned REINDEX

On Wed, Feb 17, 2021 at 02:55:04PM +0900, Michael Paquier wrote: > I see no bug here. pg_stat_progress_create_index includes partitions_{done,total} for CREATE INDEX p, so isn't it strange if it wouldn't do likewise for REINDEX INDEX p ? -- Justin

Re: progress reporting for partitioned REINDEX

On Wed, Feb 17, 2021 at 12:10:43AM -0600, Justin Pryzby wrote: > On Wed, Feb 17, 2021 at 02:55:04PM +0900, Michael Paquier wrote: >> I see no bug here. > > pg_stat_progress_create_index includes partitions_{done,total} for > CREATE INDEX p, so isn't it strange if it wouldn't do likewise for > REIN

Re: pg_collation_actual_version() ERROR: cache lookup failed for collation 123

On Wed, Feb 17, 2021 at 03:08:36PM +1300, Thomas Munro wrote: > tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid)); > if (!HeapTupleIsValid(tp)) > + { > + if (found) > + { > + *found = fals

Re: New Table Access Methods for Multi and Single Inserts

Hi, I addressed the following review comments and attaching v3 patch set. 1) ExecClearTuple happens before ExecCopySlot in heap_multi_insert_v2 and this allowed us to remove clear_mi_slots flag from TableInsertState. 2) I retained the flushed variable inside TableInsertState so that the callers c