Re: do only critical work during single-user vacuum?

2021-12-20 Thread Masahiko Sawada
On Tue, Dec 21, 2021 at 1:53 PM Peter Geoghegan wrote: > > On Mon, Dec 20, 2021 at 8:40 PM Masahiko Sawada wrote: > > BTW a vacuum automatically enters failsafe mode under the situation > > where the user has to run a vacuum in the single-user mode, right? > > Only for the table that had the

Re: Add id's to various elements in protocol.sgml

2021-12-20 Thread Brar Piening
On 20.12.2021 at 16:09, Robert Haas wrote: As a data point, this is something I have also wanted to do, from time to time. I am generally of the opinion that any place the documentation has a long list of things, which should add ids, so that people can link to the particular thing in the list

Re: row filtering for logical replication

2021-12-20 Thread Amit Kapila
On Tue, Dec 21, 2021 at 10:53 AM vignesh C wrote: > > On Mon, Dec 20, 2021 at 8:41 AM houzj.f...@fujitsu.com > wrote: > > > > On Fri, Dec 17, 2021 6:09 PM Amit Kapila wrote: > > > On Fri, Dec 17, 2021 at 4:11 AM Peter Smith wrote: > > > > > > > > PSA the v47* patch set. > > Thanks for the

Re: row filtering for logical replication

2021-12-20 Thread Amit Kapila
On Tue, Dec 21, 2021 at 6:17 AM Ajin Cherian wrote: > > On Tue, Dec 21, 2021 at 5:58 AM Euler Taveira wrote: > > > >In pgoutput_row_filter_update(), first, we are deforming the tuple in > >local datum, then modifying the tuple, and then reforming the tuple. > >I think we can surely do better

RE: row filtering for logical replication

2021-12-20 Thread tanghy.f...@fujitsu.com
On Tuesday, December 21, 2021 3:03 PM, tanghy.f...@fujitsu.com wrote: > To: Amit Kapila ; Euler Taveira > Cc: Dilip Kumar ; Peter Smith ; > Greg Nancarrow ; Hou, Zhijie/侯 志杰 > ; vignesh C ; Ajin Cherian > ; Rahila Syed ; Peter Eisentraut > ; Önder Kalacı ; > japin ; Michael Paquier ; David >

RE: row filtering for logical replication

2021-12-20 Thread tanghy.f...@fujitsu.com
On Monday, December 20, 2021 4:47 PM tanghy.f...@fujitsu.com wrote: > On Monday, December 20, 2021 11:24 AM tanghy.f...@fujitsu.com > > > > > On Wednesday, December 8, 2021 2:29 PM Amit Kapila > > wrote: > > > > > > On Mon, Dec 6, 2021 at 6:04 PM Euler Taveira wrote: > > > > > > > > On Mon,

Re: [Proposal] Global temporary tables

2021-12-20 Thread Andrew Bille
Hi! Thanks for new patches. Yet another crash reproduced on master with v63 patches: CREATE TABLESPACE ts LOCATION '/tmp/ts'; CREATE GLOBAL TEMP TABLE tbl (num1 bigint); INSERT INTO tbl (num1) values (1); CREATE INDEX tbl_idx ON tbl (num1); REINDEX (TABLESPACE ts) TABLE tbl; Got error: CREATE

Re: parallel vacuum comments

2021-12-20 Thread Masahiko Sawada
On Tue, Dec 21, 2021 at 2:04 PM Amit Kapila wrote: > > On Tue, Dec 21, 2021 at 10:05 AM Masahiko Sawada > wrote: > > > > On Tue, Dec 21, 2021 at 12:05 PM Amit Kapila > > wrote: > > > > > > On Mon, Dec 20, 2021 at 6:29 PM Masahiko Sawada > > > wrote: > > > > > BTW, if we go with that then

Re: Multi-Column List Partitioning

2021-12-20 Thread Ashutosh Sharma
On Mon, Dec 20, 2021 at 7:04 PM Amit Langote wrote: > Hi, > > On Mon, Dec 13, 2021 at 11:37 PM Ashutosh Sharma > wrote: > > > > Hi, > > > > Is this okay? > > > > postgres=# CREATE TABLE t1 (a int, b int) PARTITION BY LIST ( a, a, a ); > > CREATE TABLE > > > > postgres=# CREATE TABLE t1_1

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-12-20 Thread Ashutosh Sharma
I am getting the below error when running the same test-case that Neha shared in her previous email. ERROR: 55000: some relations of database "test1" are already in tablespace "tab1" HINT: You must move them back to the database's default tablespace before using this command. LOCATION: movedb,

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2021-12-20 Thread Peter Geoghegan
On Mon, Dec 20, 2021 at 8:29 PM Masahiko Sawada wrote: > > Can we fully get rid of vacuum_freeze_table_age? > > Does it mean that a vacuum always is an aggressive vacuum? No. Just somewhat more like one. Still no waiting for cleanup locks, though. Also, autovacuum is still cancelable (that's

Re: row filtering for logical replication

2021-12-20 Thread vignesh C
On Mon, Dec 20, 2021 at 8:41 AM houzj.f...@fujitsu.com wrote: > > On Fri, Dec 17, 2021 6:09 PM Amit Kapila wrote: > > On Fri, Dec 17, 2021 at 4:11 AM Peter Smith wrote: > > > > > > PSA the v47* patch set. > Thanks for the comments, I agree with all the comments. > Attach the V49 patch set,

Re: BufferAlloc: don't take two simultaneous locks

2021-12-20 Thread Yura Sokolov
В Сб, 02/10/2021 в 01:25 +0300, Yura Sokolov пишет: > Good day. > > I found some opportunity in Buffer Manager code in BufferAlloc > function: > - When valid buffer is evicted, BufferAlloc acquires two partition > lwlocks: for partition for evicted block is in and partition for new > block

Re: parallel vacuum comments

2021-12-20 Thread Amit Kapila
On Tue, Dec 21, 2021 at 10:05 AM Masahiko Sawada wrote: > > On Tue, Dec 21, 2021 at 12:05 PM Amit Kapila wrote: > > > > On Mon, Dec 20, 2021 at 6:29 PM Masahiko Sawada > > wrote: > > > BTW, if we go with that then we should set the correct phase > > for workers as well? > > If we have

Re: Getting rid of regression test input/ and output/ files

2021-12-20 Thread Greg Stark
On Sun, 19 Dec 2021 at 18:41, Corey Huinker wrote: > > Which brings up a tangential question, is there value in having something > that brings in one or more env vars as psql vars directly. I'm thinking > something like: > > \importenv pattern [prefix] Oof. That gives me the security heebie

Re: do only critical work during single-user vacuum?

2021-12-20 Thread Peter Geoghegan
On Mon, Dec 20, 2021 at 8:40 PM Masahiko Sawada wrote: > BTW a vacuum automatically enters failsafe mode under the situation > where the user has to run a vacuum in the single-user mode, right? Only for the table that had the problem. Maybe there are no other tables that a database level

Re: do only critical work during single-user vacuum?

2021-12-20 Thread Masahiko Sawada
On Tue, Dec 21, 2021 at 12:46 PM Andres Freund wrote: > > Hi, > > On 2021-12-20 17:17:26 -0800, Peter Geoghegan wrote: > > On Thu, Dec 9, 2021 at 8:41 PM Bossart, Nathan wrote: > > > I like the idea of having a built-in function that does the bare > > > minimum to resolve wraparound emergencies,

Re: parallel vacuum comments

2021-12-20 Thread Masahiko Sawada
On Tue, Dec 21, 2021 at 12:05 PM Amit Kapila wrote: > > On Mon, Dec 20, 2021 at 6:29 PM Masahiko Sawada wrote: > > > > On Mon, Dec 20, 2021 at 1:08 PM Amit Kapila wrote: > > > > > > > > > > > > > 2. What is the reason for not moving > > > > > lazy_vacuum_one_index/lazy_cleanup_one_index to

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2021-12-20 Thread Masahiko Sawada
On Sat, Dec 18, 2021 at 11:29 AM Peter Geoghegan wrote: > > On Thu, Dec 16, 2021 at 10:46 PM Masahiko Sawada > wrote: > > > My emphasis here has been on making non-aggressive VACUUMs *always* > > > advance relfrozenxid, outside of certain obvious edge cases. And so > > > with all the patches

Re: row filtering for logical replication

2021-12-20 Thread Amit Kapila
On Tue, Dec 21, 2021 at 12:28 AM Euler Taveira wrote: > > On Mon, Dec 20, 2021, at 12:10 AM, houzj.f...@fujitsu.com wrote: > > Attach the V49 patch set, which addressed all the above comments on the 0002 > patch. > > I've been testing the latest versions of this patch set. I'm attaching a new >

FW: Question about HEAP_XMIN_COMMITTED

2021-12-20 Thread haiming
Hello, hackers. I have the same question. Hope to reply, thanks. -Original Message- From: liuhuail...@fujitsu.com Sent: Tuesday, December 14, 2021 4:55 PM To: pgsql-hack...@postgresql.org Subject: Question about HEAP_XMIN_COMMITTED Hi I did the following steps on PG. 1. Building a

Re: do only critical work during single-user vacuum?

2021-12-20 Thread Andres Freund
Hi, On 2021-12-20 17:17:26 -0800, Peter Geoghegan wrote: > On Thu, Dec 9, 2021 at 8:41 PM Bossart, Nathan wrote: > > I like the idea of having a built-in function that does the bare > > minimum to resolve wraparound emergencies, and I think providing some > > sort of simple progress indicator

Re: pg_upgrade should truncate/remove its logs before running

2021-12-20 Thread Justin Pryzby
On Mon, Dec 20, 2021 at 08:21:51PM +0900, Michael Paquier wrote: > On Fri, Dec 17, 2021 at 11:21:13AM -0600, Justin Pryzby wrote: > + log_opts.basedir = "pg_upgrade_log.d"; > we could choose something simpler for the default, like > "pg_upgrade_log". I don't have a good history in

Re: parallel vacuum comments

2021-12-20 Thread Amit Kapila
On Mon, Dec 20, 2021 at 6:29 PM Masahiko Sawada wrote: > > On Mon, Dec 20, 2021 at 1:08 PM Amit Kapila wrote: > > > > > > > > > > 2. What is the reason for not moving > > > > lazy_vacuum_one_index/lazy_cleanup_one_index to vacuum.c so that they > > > > can be called from vacuumlazy.c and

Re: sequences vs. synchronous replication

2021-12-20 Thread Tomas Vondra
On 12/21/21 02:01, Tom Lane wrote: Tomas Vondra writes: OK, I did a quick test with two very simple benchmarks - simple select from a sequence, and 'pgbench -N' on scale 1. Benchmark was on current master, patched means SEQ_LOG_VALS was set to 1. But ... pgbench -N doesn't use sequences at

Re: simplifying foreign key/RI checks

2021-12-20 Thread Corey Huinker
> > > > Good catch, thanks. Patch updated. > > > Applies clean. Passes check-world.

Re: PublicationActions - use bit flags.

2021-12-20 Thread Greg Nancarrow
On Tue, Dec 21, 2021 at 11:56 AM Tom Lane wrote: > > Removing this is not good: > > if (relation->rd_pubactions) > - { > pfree(relation->rd_pubactions); > - relation->rd_pubactions = NULL; > - } > > If the subsequent palloc fails, you've created a

Re: Emit a warning if the extension's GUC is set incorrectly

2021-12-20 Thread Kyotaro Horiguchi
At Fri, 17 Dec 2021 11:25:22 +0900, Shinya Kato wrote in > On 2021-12-17 01:55, Fujii Masao wrote: > > I'm still not sure why you were thinking that ERROR is more proper > > here. > > Since I get an ERROR when I set the wrong normal GUCs, I thought I > should also get an ERROR when I set the

RE: Confused comment about drop replica identity index

2021-12-20 Thread wangw.f...@fujitsu.com
On Tue, Dec 20, 2021 at 19:11PM, Michael Paquier wrote: > That's mostly fine. I have made some adjustments as per the attached. Thanks for reviewing. > + The default for non-system tables. Records the old values of the > columns > + of the primary key, if any. The default for

Re: do only critical work during single-user vacuum?

2021-12-20 Thread Peter Geoghegan
On Thu, Dec 9, 2021 at 8:41 PM Bossart, Nathan wrote: > I like the idea of having a built-in function that does the bare > minimum to resolve wraparound emergencies, and I think providing some > sort of simple progress indicator (even if rudimentary) would be very > useful. If John doesn't have

Re: Emit a warning if the extension's GUC is set incorrectly

2021-12-20 Thread Kyotaro Horiguchi
At Mon, 20 Dec 2021 21:05:23 +0900, Shinya Kato wrote in > - v6-01-Add-EmitWarningsOnPlaceholders.patch > We should use EmitWarningsOnPlaceholders when we use > DefineCustomXXXVariable. > I don't think there is any room for debate. Unfortunately, pltcl.c defines variables both in pltcl and

Re: sequences vs. synchronous replication

2021-12-20 Thread Tom Lane
Tomas Vondra writes: > OK, I did a quick test with two very simple benchmarks - simple select > from a sequence, and 'pgbench -N' on scale 1. Benchmark was on current > master, patched means SEQ_LOG_VALS was set to 1. But ... pgbench -N doesn't use sequences at all, does it? Probably inserts

Re: PublicationActions - use bit flags.

2021-12-20 Thread Tom Lane
Greg Nancarrow writes: > I've attached a patch which addresses that and replaces a couple of > memcpy()s with struct assignment, as suggested. Removing this is not good: if (relation->rd_pubactions) - { pfree(relation->rd_pubactions); -

Re: sequences vs. synchronous replication

2021-12-20 Thread Tomas Vondra
On 12/20/21 17:40, Tomas Vondra wrote: On 12/20/21 15:31, Peter Eisentraut wrote: On 18.12.21 22:48, Tomas Vondra wrote: What do you mean by "not caching unused sequence numbers"? Reducing SEQ_LOG_VALS to 1, i.e. WAL-logging every sequence increment? That'd work, but I wonder how significant

Re: row filtering for logical replication

2021-12-20 Thread Ajin Cherian
On Tue, Dec 21, 2021 at 5:58 AM Euler Taveira wrote: > > I reviewed 0003. It uses TupleTableSlot instead of HeapTuple. I probably > missed > the explanation but it requires more changes (logicalrep_write_tuple and 3 new > entries into RelationSyncEntry). I replaced this patch with a slightly >

Re: Confused comment about drop replica identity index

2021-12-20 Thread Michael Paquier
On Mon, Dec 20, 2021 at 11:57:32AM -0300, Euler Taveira wrote: > What do you think about the attached patch? It forbids the DROP INDEX. We > might > add a detail message but I didn't in this patch. Yeah. I'd agree about doing something like that on HEAD, and that would help with some of the

Re: Refactoring of compression options in pg_basebackup

2021-12-20 Thread Michael Paquier
On Mon, Dec 20, 2021 at 10:19:44AM -0500, Robert Haas wrote: > One thing we should keep in mind is that I'm also working on adding > server-side compression, initially with gzip, but Jeevan Ladhe has > posted patches to extend that to LZ4. So however we structure the > options they should take

Re: PublicationActions - use bit flags.

2021-12-20 Thread Greg Nancarrow
On Tue, Dec 21, 2021 at 4:14 AM Alvaro Herrera wrote: > > On 2021-Dec-20, Peter Eisentraut wrote: > > > I don't see why this is better. It just makes the code longer and adds more > > punctuation and reduces type safety. > > Removing one palloc is I think the most important consequence ... >

Re: sqlsmith: ERROR: XX000: bogus varno: 2

2021-12-20 Thread Tom Lane
Here's a less hasty version of the patch. regards, tom lane diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c index e276264882..5d4700430c 100644 --- a/src/backend/nodes/nodeFuncs.c +++ b/src/backend/nodes/nodeFuncs.c @@ -2201,6 +2201,26 @@

Re: Schema variables - new implementation for Postgres 15

2021-12-20 Thread Justin Pryzby
I don't understand what 0002 patch does relative to the 0001 patch. Is 0002 to change the error messages from "schema variables" to "session variables" , in a separate commit to show that the main patch doesn't change regression results ? Could you add commit messages ? I mentioned before that

Re: relcache not invalidated when ADD PRIMARY KEY USING INDEX

2021-12-20 Thread Euler Taveira
On Mon, Dec 20, 2021, at 3:45 AM, houzj.f...@fujitsu.com wrote: > Hi, > > When reviewing some replica identity related patches, I found that when adding > primary key using an existing unique index on not null columns, the > target table's relcache won't be invalidated. Good catch. It seems you

Re: row filtering for logical replication

2021-12-20 Thread Peter Smith
On Tue, Dec 21, 2021 at 5:58 AM Euler Taveira wrote: > > On Mon, Dec 20, 2021, at 12:10 AM, houzj.f...@fujitsu.com wrote: > > Attach the V49 patch set, which addressed all the above comments on the 0002 > patch. > > I've been testing the latest versions of this patch set. I'm attaching a new >

Re: sqlsmith: ERROR: XX000: bogus varno: 2

2021-12-20 Thread Tom Lane
Robert Haas writes: > OK ... but my point is that dump and restore does work. So whatever > cases pg_get_expr() doesn't work must be cases that aren't needed for > that to happen. Otherwise this problem would have been found long ago. pg_get_expr doesn't (or didn't) depend on

Re: Allow DELETE to use ORDER BY and LIMIT/OFFSET

2021-12-20 Thread Corey Huinker
> > Out of curiosity, could you please tell me the concrete situations > where you wanted to delete one of two identical records? > In my case, there is a table with known duplicates, and we would like to delete all but the one with the lowest ctid, and then add a unique index to the table which

Re: sqlsmith: ERROR: XX000: bogus varno: 2

2021-12-20 Thread Robert Haas
On Mon, Dec 20, 2021 at 2:36 PM Tom Lane wrote: > I'm not sure why you're astonished by that, considering that > psql has applied pg_get_expr to relpartbound since f0e44751d, > which was the same commit that put code into ruleutils.c to > make pg_get_expr work on relpartbounds. > > It seems a bit

Re: sqlsmith: ERROR: XX000: bogus varno: 2

2021-12-20 Thread Tom Lane
Robert Haas writes: > Right. I'm not surprised that relpartbound uses those node types. I > *am* surprised that pg_get_expr() is expected to be able to handle > them. IOW, they ARE node trees, consonant with the fact that the > column type is pg_node_tree, but they're NOT expressions. I'm not

Re: Add index scan progress to pg_stat_progress_vacuum

2021-12-20 Thread Justin Pryzby
This view also doesn't show vacuum progress across a partitioned table. For comparison: pg_stat_progress_create_index (added in v12) has: partitions_total partitions_done pg_stat_progress_analyze (added in v13) has: child_tables_total child_tables_done pg_stat_progress_cluster should have

Re: Adding CI to our tree

2021-12-20 Thread Andres Freund
Hi, Attached is v4 of the CI patch. Changes: - Move docker image specification out of the patch and generate them together with the VM images. The main reason for this is that I got worried about all repositories having to recreate the images - they're large. - Moved the core dump handling

Re: Add index scan progress to pg_stat_progress_vacuum

2021-12-20 Thread Peter Geoghegan
On Wed, Dec 1, 2021 at 2:59 PM Imseih (AWS), Sami wrote: > The current implementation of pg_stat_progress_vacuum does not provide > progress on which index is being vacuumed making it difficult for a user to > determine if the "vacuuming indexes" phase is making progress. I notice that your

Re: sqlsmith: ERROR: XX000: bogus varno: 2

2021-12-20 Thread Robert Haas
On Mon, Dec 20, 2021 at 1:13 PM Tom Lane wrote: > The reason the regression tests fail if I only patch ruleutils is > that psql \d on a partitioned table invokes > ... pg_get_expr(c.relpartbound, c.oid) FROM pg_catalog.pg_class c > and evidently relpartbound does contain precisely these

Re: Add index scan progress to pg_stat_progress_vacuum

2021-12-20 Thread Peter Geoghegan
On Wed, Dec 15, 2021 at 2:10 PM Bossart, Nathan wrote: > nitpick: Shouldn't index_blks_scanned be index_blks_vacuumed? IMO it > is more analogous to heap_blks_vacuumed. +1. > This will tell us which indexes are currently being vacuumed and the > current progress of those operations, but it

Re: sqlsmith: ERROR: XX000: bogus varno: 2

2021-12-20 Thread Tom Lane
Robert Haas writes: > The commit that added PartitionBoundSpec and PartitionRangeDatum was > committed by me and authored by Amit Langote. It is the original table > partitioning commit -- f0e44751d7175fa3394da2c8f85e3ceb3cdbfe63. I'm > reasonably sure that the reason why those didn't get added

Re: [PATCH] Improve amcheck to also check UNIQUE constraint in btree index.

2021-12-20 Thread Mark Dilger
> On Dec 20, 2021, at 7:37 AM, Pavel Borisov wrote: > > The patch is pgindented and rebased on the current PG master code. Thank you, Pavel. The tests in check_btree.sql no longer create a bttest_unique table, so the DROP TABLE is surplusage: +DROP TABLE bttest_unique; +ERROR: table

Re: sqlsmith: ERROR: XX000: bogus varno: 2

2021-12-20 Thread Robert Haas
On Mon, Dec 20, 2021 at 11:25 AM Tom Lane wrote: > I figured this would be just a quick hack in ruleutils.c, but was > dismayed to find the regression tests falling over, because some > bozo neglected to teach nodeFuncs.c about partition expressions. > It might be a good idea to back-patch that

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2021-12-20 Thread Peter Geoghegan
On Mon, Nov 29, 2021 at 6:51 PM Peter Geoghegan wrote: > Attached is a WIP patch doing this. This has bitrot, so I attach v2, mostly just to keep the CFTester status green. The only real change is one minor simplification to how we set everything up, inside heap_vacuum_rel(). -- Peter

Re: Support for NSS as a libpq TLS backend

2021-12-20 Thread Joshua Brindle
On Wed, Dec 15, 2021 at 5:05 PM Daniel Gustafsson wrote: > > > On 25 Nov 2021, at 14:39, Joshua Brindle > > wrote: > > On Wed, Nov 24, 2021 at 8:49 AM Joshua Brindle > > wrote: > >> > >> On Wed, Nov 24, 2021 at 8:46 AM Joshua Brindle > >> wrote: > > >> I don't know enough about NSS to know if

Re: PublicationActions - use bit flags.

2021-12-20 Thread Alvaro Herrera
On 2021-Dec-20, Peter Eisentraut wrote: > I don't see why this is better. It just makes the code longer and adds more > punctuation and reduces type safety. Removing one palloc is I think the most important consequence ... probably not a big deal though. I think we could change the memcpy

Re: PublicationActions - use bit flags.

2021-12-20 Thread Tom Lane
Peter Eisentraut writes: > On 20.12.21 01:18, Peter Smith wrote: >> I felt it is more natural to implement boolean flag combinations using >> a bitmask instead of a struct of bools. IMO using the bitmask also >> simplifies assignment and checking of said flags. > I don't see why this is better.

Re: Getting rid of regression test input/ and output/ files

2021-12-20 Thread Andrew Dunstan
On 12/18/21 18:53, Tom Lane wrote: > > 2. Export the values from pg_regress as environment variables, > and then add a way for the test scripts to read those variables. > I was a bit surprised to realize that we didn't have any way > to do that already --- psql has \setenv, so why did we never >

Re: sequences vs. synchronous replication

2021-12-20 Thread Tomas Vondra
On 12/20/21 15:31, Peter Eisentraut wrote: On 18.12.21 22:48, Tomas Vondra wrote: What do you mean by "not caching unused sequence numbers"? Reducing SEQ_LOG_VALS to 1, i.e. WAL-logging every sequence increment? That'd work, but I wonder how significant the impact will be. It'd bet it hurts

Re: sqlsmith: ERROR: XX000: bogus varno: 2

2021-12-20 Thread Tom Lane
Robert Haas writes: > On Sun, Dec 19, 2021 at 4:17 PM Tom Lane wrote: >> I don't see anything particularly surprising here. pg_get_expr is only >> able to cope with expression trees over a single relation, but ON UPDATE >> rules can refer to both OLD and NEW relations. Maybe we could make the

Re: [PATCH] Improve amcheck to also check UNIQUE constraint in btree index.

2021-12-20 Thread Pavel Borisov
> > >> I completely agree that checking uniqueness requires looking at the > heap, but I don't agree that every caller of bt_index_check on an index > wants that particular check to be performed. There are multiple ways in > which an index might be corrupt, and Peter wrote the code to only check

Re: psql format output

2021-12-20 Thread Alvaro Herrera
On 2021-Dec-20, Tom Lane wrote: > -[ RECORD 1 > ]---+--- > Schema | pg_catalog > Name| pg_copy_logical_replication_slot > Result data type| record >

Re: psql format output

2021-12-20 Thread Tom Lane
Robert Haas writes: > It's hard to make any general judgment about how wide people's > terminal windows are likely to be, but it is my opinion that the root > of the problem is that \df+ just wants to display a whole lot of stuff > - and as hackers add more function properties in the future,

Re: Refactoring of compression options in pg_basebackup

2021-12-20 Thread Robert Haas
On Sat, Dec 18, 2021 at 6:29 AM Michael Paquier wrote: > This is one step toward the introduction of LZ4 in pg_basebackup, but > this refactoring is worth doing on its own, hence a separate thread to > deal with this problem first. The options of pg_basebackup are > reworked to be consistent

Re: Add id's to various elements in protocol.sgml

2021-12-20 Thread Robert Haas
On Fri, Dec 17, 2021 at 6:54 PM Brar Piening wrote: > The purpose is that you can directly link to the id in the public html > docs which still gets generated (e. g. > https://www.postgresql.org/docs/14/protocol-replication.html#PROTOCOL-REPLICATION-BASE-BACKUP). > > Essentially it gives people

Re: Getting rid of regression test input/ and output/ files

2021-12-20 Thread Tom Lane
Peter Eisentraut writes: > On 19.12.21 00:53, Tom Lane wrote: >> I was a bit surprised to realize that we didn't have any way >> to do that already --- psql has \setenv, so why did we never >> invent \getenv? > You can do > \set foo `echo $ENVVAR` > but that's probably not portable enough for

Re: Is my home $HOME or is it getpwent()->pw_dir ?

2021-12-20 Thread Chapman Flack
On 12/20/21 09:15, Peter Eisentraut wrote: > On 18.12.21 21:57, Chapman Flack wrote: >> When I start psql, strace shows $HOME being honored when looking >> for .terminfo and .inputrc, and getpwent()->pw_dir being used >> to look for .pgpass, .psqlrc, and .psql_history, which of course >> aren't

Re: Confused comment about drop replica identity index

2021-12-20 Thread Euler Taveira
On Mon, Dec 20, 2021, at 8:11 AM, Michael Paquier wrote: > On Mon, Dec 20, 2021 at 03:46:13AM +, wangw.f...@fujitsu.com wrote: > > Here is a patch to correct wrong comment about > > REPLICA_IDENTITY_INDEX, And improve the pg-doc. > > That's mostly fine. I have made some adjustments as per

Re: sqlsmith: ERROR: XX000: bogus varno: 2

2021-12-20 Thread Robert Haas
On Sun, Dec 19, 2021 at 4:17 PM Tom Lane wrote: > Justin Pryzby writes: > > I reduced the problematic query to this. > > SELECT 1 FROM pg_rewrite WHERE > > pg_get_function_arg_default(ev_class, 1) !~~ > > pg_get_expr(ev_qual, ev_class, false); > > Or more simply, > > regression=# select

Re: psql format output

2021-12-20 Thread Robert Haas
On Fri, Dec 17, 2021 at 5:08 AM Peter Eisentraut wrote: > That's a reasonable idea. I wonder if it would work in general. If > someone had a C function (so no source code) with three arguments, they > might be annoyed if it now displayed as three lines by default. The problem I see is that

Re: sequences vs. synchronous replication

2021-12-20 Thread Peter Eisentraut
On 18.12.21 22:48, Tomas Vondra wrote: What do you mean by "not caching unused sequence numbers"? Reducing SEQ_LOG_VALS to 1, i.e. WAL-logging every sequence increment? That'd work, but I wonder how significant the impact will be. It'd bet it hurts the patch adding logical decoding of

Re: speed up verifying UTF-8

2021-12-20 Thread John Naylor
On Fri, Dec 17, 2021 at 9:29 AM John Naylor wrote: > > I plan to push v25 early next week, unless there are further comments. Pushed, thanks everyone! -- John Naylor EDB: http://www.enterprisedb.com

Re: Is my home $HOME or is it getpwent()->pw_dir ?

2021-12-20 Thread Peter Eisentraut
On 18.12.21 21:57, Chapman Flack wrote: I sometimes do some testing as nobody, on a distro where getpwent(nobody)->pw_dir is a directory that nobody can't write. So I end up setting $HOME to a directory that, um, is writable. When I start psql, strace shows $HOME being honored when looking for

Re: Getting rid of regression test input/ and output/ files

2021-12-20 Thread Peter Eisentraut
On 19.12.21 00:53, Tom Lane wrote: 2. Export the values from pg_regress as environment variables, and then add a way for the test scripts to read those variables. I was a bit surprised to realize that we didn't have any way to do that already --- psql has \setenv, so why did we never invent

Re: PublicationActions - use bit flags.

2021-12-20 Thread Peter Eisentraut
On 20.12.21 01:18, Peter Smith wrote: For some reason the current HEAD PublicationActions is a struct of boolean representing combinations of the 4 different "publication actions". I felt it is more natural to implement boolean flag combinations using a bitmask instead of a struct of bools. IMO

Re: Column Filtering in Logical Replication

2021-12-20 Thread Peter Eisentraut
On 17.12.21 22:07, Alvaro Herrera wrote: So I've been thinking about this as a "security" item (you can see my comments to that effect sprinkled all over this thread), in the sense that if a publication "hides" some column, then the replica just won't get access to it. But in reality that's

Re: Addition of --no-sync to pg_upgrade for test speedup

2021-12-20 Thread Alvaro Herrera
On 2021-Dec-16, Michael Paquier wrote: > In pg_upgrade, we let the flush happen with initdb --sync-only, based > on the binary path of the new cluster, so I think that we are not > going to miss any test coverage by skipping that. There was one patch of mine with breakage that only manifested in

RE: In-placre persistance change of a relation

2021-12-20 Thread Jakub Wartak
Hi Kyotaro, > At Mon, 20 Dec 2021 17:39:27 +0900 (JST), Kyotaro Horiguchi > wrote in > > At Mon, 20 Dec 2021 07:59:29 +, Jakub Wartak > > wrote in > > > BTW fast feedback regarding that ALTER patch (there were 4 unlogged > tables): > > > # ALTER TABLE ALL IN TABLESPACE tbs1 set logged; >

Re: Multi-Column List Partitioning

2021-12-20 Thread Amit Langote
Hi, On Mon, Dec 13, 2021 at 11:37 PM Ashutosh Sharma wrote: > > Hi, > > Is this okay? > > postgres=# CREATE TABLE t1 (a int, b int) PARTITION BY LIST ( a, a, a ); > CREATE TABLE > > postgres=# CREATE TABLE t1_1 PARTITION OF t1 FOR VALUES IN ((1, 2, 3), (4, 5, > 6)); > CREATE TABLE > >

[PATCH] pg_stat_toast v0.3

2021-12-20 Thread Gunnar "Nick" Bluth
Am 12.12.21 um 17:20 schrieb Gunnar "Nick" Bluth: Hello -hackers! Please have a look at the attached patch, which implements some statistics for TOAST. The attached v0.3 includes * a proper GUC "track_toast" incl. postgresql.conf.sample line * gathering timing information * the system view

Re: simplifying foreign key/RI checks

2021-12-20 Thread Amit Langote
On Mon, Dec 20, 2021 at 6:19 PM Zhihong Yu wrote: > On Sun, Dec 19, 2021 at 10:20 PM Amit Langote wrote: >> >> On Mon, Dec 20, 2021 at 2:00 PM Corey Huinker >> wrote: >> > Sorry for the delay. This patch no longer applies, it has some conflict >> > with

Re: parallel vacuum comments

2021-12-20 Thread Masahiko Sawada
On Mon, Dec 20, 2021 at 1:08 PM Amit Kapila wrote: > > On Mon, Dec 20, 2021 at 8:33 AM Masahiko Sawada wrote: > > > > On Sat, Dec 18, 2021 at 3:38 PM Amit Kapila wrote: > > > > > > Few comments: > > > = > > > 1. > > > + * dead_items stores TIDs whose index tuples are deleted by

Re: a misbehavior of partition row movement (?)

2021-12-20 Thread Amit Langote
On Thu, Oct 14, 2021 at 6:00 PM Amit Langote wrote: > On Mon, Sep 20, 2021 at 3:32 PM Amit Langote wrote: > > The problem was that the tuplestore > > (afterTriggers.query_stack[query_level].tuplestore) that I decided to > > use to store the AFTER trigger tuples of a partitioned table that is > >

Re: Emit a warning if the extension's GUC is set incorrectly

2021-12-20 Thread Shinya Kato
On 2021-12-17 15:42, Peter Eisentraut wrote: On 17.12.21 03:25, Shinya Kato wrote: For now, I'v attached the patch that fixed the compilation error. I think it would be good if you could split the uncontroversial new EmitErrorsOnPlaceholders() calls into a separate patch. And please add an

Use JOIN USING aliases in ruleutils.c

2021-12-20 Thread Peter Eisentraut
When reverse-compiling a query, ruleutils.c has some complicated code to handle the join output columns of a JOIN USING join. There used to be no way to qualify those columns, and so if there was a naming conflict anywhere in the query, those output columns had to be renamed to be unique

Re: pg_upgrade should truncate/remove its logs before running

2021-12-20 Thread Michael Paquier
On Fri, Dec 17, 2021 at 11:21:13AM -0600, Justin Pryzby wrote: > I put this together in the simplest way, prefixing all the filenames with the > configured path.. Well, why not. > Another options is to chdir() into the given path. But, pg_upgrade takes (and > requires) a bunch of other paths,

Re: Confused comment about drop replica identity index

2021-12-20 Thread Michael Paquier
On Mon, Dec 20, 2021 at 03:46:13AM +, wangw.f...@fujitsu.com wrote: > Here is a patch to correct wrong comment about > REPLICA_IDENTITY_INDEX, And improve the pg-doc. That's mostly fine. I have made some adjustments as per the attached. + The default for non-system tables. Records

Re: Allow DELETE to use ORDER BY and LIMIT/OFFSET

2021-12-20 Thread Yugo NAGATA
Hello Greg, On Fri, 17 Dec 2021 01:40:45 -0500 Greg Stark wrote: > On Thu, 16 Dec 2021 at 22:18, Tom Lane wrote: > > > > * If the sort order is underspecified, or you omit ORDER BY > > entirely, then it's not clear which rows will be operated on. > > The LIMIT might stop after just some of the

Re: row filtering for logical replication

2021-12-20 Thread Amit Kapila
On Mon, Dec 20, 2021 at 8:41 AM houzj.f...@fujitsu.com wrote: > > Thanks for the comments, I agree with all the comments. > Attach the V49 patch set, which addressed all the above comments on the 0002 > patch. > Few comments/suugestions: == 1. + Oid publish_as_relid =

Re: [PATCH] proposal for regexp_count, regexp_instr, regexp_substr and regexp_replace

2021-12-20 Thread Peter Eisentraut
On 15.12.21 14:15, Gilles Darold wrote: Le 15/12/2021 à 13:41, Peter Eisentraut a écrit : On 03.08.21 19:10, Tom Lane wrote: Gilles Darold writes: Sorry I have missed that, but I'm fine with this implemenation so let's keep the v6 version of the patch and drop this one. Pushed, then. 

RE: Failed transaction statistics to measure the logical replication progress

2021-12-20 Thread osumi.takami...@fujitsu.com
Friday, December 17, 2021 2:03 PM Kyotaro Horiguchi wrote: > It sends stats packets at every commit-like operation on apply workers. The > current pgstat is so smart that it refrain from sending stats packets at too > high > frequency. We already suffer frequent stats packets so apply workers

Re: simplifying foreign key/RI checks

2021-12-20 Thread Zhihong Yu
On Sun, Dec 19, 2021 at 10:20 PM Amit Langote wrote: > On Mon, Dec 20, 2021 at 2:00 PM Corey Huinker > wrote: > > Sorry for the delay. This patch no longer applies, it has some conflict > with d6f96ed94e73052f99a2e545ed17a8b2fdc1fb8a > > Thanks Corey for the heads up. Rebased with some

Re: In-placre persistance change of a relation

2021-12-20 Thread Kyotaro Horiguchi
At Mon, 20 Dec 2021 17:39:27 +0900 (JST), Kyotaro Horiguchi wrote in > At Mon, 20 Dec 2021 07:59:29 +, Jakub Wartak > wrote in > > BTW fast feedback regarding that ALTER patch (there were 4 unlogged > > tables): > > # ALTER TABLE ALL IN TABLESPACE tbs1 set logged; > > WARNING:

RE: row filtering for logical replication

2021-12-20 Thread tanghy.f...@fujitsu.com
On Monday, December 20, 2021 11:24 AM tanghy.f...@fujitsu.com > > On Wednesday, December 8, 2021 2:29 PM Amit Kapila > wrote: > > > > On Mon, Dec 6, 2021 at 6:04 PM Euler Taveira wrote: > > > > > > On Mon, Dec 6, 2021, at 3:35 AM, Dilip Kumar wrote: > > > > > > On Mon, Dec 6, 2021 at 6:49 AM

Re: [PATCH] pg_stat_toast

2021-12-20 Thread Gunnar "Nick" Bluth
Am 20.12.2021 um 04:20 schrieb kuroda.hay...@fujitsu.com: Dear Gunnar, Hi Kuroda-San! postgres=# CREATE TABLE test (i int, lz4 text COMPRESSION lz4, std text); postgres=# INSERT INTO test SELECT i,repeat(md5(i::text),100),repeat(md5(i::text),100) FROM generate_series(0,10) x(i);

Re: In-placre persistance change of a relation

2021-12-20 Thread Kyotaro Horiguchi
At Mon, 20 Dec 2021 07:59:29 +, Jakub Wartak wrote in > Hi Kyotaro, I'm glad you are still into this > > > I didn't register for some reasons. > > Right now in v8 there's a typo in ./src/backend/catalog/storage.c : > > storage.c: In function 'RelationDropInitFork': > storage.c:385:44: