Re: row filtering for logical replication

2021-12-24 Thread Amit Kapila
On Fri, Dec 24, 2021 at 11:04 AM Peter Smith wrote: > > The current PG docs text for CREATE PUBLICATION (in the v54-0001 > patch) has a part that says > > + A nullable column in the WHERE clause could cause the > + expression to evaluate to false; avoid using columns without not-null > + con

generic plans and "initial" pruning

2021-12-24 Thread Amit Langote
Executing generic plans involving partitions is known to become slower as partition count grows due to a number of bottlenecks, with AcquireExecutorLocks() showing at the top in profiles. Previous attempt at solving that problem was by David Rowley [1], where he proposed delaying locking of *all*

Re: How to retain lesser paths at add_path()?

2021-12-24 Thread Nils Dijk
Hi, While researching the viability to change Citus' planner in a way to take more advantage of the postgres planner I ran into the same limitations in add_path as described on this thread. For Citus we are investigating a possibility to introduce Path nodes that describe the operation of transer

Re: Proposal: sslmode=tls-only

2021-12-24 Thread Tom Lane
Keith Burdis writes: > Has consideration been given to having something like ssl-mode=tls-only > where the SSLRequest message is skipped and the TLS handshake starts > immediately with the protocol continuing after that? https://www.postgresql.org/message-id/flat/fcc3ebeb7f05775b63f3207ed52a54ea5

Re: minor gripe about lax reloptions parsing for views

2021-12-24 Thread Alvaro Herrera
On 2021-Dec-21, Mark Dilger wrote: > Rebased patch attached: These tests are boringly repetitive. Can't we have something like a nested loop, with AMs on one and reloptions on the other, where each reloption is tried on each AM and an exception block to report the failure or success for each cas

Re: Proposal: sslmode=tls-only

2021-12-24 Thread Keith Burdis
Servers that use sslmode=tls-only would not be compatible with clients that do not yet support it, but that is same for any similar server-side change, for example if the server requires a minimum of TLS 1.3 but the client only supports TLS 1.2. IIUC with the default sslmode=prefer a client curre

[PATCH] reduce page overlap of GiST indexes built using sorted method

2021-12-24 Thread Aliaksandr Kalenik
Hey! Postgres 14 introduces an option to create a GiST index using a sort method. It allows to create indexes much faster but as it had been mentioned in sort support patch discussion the faster build performance comes at cost of higher degree of overlap between pages than for indexes built with r

Re: Proposal: sslmode=tls-only

2021-12-24 Thread Andrew Dunstan
On 12/24/21 09:08, Keith Burdis wrote: > From 53.2.9. SSL Session Encryption: >   > > When SSL encryption can be performed, the server is expected to > send only the single S byte and then wait for the frontend to > initiate an SSL handshake. If additional bytes are available to >

Re: 回复:Re: Re: 回复:Re: Is it worth pushing conditions to sublink/subplan?

2021-12-24 Thread Zhihong Yu
On Thu, Dec 23, 2021 at 3:52 AM 曾文旌(义从) wrote: > > Fixed a bug found during testing. > > > Wenjing > > >>> Hi, + if (condition_is_safe_pushdown_to_sublink(rinfo, expr_info->outer)) + { + /* replace qual expr from outer var = const to var = const and push down to

Re: add recovery, backup, archive, streaming etc. activity messages to server logs along with ps display

2021-12-24 Thread Bharath Rupireddy
On Fri, Dec 24, 2021 at 7:19 PM Justin Pryzby wrote: > > On Sun, Dec 12, 2021 at 06:08:05PM +0530, Bharath Rupireddy wrote: > > Another idea could be to use the infrastructure laid out by the commit > > 9ce346e [1]. With ereport_startup_progress, we can emit the LOGs(of > > current recovering WAL

Proposal: sslmode=tls-only

2021-12-24 Thread Keith Burdis
>From 53.2.9. SSL Session Encryption: > When SSL encryption can be performed, the server is expected to send only > the single S byte and then wait for the frontend to initiate an SSL > handshake. If additional bytes are available to read at this point, it > likely means that a man-in-the-middle

Re: add recovery, backup, archive, streaming etc. activity messages to server logs along with ps display

2021-12-24 Thread Justin Pryzby
On Sun, Dec 12, 2021 at 06:08:05PM +0530, Bharath Rupireddy wrote: > Another idea could be to use the infrastructure laid out by the commit > 9ce346e [1]. With ereport_startup_progress, we can emit the LOGs(of > current recovering WAL file) for every log_startup_progress_interval > seconds/millisec

RE: Delay the variable initialization in get_rel_sync_entry

2021-12-24 Thread houzj.f...@fujitsu.com
On Friday, December 24, 2021 8:13 AM Michael Paquier wrote: > > On Thu, Dec 23, 2021 at 12:54:41PM -0300, Euler Taveira wrote: > > On Wed, Dec 22, 2021, at 10:11 AM, houzj.f...@fujitsu.com wrote: > >> The extra cost could sometimes be noticeable because get_rel_sync_entry is > a > >> hot functio

Re: Add new function to convert 32-bit XID to 64-bit

2021-12-24 Thread Ashutosh Bapat
On Thu, Dec 23, 2021 at 8:33 PM Fujii Masao wrote: > > Hi, > > When periodically collecting and accumulating statistics or status > information like pg_locks, pg_stat_activity, pg_prepared_xacts, etc for > future troubleshooting or some reasons, I'd like to store a transaction ID of > such info

Re: correct the sizes of values and nulls arrays in pg_control_checkpoint

2021-12-24 Thread Bharath Rupireddy
On Thu, Dec 23, 2021 at 9:16 PM Bharath Rupireddy wrote: > > On Thu, Dec 23, 2021 at 9:13 PM Euler Taveira wrote: > > > > On Thu, Dec 23, 2021, at 8:39 AM, Bharath Rupireddy wrote: > > > > pg_control_checkpoint emits 18 columns whereas the values and nulls > > arrays are defined to be of size 19.

Re: Be clear about what log_checkpoints emits in the documentation

2021-12-24 Thread Bharath Rupireddy
On Fri, Dec 24, 2021 at 11:30 AM Kyotaro Horiguchi wrote: > > At Thu, 23 Dec 2021 20:56:22 +0530, Bharath Rupireddy > wrote in > > Hi, > > > > Currently the documentation of the log_checkpoints GUC says the following: > > > > Some statistics are included in the log messages, including th

Re: Add checkpoint and redo LSN to LogCheckpointEnd log message

2021-12-24 Thread Bharath Rupireddy
On Fri, Dec 24, 2021 at 5:42 PM Michael Paquier wrote: > > On Fri, Dec 24, 2021 at 02:51:34PM +0900, Kyotaro Horiguchi wrote: > > I thougt about something like the following, but your proposal may be > > clearer. > > +"LSN=%X/%X, REDO LSN=%X/%X", > This could be rather confusing for the averag

Re: Add checkpoint and redo LSN to LogCheckpointEnd log message

2021-12-24 Thread Michael Paquier
On Fri, Dec 24, 2021 at 02:51:34PM +0900, Kyotaro Horiguchi wrote: > I thougt about something like the following, but your proposal may be > clearer. +"LSN=%X/%X, REDO LSN=%X/%X", This could be rather confusing for the average user, even if I agree that this is some information that only an ad

Re: Add checkpoint and redo LSN to LogCheckpointEnd log message

2021-12-24 Thread Bharath Rupireddy
On Fri, Dec 24, 2021 at 11:21 AM Kyotaro Horiguchi wrote: > > At Thu, 23 Dec 2021 20:35:54 +0530, Bharath Rupireddy > wrote in > > Hi, > > > > It is useful (for debugging purposes) if the checkpoint end message > > has the checkpoint LSN and REDO LSN [1]. It gives more context while > > analyzin

Re: An obsolete comment of pg_stat_statements

2021-12-24 Thread Michael Paquier
On Fri, Dec 24, 2021 at 03:32:10PM +0900, Kyotaro Horiguchi wrote: > Thanks! Looks better. It is used as-is in the attached. > > And I will register this to the next CF. Do we really need to have this comment in the function header? The same is explained a couple of lines down so this feels lik

Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes

2021-12-24 Thread Bharath Rupireddy
On Fri, Dec 24, 2021 at 4:43 PM Dilip Kumar wrote: > > On Fri, Dec 24, 2021 at 3:27 AM SATYANARAYANA NARLAPURAM > wrote: >> >> XLogInsert in my opinion is the best place to call it and the hook can be >> something like this "void xlog_insert_hook(NULL)" as all the throttling >> logic required

Re: more descriptive message for process termination due to max_slot_wal_keep_size

2021-12-24 Thread Masahiko Sawada
On Fri, Dec 24, 2021 at 5:30 PM Kyotaro Horiguchi wrote: > > Thank you for the comment. > > At Fri, 24 Dec 2021 17:06:57 +0900, Masahiko Sawada > wrote in > > Thank you for the patch! +1 for improving the messages. > > > > > > > > > LOG: terminating process %d to release replication slot \"%s\"

Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes

2021-12-24 Thread Dilip Kumar
On Fri, Dec 24, 2021 at 3:27 AM SATYANARAYANA NARLAPURAM < satyanarlapu...@gmail.com> wrote: > >> > XLogInsert in my opinion is the best place to call it and the hook can be > something like this "void xlog_insert_hook(NULL)" as all the throttling > logic required is the current flush position whi

Re: sequences vs. synchronous replication

2021-12-24 Thread Tomas Vondra
On 12/24/21 09:04, Kyotaro Horiguchi wrote: ... So, strictly speaking, that is a violation of the constraint I mentioned regardless whether the transaction is committed or not. However we have technical limitations as below. I don't follow. What violates what? If the transaction commits (a

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2021-12-24 Thread Kyotaro Horiguchi
Just a complaint.. At Fri, 12 Nov 2021 16:43:27 +0900 (JST), Kyotaro Horiguchi wrote in > "" on Japanese (CP-932) environment. I didn't actually > tested it on Windows and msys environment ...yet. Active perl cannot be installed because of (perhaps) a powershell version issue... Annoying.. ht

Re: more descriptive message for process termination due to max_slot_wal_keep_size

2021-12-24 Thread Kyotaro Horiguchi
Thank you for the comment. At Fri, 24 Dec 2021 17:06:57 +0900, Masahiko Sawada wrote in > Thank you for the patch! +1 for improving the messages. > > > > > > LOG: terminating process %d to release replication slot \"%s\" because > > > its restart_lsn %X/%X exceeds max_slot_wal_keep_size > >

Re: more descriptive message for process termination due to max_slot_wal_keep_size

2021-12-24 Thread Masahiko Sawada
On Fri, Dec 24, 2021 at 1:42 PM Kyotaro Horiguchi wrote: > > At Thu, 23 Dec 2021 18:08:08 +0530, Ashutosh Bapat > wrote in > > On Wed, Dec 15, 2021 at 9:42 AM Kyotaro Horiguchi > > wrote: > > > > LOG: invalidating slot "s1" > > > > DETAIL: The slot's restart_lsn 0/1D68 is behind the limit

Re: sequences vs. synchronous replication

2021-12-24 Thread Kyotaro Horiguchi
At Fri, 24 Dec 2021 08:23:13 +0100, Tomas Vondra wrote in > > > On 12/24/21 06:37, Kyotaro Horiguchi wrote: > > At Thu, 23 Dec 2021 19:50:22 +0100, Tomas Vondra > > wrote in > >> On 12/23/21 15:42, Fujii Masao wrote: > >>> On 2021/12/23 3:49, Tomas Vondra wrote: > Attached is a patch twe

Re: Allow escape in application_name

2021-12-24 Thread Fujii Masao
On 2021/12/24 13:49, Kyotaro Horiguchi wrote: I'm ok to remove the check "values[i] != NULL", but think that it's better to keep the other check "*(values[i]) != '\0'" as it is. Because *(values[i]) can be null character and it's a waste of cycles to call process_pgfdw_appname() in that case.