Re: Helper functions for wait_for_catchup() in Cluster.pm

2023-01-17 Thread Drouvot, Bertrand
Hi, On 1/17/23 12:23 PM, Alvaro Herrera wrote: On 2023-Jan-17, Drouvot, Bertrand wrote: The idea has been raised in [1], where we are adding more calls to wait_for_catchup() in 'replay' mode. This seems mostly useless as presented. Maybe if you're able to reduce the noise on the second

Re: Improve GetConfigOptionValues function

2023-01-17 Thread Nitin Jadhav
Attaching the patch. On Wed, Jan 18, 2023 at 1:21 PM Nitin Jadhav wrote: > > Hi, > > GetConfigOptionValues function extracts the config parameters for the > given variable irrespective of whether it results in noshow or not. > But the parent function show_all_settings ignores the values

Improve GetConfigOptionValues function

2023-01-17 Thread Nitin Jadhav
Hi, GetConfigOptionValues function extracts the config parameters for the given variable irrespective of whether it results in noshow or not. But the parent function show_all_settings ignores the values parameter if it results in noshow. It's unnecessary to fetch all the values during noshow. So

Deduplicate logicalrep_read_tuple()

2023-01-17 Thread Bharath Rupireddy
Hi, logicalrep_read_tuple() duplicates code for LOGICALREP_COLUMN_TEXT and LOGICALREP_COLUMN_BINARY introduced by commit 9de77b5. While it doesn't hurt anyone, deduplication makes code a bit leaner by 57 bytes [1]. I've attached a patch for $SUBJECT. Thoughts? [1] size

Re: Allow logical replication to copy tables in binary format

2023-01-17 Thread Bharath Rupireddy
On Thu, Jan 12, 2023 at 1:53 PM Melih Mutlu wrote: > > Hi, > > Thanks for your reviews. Thanks. I have some comments: 1. The performance numbers posted upthread [1] look impressive for the use-case tried, that's a 2.25X improvement or 55.6% reduction in execution times. However, it'll be great

Re: Modify the document of Logical Replication configuration settings

2023-01-17 Thread Michael Paquier
On Wed, Jan 18, 2023 at 07:00:43AM +, Takamichi Osumi (Fujitsu) wrote: > The attached patch includes below two changes for the description of > Logical Replication "Configuration Settings". > > 1. Add one brief description about wal_sender_timeout. >I made it similar to one other sentence

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-17 Thread Peter Smith
Here are my review comments for the latest patch v16-0001. (excluding the test code) == General 1. Since the value of min_apply_delay cannot be < 0, I was thinking probably it should have been declared everywhere in this patch as a uint64 instead of an int64, right? == Commit

Re: Generating code for query jumbling through gen_node_support.pl

2023-01-17 Thread Michael Paquier
On Tue, Jan 17, 2023 at 04:52:28PM +0900, Michael Paquier wrote: > On Tue, Jan 17, 2023 at 08:43:44AM +0100, Peter Eisentraut wrote: >> Ok, I understand now, and I agree with this approach over the opposite. I >> was confused because the snippet you showed above used "jumble_ignore", but >> your

Re: Fix GUC_NO_SHOW_ALL test scenario in 003_check_guc.pl

2023-01-17 Thread Nitin Jadhav
> We would miss the names of the parameters that are marked as NO_SHOW, > missing from pg_settings, making debugging harder. > > This would make the test more costly by forcing one SQL for each > GUC.. I agree. > We could extend pg_show_all_settings() with a boolean parameter to > enforce

Modify the document of Logical Replication configuration settings

2023-01-17 Thread Takamichi Osumi (Fujitsu)
Hi, hackers The attached patch includes below two changes for the description of Logical Replication "Configuration Settings". 1. Add one brief description about wal_sender_timeout. I made it similar to one other sentence for subscriber. 2. Fix a wrong GUC name "wal_receiver_retry_interval".

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-17 Thread Amit Kapila
On Fri, Jan 13, 2023 at 11:50 AM Peter Smith wrote: > > Here are some review comments for patch v79-0002. > So, this is about the latest v84-0001-Stop-extra-worker-if-GUC-was-changed. > == > > General > > 1. > > I saw that earlier in this thread Hou-san [1] and Amit [2] also seemed > to say

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-01-17 Thread Maciek Sakrejda
On Tue, Jan 17, 2023 at 9:22 AM Melanie Plageman wrote: > On Mon, Jan 16, 2023 at 4:42 PM Maciek Sakrejda wrote: > > I missed a couple of versions, but I think the docs are clearer now. > > I'm torn on losing some of the detail, but overall I do think it's a > > good trade-off. Moving some

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-17 Thread Takamichi Osumi (Fujitsu)
Hi, On Wednesday, January 18, 2023 2:19 PM Amit Kapila wrote: > On Wed, Jan 18, 2023 at 6:37 AM Takamichi Osumi (Fujitsu) > wrote: > > > > > > > > Can you please explain a bit more as asked above to understand the > > > difference? > > So, the current difference is that the time-delayed apply

Re: doc: add missing "id" attributes to extension packaging page

2023-01-17 Thread Brar Piening
On 17.01.2023 at 23:43, Karl O. Pinc wrote: It's good you asked. After poking about the XSLT 1.0 spec to refresh my memory I abandoned the "mode" approach entirely. The real "right way" is to use the import mechanism. I've attached a patch that "wraps" the section.heading template and adds

Re: Removing redundant grouping columns

2023-01-17 Thread David Rowley
On Wed, 18 Jan 2023 at 11:51, Tom Lane wrote: > If nobody has any comments on this, I'm going to go ahead and push > it. The value of the improvement is rapidly paling in comparison > to the patch's maintenance effort. No objections from me. David

Re: Logical replication timeout problem

2023-01-17 Thread Amit Kapila
On Tue, Jan 17, 2023 at 6:41 PM Ashutosh Bapat wrote: > > On Tue, Jan 17, 2023 at 3:34 PM Amit Kapila wrote: > > > > > > > I am a bit worried about the indirections that the wrappers and hooks > > > create. Output plugins call OutputPluginUpdateProgress() in callbacks > > > but I don't see why

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-17 Thread Amit Kapila
On Wed, Jan 18, 2023 at 6:37 AM Takamichi Osumi (Fujitsu) wrote: > > > > > Can you please explain a bit more as asked above to understand the > > difference? > So, the current difference is that the time-delayed apply > worker of logical replication doesn't apply the delayed transaction on the >

Re: Remove source code display from \df+?

2023-01-17 Thread Pavel Stehule
Hi út 17. 1. 2023 v 20:29 odesílatel Isaac Morland napsal: > On Thu, 12 Jan 2023 at 12:06, Isaac Morland > wrote: > > Thanks everybody. So based on the latest discussion I will: >> >> 1) rename the column from “Source code” to “Internal name”; and >> 2) change the contents to NULL except when

RE: Perform streaming logical transactions by background workers and parallel apply

2023-01-17 Thread wangw.f...@fujitsu.com
On Wed, Jan 18, 2023 12:36 PM Amit Kapila wrote: > On Tue, Jan 17, 2023 at 8:07 PM Masahiko Sawada > wrote: > > > > On Tue, Jan 17, 2023 at 6:14 PM houzj.f...@fujitsu.com > > wrote: > > > > > > On Tuesday, January 17, 2023 2:46 PM Masahiko Sawada > wrote: > > > > > > > > On Tue, Jan 17, 2023

Re: recovery modules

2023-01-17 Thread Nathan Bossart
On Wed, Jan 18, 2023 at 11:29:20AM +0900, Michael Paquier wrote: > And 0002 is applied. Thanks. Here's a rebased version of the last patch. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From c262ed4701fdb417c1af3a6730499cebb9e25030 Mon Sep 17 00:00:00 2001 From: Nathan Bossart

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-01-17 Thread John Naylor
On Mon, Jan 16, 2023 at 3:18 PM Masahiko Sawada wrote: > > On Mon, Jan 16, 2023 at 2:02 PM John Naylor > wrote: > > + * Add Tids on a block to TidStore. The caller must ensure the offset numbers > > + * in 'offsets' are ordered in ascending order. > > > > Must? What happens otherwise? > > It

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-17 Thread Amit Kapila
On Tue, Jan 17, 2023 at 8:07 PM Masahiko Sawada wrote: > > On Tue, Jan 17, 2023 at 6:14 PM houzj.f...@fujitsu.com > wrote: > > > > On Tuesday, January 17, 2023 2:46 PM Masahiko Sawada > > wrote: > > > > > > On Tue, Jan 17, 2023 at 12:37 PM houzj.f...@fujitsu.com > > > wrote: > > > I'm

Re: recovery modules

2023-01-17 Thread Michael Paquier
On Tue, Jan 17, 2023 at 10:23:56AM -0800, Nathan Bossart wrote: > Yeah, this seems cleaner. I removed BuildRestoreCommand() in v8. if (*sp == *lp) { - if (val) - { - appendStringInfoString(, val); - found = true; - } -

Re: Removing redundant grouping columns

2023-01-17 Thread David Rowley
On Wed, 18 Jan 2023 at 14:55, Richard Guo wrote: > Yeah, I noticed this too yesterday. I reviewed through these two > patches yesterday and I think they are in good shape now. I'm currently reviewing the two patches. David

Re: Removing redundant grouping columns

2023-01-17 Thread Richard Guo
On Wed, Jan 18, 2023 at 6:51 AM Tom Lane wrote: > I wrote: > > Yeah, sideswiped by 3c6fc5820 apparently. No substantive change needed. > > And immediately sideswiped by da5800d5f. Yeah, I noticed this too yesterday. I reviewed through these two patches yesterday and I think they are in good

Re: [PoC] Federated Authn/z with OAUTHBEARER

2023-01-17 Thread Andrey Chudnovsky
> All of this points at a bigger question to the community: if we choose > not to provide a flow implementation in libpq, is adding OAUTHBEARER > worth the additional maintenance cost? > My personal vote would be "no". I think the hook-only approach proposed > here would ensure that only larger

Re: Backends stalled in 'startup' state

2023-01-17 Thread Ashwin Agrawal
On Tue, Jan 17, 2023 at 4:52 PM Ashwin Agrawal wrote: > > We recently saw many backends (close to max_connection limit) get stalled > in 'startup' in one of the production environments for Greenplum (fork of > PostgreSQL). Tracing the reason, it was found all the tuples created by > bootstrap

Re: CI and test improvements

2023-01-17 Thread Justin Pryzby
On Tue, Jan 17, 2023 at 11:56:42AM -0800, Andres Freund wrote: > > $(CF_PGP_TESTS) is missing from contrib/pgcrypto/meson.build > > Assume that's the false positive? Yes > > I also tried but failed to write something to warn if "meson test" was > > run with a list of tests but without

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-17 Thread Takamichi Osumi (Fujitsu)
Hi, On Tuesday, January 17, 2023 9:54 PM Amit Kapila wrote: > On Tue, Jan 17, 2023 at 4:30 PM Takamichi Osumi (Fujitsu) > wrote: > > > > On Saturday, January 14, 2023 3:27 PM vignesh C > wrote: > > > > > 2) I'm not sure if this will add any extra coverage as the altering > > > value of

Re: almost-super-user problems that we haven't fixed yet

2023-01-17 Thread Nathan Bossart
On Tue, Jan 17, 2023 at 02:59:31PM -0500, Robert Haas wrote: > On Tue, Jan 17, 2023 at 1:42 PM Nathan Bossart > wrote: >> If we create a new batch of reserved connections, only roles with >> privileges of pg_use_reserved_connections would be able to connect if the >> number of remaining slots is

Re: constify arguments of copy_file() and copydir()

2023-01-17 Thread Nathan Bossart
On Wed, Jan 18, 2023 at 09:05:52AM +0900, Michael Paquier wrote: > Thanks, done. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: constify arguments of copy_file() and copydir()

2023-01-17 Thread Michael Paquier
On Tue, Jan 17, 2023 at 08:23:49PM +0100, Peter Eisentraut wrote: > Looks good to me. Thanks, done. -- Michael signature.asc Description: PGP signature

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-01-17 Thread Tomas Vondra
On 1/17/23 21:59, Justin Pryzby wrote: > On Tue, Jan 17, 2023 at 08:44:36PM +0100, Tomas Vondra wrote: >> On 1/9/23 09:44, Ilya Gladyshev wrote: >>> On Sun, 2023-01-08 at 10:48 -0600, Justin Pryzby wrote: On Sat, Dec 17, 2022 at 08:30:02AM -0600, Justin Pryzby wrote: > ... > >

Re: Can we let extensions change their dumped catalog schemas?

2023-01-17 Thread Jacob Champion
On 1/12/23 11:04, Jacob Champion wrote: > On Wed, Jan 11, 2023 at 1:03 PM Tom Lane wrote: >> Jacob Champion writes: >>> Right, I think it would have to be opt-in. Say, a new control file >>> option dump_version or some such. >> >> That would require all the installed extensions to cope with this

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-17 Thread Peter Geoghegan
On Tue, Jan 17, 2023 at 2:11 PM Robert Haas wrote: > On Tue, Jan 17, 2023 at 3:08 PM Peter Geoghegan wrote: > > If you assume that there is chronic undercounting of dead tuples > > (which I think is very common), ... > > Why do you think that? For the reasons I gave about statistics, random

Re: [EXTERNAL] Re: Support load balancing in libpq

2023-01-17 Thread Jacob Champion
On Fri, Jan 13, 2023 at 10:44 AM Jacob Champion wrote: > And my thought was that the one-time > initialization could be moved to a place that doesn't need to know the > connection options at all, to make it easier to reason about the > architecture. Say, next to the WSAStartup machinery. (And

Re: [PATCH] Add native windows on arm64 support

2023-01-17 Thread Andres Freund
Hi, On 2022-12-16 10:52:23 +, Niyas Sait wrote: > Subject: [PATCH v7] Enable postgres native build for windows-arm64 platform > elif host_cpu == 'arm' or host_cpu == 'aarch64' > > - prog = ''' > + if cc.get_id() == 'msvc' > +cdata.set('USE_ARMV8_CRC32C', false) > +

Re: Removing redundant grouping columns

2023-01-17 Thread Tom Lane
I wrote: > Yeah, sideswiped by 3c6fc5820 apparently. No substantive change needed. And immediately sideswiped by da5800d5f. If nobody has any comments on this, I'm going to go ahead and push it. The value of the improvement is rapidly paling in comparison to the patch's maintenance effort.

Re: [PoC] Federated Authn/z with OAUTHBEARER

2023-01-17 Thread Jacob Champion
On Sun, Jan 15, 2023 at 12:03 PM Andrey Chudnovsky wrote: > 2. Removed Device Code implementation in libpq. Several reasons: >- Reduce scope and focus on the protocol first. >- Device code implementation uses iddawc dependency. Taking this > dependency is a controversial step which

Re: doc: add missing "id" attributes to extension packaging page

2023-01-17 Thread Karl O. Pinc
On Tue, 17 Jan 2023 19:13:38 +0100 Brar Piening wrote: > On 17.01.2023 at 14:12, Karl O. Pinc wrote: > > If you're not willing to try I am willing to see if I can > > produce an example to work from. My XSLT is starting to > > come back. > > I'm certainly willing to try but I'd appreciate an

PGDOCS - sgml linkend using single-quotes

2023-01-17 Thread Peter Smith
Hi, I happened to notice some examples of SGML linkends that were using single quotes instead of double quotes. It didn't seem to be the conventional style because grepping (from doc/src/sgml folder) showed only a tiny fraction using single quotes. (single-quotes) $ grep --include=*.sgml -rn .

Re: Rework of collation code, extensibility

2023-01-17 Thread Peter Geoghegan
On Sat, Jan 14, 2023 at 12:03 PM Jeff Davis wrote: > The first goal I had was simply that the code was really hard to > understand and work on, and refactoring was justified to improve the > situation. > > The second goal, which is somewhat dependent on the first goal, is that > we really need an

Re: pgsql: Doc: add XML ID attributes to and tags.

2023-01-17 Thread Tom Lane
Peter Eisentraut writes: > On 12.01.23 00:05, Tom Lane wrote: >> That reminds me that I was going to suggest fixing the few existing >> variances from the "use '-' not '_'" policy: >> >> $ grep 'id="[a-zA-Z0-9-]*_' *sgml ref/*sgml >> config.sgml: > xreflabel="plan_cache_mode"> > should be

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-17 Thread Robert Haas
On Tue, Jan 17, 2023 at 3:08 PM Peter Geoghegan wrote: > If you assume that there is chronic undercounting of dead tuples > (which I think is very common), ... Why do you think that? > How many dead heap-only tuples are equivalent to one LP_DEAD item? > What about page-level concentrations, and

Re: Update comments in multixact.c

2023-01-17 Thread Peter Geoghegan
On Tue, Jan 17, 2023 at 1:33 AM shiy.f...@fujitsu.com wrote: > I noticed that commit 5212d447fa updated some comments in multixact.c because > SLRU truncation for multixacts is performed during VACUUM, instead of > checkpoint. Should the following comments which mentioned checkpointer be >

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-01-17 Thread Melanie Plageman
v49 attached On Tue, Jan 17, 2023 at 2:12 PM Andres Freund wrote: > On 2023-01-17 12:22:14 -0500, Melanie Plageman wrote: > > > > > +typedef struct PgStat_BackendIO > > > > +{ > > > > + PgStat_Counter > > > > data[IOCONTEXT_NUM_TYPES][IOOBJECT_NUM_TYPES][IOOP_NUM_TYPES]; > > > > +}

Re: document the need to analyze partitioned tables

2023-01-17 Thread Bruce Momjian
On Tue, Jan 17, 2023 at 03:00:50PM -0600, Justin Pryzby wrote: > On Tue, Jan 17, 2023 at 03:53:24PM -0500, Bruce Momjian wrote: > > On Thu, Jan 12, 2023 at 03:27:47PM -0800, Nathan Bossart wrote: > > > Here is my take on the wording: > > > > > > Since all the data for a partitioned table is

Re: Extracting cross-version-upgrade knowledge from buildfarm client

2023-01-17 Thread Tom Lane
Andrew Dunstan writes: > On 2023-01-16 Mo 21:58, Tom Lane wrote: >> I dunno if we want to stretch buildfarm owners' patience with yet >> another BF client release right now. On the other hand, I'm antsy >> to see if we can un-revert 1b4d280ea after doing a little more >> work in

Re: document the need to analyze partitioned tables

2023-01-17 Thread Justin Pryzby
On Tue, Jan 17, 2023 at 03:53:24PM -0500, Bruce Momjian wrote: > On Thu, Jan 12, 2023 at 03:27:47PM -0800, Nathan Bossart wrote: > > On Wed, Oct 05, 2022 at 10:37:01AM +0200, Laurenz Albe wrote: > > > On Mon, 2022-03-28 at 15:05 +0200, Tomas Vondra wrote: > > >> I've pushed the last version, and

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-01-17 Thread Justin Pryzby
On Tue, Jan 17, 2023 at 08:44:36PM +0100, Tomas Vondra wrote: > On 1/9/23 09:44, Ilya Gladyshev wrote: > > On Sun, 2023-01-08 at 10:48 -0600, Justin Pryzby wrote: > >> On Sat, Dec 17, 2022 at 08:30:02AM -0600, Justin Pryzby wrote: > >>> ... > >>> > >>> @committers: Is it okay to add nparts_done to

Re: document the need to analyze partitioned tables

2023-01-17 Thread Bruce Momjian
On Thu, Jan 12, 2023 at 03:27:47PM -0800, Nathan Bossart wrote: > On Wed, Oct 05, 2022 at 10:37:01AM +0200, Laurenz Albe wrote: > > On Mon, 2022-03-28 at 15:05 +0200, Tomas Vondra wrote: > >> I've pushed the last version, and backpatched it to 10 (not sure I'd > >> call it a bugfix, but I

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-17 Thread Andres Freund
Hi, On 2023-01-17 14:57:27 -0500, Robert Haas wrote: > > In pg_class: > > > > - The number of all frozen pages, like we do for all-visible > > > > That'd give us a decent upper bound for the amount of work we need to do > > to > > increase relfrozenxid. It's important that this is crash safe

Re: cutting down the TODO list thread

2023-01-17 Thread Bruce Momjian
On Mon, Jan 16, 2023 at 05:17:23PM +0700, John Naylor wrote: > > I wrote: > > > We could also revise the developer FAQ: > > - remove phrase "Outstanding features are detailed in Todo." > > - add suggestion to to check the Todo or Not_worth_doing pages to see if the > desired feature is

Re: cutting down the TODO list thread

2023-01-17 Thread Bruce Momjian
On Wed, Jan 11, 2023 at 02:09:56PM +0700, John Naylor wrote: > I've come up with some revised language, including s/15/16/ and removing the > category of "[E]" (easier to implement), since it wouldn't be here if it were > actually easy: I think it is still possible for a simple item to be

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-17 Thread Peter Geoghegan
On Tue, Jan 17, 2023 at 10:02 AM Andres Freund wrote: > Both you and Robert said this, and I have seen it be true, but typically not > for large high-throughput OLTP databases, where I found increasing > relfrozenxid to be important. Sure, there's probably some up/down through the > day / week,

Re: almost-super-user problems that we haven't fixed yet

2023-01-17 Thread Robert Haas
On Tue, Jan 17, 2023 at 1:42 PM Nathan Bossart wrote: > Alright. The one design question I have is whether this should be a new > set of reserved connections or replace superuser_reserved_connections > entirely. I think it should definitely be something new, not a replacement. > If we create a

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-17 Thread Robert Haas
On Tue, Jan 17, 2023 at 1:02 PM Andres Freund wrote: > On 2023-01-16 13:58:21 -0800, Peter Geoghegan wrote: > > On Fri, Jan 13, 2023 at 9:55 PM Andres Freund wrote: > > When I express skepticism of very high autovacuum_freeze_max_age > > settings, it's mostly just that I don't think that

Re: CI and test improvements

2023-01-17 Thread Andres Freund
Hi, On 2023-01-17 11:35:09 -0600, Justin Pryzby wrote: > The autoconf system runs all tap tests in t/*.pl, but meson requires > enumerating them in ./meson.build. Yes. It was a mistake that we ever used t/*.pl for make. For one, it means that make can't control concurrency meaningfully, due to

Re: Sampling-based timing for EXPLAIN ANALYZE

2023-01-17 Thread Tomas Vondra
On 1/17/23 19:46, Andres Freund wrote: > Hi, > > On 2023-01-17 19:00:02 +0100, Tomas Vondra wrote: >> On 1/17/23 18:02, Andres Freund wrote: >>> On 2023-01-17 15:52:07 +0100, Tomas Vondra wrote: That also does not have issues with timestamp "rounding" - considering e.g. sample rate

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-01-17 Thread Tomas Vondra
On 1/9/23 09:44, Ilya Gladyshev wrote: > On Sun, 2023-01-08 at 10:48 -0600, Justin Pryzby wrote: >> On Sat, Dec 17, 2022 at 08:30:02AM -0600, Justin Pryzby wrote: >>> ... >>> >>> @committers: Is it okay to add nparts_done to IndexStmt ? >> >> Any hint about this ? >> AFAIK fields added at the end

Re: Remove source code display from \df+?

2023-01-17 Thread Isaac Morland
On Thu, 12 Jan 2023 at 12:06, Isaac Morland wrote: Thanks everybody. So based on the latest discussion I will: > > 1) rename the column from “Source code” to “Internal name”; and > 2) change the contents to NULL except when the language (identified by > oid) is INTERNAL or C. > > Patch

[PATCH] Teach planner to further optimize sort in distinct

2023-01-17 Thread Ankit Kumar Pandey
Hi, this is extension of `teach planner to evaluate multiple windows in the optimal order` work applied to distinct operation. Based on discussions before

Re: constify arguments of copy_file() and copydir()

2023-01-17 Thread Peter Eisentraut
On 17.01.23 07:24, Michael Paquier wrote: On Mon, Jan 16, 2023 at 10:53:40AM +0900, Michael Paquier wrote: +1. While I don't forget about this thread.. Any objections if I were to apply that? Looks good to me.

Re: Refactor recordExtObjInitPriv()

2023-01-17 Thread Peter Eisentraut
On 16.01.23 23:43, Nathan Bossart wrote: On Mon, Jan 16, 2023 at 12:01:47PM +0100, Peter Eisentraut wrote: I have updated the patch as you suggested and split out the aggregate issue into a separate patch for clarity. LGTM committed

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-01-17 Thread Andres Freund
Hi, On 2023-01-17 12:22:14 -0500, Melanie Plageman wrote: > > > @@ -359,6 +360,15 @@ static const PgStat_KindInfo > > > pgstat_kind_infos[PGSTAT_NUM_KINDS] = { > > > .snapshot_cb = pgstat_checkpointer_snapshot_cb, > > > }, > > > > > > + [PGSTAT_KIND_IO] = { > > > +

Re: Reduce timing overhead of EXPLAIN ANALYZE using rdtsc?

2023-01-17 Thread Andres Freund
Hi, On 2023-01-17 12:26:57 -0500, Tom Lane wrote: > Andres Freund writes: > > Here's an updated version of the move to representing instr_time as > > nanoseconds. It's now split into a few patches: > > I took a quick look through this. Thanks! > > 0001) Add INSTR_TIME_SET_ZERO() calls where

Re: Sampling-based timing for EXPLAIN ANALYZE

2023-01-17 Thread Andres Freund
Hi, On 2023-01-17 19:00:02 +0100, Tomas Vondra wrote: > On 1/17/23 18:02, Andres Freund wrote: > > On 2023-01-17 15:52:07 +0100, Tomas Vondra wrote: > >> That also does not have issues with timestamp "rounding" - considering > >> e.g. sample rate 1000Hz, that's 1ms between samples. And it's quite

Re: almost-super-user problems that we haven't fixed yet

2023-01-17 Thread Nathan Bossart
On Mon, Jan 16, 2023 at 09:06:10PM -0500, Robert Haas wrote: > On Mon, Jan 16, 2023 at 5:37 PM Nathan Bossart > wrote: >> On Mon, Jan 16, 2023 at 02:29:56PM -0500, Robert Haas wrote: >> > 4. You can reserve a small number of connections for the superuser >> > with superuser_reserved_connections,

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-17 Thread Andres Freund
Hi, On 2023-01-17 10:26:52 -0500, Robert Haas wrote: > On Mon, Jan 16, 2023 at 11:11 PM Peter Geoghegan wrote: > > On Mon, Jan 16, 2023 at 8:25 AM Robert Haas wrote: > > > I really dislike formulas like Min(freeze_max_age * 2, 1 billion). > > > That looks completely magical from a user

Re: recovery modules

2023-01-17 Thread Nathan Bossart
On Tue, Jan 17, 2023 at 02:32:03PM +0900, Michael Paquier wrote: > Could it be cleaner in the long term to remove entirely > BuildRestoreCommand() and move the conversion of the xlogpath with > make_native_path() one level higher in the stack? Yeah, this seems cleaner. I removed

Re: doc: add missing "id" attributes to extension packaging page

2023-01-17 Thread Brar Piening
On 17.01.2023 at 14:12, Karl O. Pinc wrote: If you're not willing to try I am willing to see if I can produce an example to work from. My XSLT is starting to come back. I'm certainly willing to try but I'd appreciate an example in any case. My XSLT skills are mostly learning by doing

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-17 Thread Andres Freund
Hi, On 2023-01-16 13:58:21 -0800, Peter Geoghegan wrote: > On Fri, Jan 13, 2023 at 9:55 PM Andres Freund wrote: > When I express skepticism of very high autovacuum_freeze_max_age > settings, it's mostly just that I don't think that age(relfrozenxid) > is at all informative in the way that

Re: Sampling-based timing for EXPLAIN ANALYZE

2023-01-17 Thread Tomas Vondra
On 1/17/23 18:02, Andres Freund wrote: > Hi, > > On 2023-01-17 15:52:07 +0100, Tomas Vondra wrote: >> I don't understand why we would even use timestamps, in this case? AFAIK >> "sampling profilers" simply increment a counter for the executing node, >> and then approximate the time as

Re: CI and test improvements

2023-01-17 Thread Justin Pryzby
The autoconf system runs all tap tests in t/*.pl, but meson requires enumerating them in ./meson.build. This checks for and finds no missing tests in the current tree: $ for pl in `find src contrib -path '*/t/*.pl'`; do base=${pl##*/}; dir=${pl%/*}; meson=${dir%/*}/meson.build; grep "$base"

Re: Reduce timing overhead of EXPLAIN ANALYZE using rdtsc?

2023-01-17 Thread Tom Lane
Andres Freund writes: > Here's an updated version of the move to representing instr_time as > nanoseconds. It's now split into a few patches: I took a quick look through this. > 0001) Add INSTR_TIME_SET_ZERO() calls where otherwise 0002 causes gcc to > warn > Alternatively we can

Re: Sampling-based timing for EXPLAIN ANALYZE

2023-01-17 Thread Andres Freund
Hi, On 2023-01-17 15:52:07 +0100, Tomas Vondra wrote: > I don't understand why we would even use timestamps, in this case? AFAIK > "sampling profilers" simply increment a counter for the executing node, > and then approximate the time as proportional to the count. The timer interrupt distances

Re: Reduce timing overhead of EXPLAIN ANALYZE using rdtsc?

2023-01-17 Thread Andres Freund
Hi, On 2023-01-17 08:46:12 -0500, Robert Haas wrote: > On Fri, Jan 13, 2023 at 2:56 PM Andres Freund wrote: > > Does anybody see a reason to not move forward with this aspect? We do a fair > > amount of INSTR_TIME_ACCUM_DIFF() etc, and that gets a good bit cheaper by > > just using nanoseconds.

Re: Extracting cross-version-upgrade knowledge from buildfarm client

2023-01-17 Thread Tom Lane
Andrew Dunstan writes: > FYI crake has just passed the test with flying colours. Cool. I await the Windows machines' results with interest. regards, tom lane

Re: Extracting cross-version-upgrade knowledge from buildfarm client

2023-01-17 Thread Andrew Dunstan
On 2023-01-17 Tu 10:18, Tom Lane wrote: > Andrew Dunstan writes: >> On 2023-01-16 Mo 21:58, Tom Lane wrote: >>> I dunno if we want to stretch buildfarm owners' patience with yet >>> another BF client release right now. On the other hand, I'm antsy >>> to see if we can un-revert 1b4d280ea after

Re: Issue attaching a table to a partitioned table with an auto-referenced foreign key

2023-01-17 Thread Guillaume Lelarge
Quick ping, just to make sure someone can get a look at this issue :) Thanks. Le ven. 6 janv. 2023 à 11:07, Guillaume Lelarge a écrit : > Hello, > > One of our customers has an issue with partitions and foreign keys. He > works on a v13, but the issue is also present on v15. > > I attach a SQL

Re: minor bug

2023-01-17 Thread Tom Lane
Laurenz Albe writes: > On Mon, 2023-01-16 at 19:59 +0100, Torsten Förtsch wrote: >> So, the timestamp displayed in the log message is certainly wrong. > If recovery stops at a WAL record that has no timestamp, you get this > bogus recovery stop time. I think we should show the recovery stop

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-17 Thread Robert Haas
On Mon, Jan 16, 2023 at 11:11 PM Peter Geoghegan wrote: > On Mon, Jan 16, 2023 at 8:25 AM Robert Haas wrote: > > I really dislike formulas like Min(freeze_max_age * 2, 1 billion). > > That looks completely magical from a user perspective. Some users > > aren't going to understand autovacuum

Re: Extracting cross-version-upgrade knowledge from buildfarm client

2023-01-17 Thread Tom Lane
Andrew Dunstan writes: > On 2023-01-16 Mo 21:58, Tom Lane wrote: >> I dunno if we want to stretch buildfarm owners' patience with yet >> another BF client release right now. On the other hand, I'm antsy >> to see if we can un-revert 1b4d280ea after doing a little more >> work in

Re: [PATCH] Add <> support to sepgsql_restorecon

2023-01-17 Thread Ted X Toth
The intent of this patch is not to stop all relabeling only to stop sepgsql_restorecon from doing a bulk relabel. I believe sepgsql_object_relabel is called by the 'SECURITY LABEL' statement which I'm using to set the label of db objects to a specific context which I would not want altered

Re: Sampling-based timing for EXPLAIN ANALYZE

2023-01-17 Thread Tomas Vondra
On 1/15/23 21:22, Andres Freund wrote: > Hi, > > On 2023-01-13 09:11:06 +0100, David Geier wrote: >> Mostly I'm wondering if the sampling based approach gains us enough to be >> worth it, once the patch to use RDTSC hopefully landed (see [1]). > > Well, I'm not sure we have a path forward on

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-17 Thread Masahiko Sawada
On Tue, Jan 17, 2023 at 6:14 PM houzj.f...@fujitsu.com wrote: > > On Tuesday, January 17, 2023 2:46 PM Masahiko Sawada > wrote: > > > > On Tue, Jan 17, 2023 at 12:37 PM houzj.f...@fujitsu.com > > wrote: > > > Attach the new version 0001 patch which addressed all other comments. > > > > > > >

Re: Make use of assign_checkpoint_completion_target() to calculate CheckPointSegments correctly

2023-01-17 Thread Bharath Rupireddy
On Tue, Jan 17, 2023 at 12:31 PM Michael Paquier wrote: > > On Mon, Dec 26, 2022 at 06:12:34PM +0530, Bharath Rupireddy wrote: > > It looks like assign_checkpoint_completion_target() is defined [1], > > but never used, because of which CheckPointSegments may miss to > > account for changed

Re: daitch_mokotoff module

2023-01-17 Thread Dag Lem
Paul Ramsey writes: >> On Jan 12, 2023, at 7:30 AM, Dag Lem wrote: >> [...] >> >> Sure, I can do that. You don't think this much example text will be >> TL;DR? > > I can only speak for myself, but examples are the meat of > documentation learning, so as long as they come with enough >

Re: Switching XLog source from archive to streaming when primary available

2023-01-17 Thread Bharath Rupireddy
On Thu, Jan 12, 2023 at 6:21 AM Nathan Bossart wrote: > > On Tue, Oct 18, 2022 at 07:31:33AM +0530, Bharath Rupireddy wrote: > > In summary, the standby state machine in WaitForWALToBecomeAvailable() > > exhausts all the WAL in pg_wal before switching to streaming after > > failing to fetch from

GSoC 2023

2023-01-17 Thread Ilaria Battiston
Greetings -hackers, Our beloved Google Summer of Code is back for 2023, with a format similar to 2022: both medium and large sized projects can be proposed, with more flexibility on end dates. The program will be open to students and open source beginners, as stated in this blog post:

Re: Record queryid when auto_explain.log_verbose is on

2023-01-17 Thread torikoshia
On 2023-01-16 22:07, Julien Rouhaud wrote: Hi, On Mon, Jan 16, 2023 at 09:36:59PM +0900, torikoshia wrote: As far as I read the manual below, auto_explain.log_verbose should record logs equivalent to VERBOSE option of EXPLAIN. Ah good catch, that's clearly an oversight! Attached patch

Re: Reduce timing overhead of EXPLAIN ANALYZE using rdtsc?

2023-01-17 Thread Robert Haas
On Fri, Jan 13, 2023 at 2:56 PM Andres Freund wrote: > Does anybody see a reason to not move forward with this aspect? We do a fair > amount of INSTR_TIME_ACCUM_DIFF() etc, and that gets a good bit cheaper by > just using nanoseconds. We'd also save memory in BufferUsage (144-122 bytes), >

Re: Extracting cross-version-upgrade knowledge from buildfarm client

2023-01-17 Thread Andrew Dunstan
On 2023-01-16 Mo 21:58, Tom Lane wrote: > I've pushed the per-branch AdjustUpgrade.pm files and tested by performing > a fresh round of buildfarm runs with the patched TestUpgradeXversion.pm > file. I think we're in good shape with this project. > > I dunno if we want to stretch buildfarm

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2023-01-17 Thread Aleksander Alekseev
Hi hackers, > Yeah, pg_upgrade will briefly start and stop the old server to make > sure all the WAL is replayed, and won't transfer any of the files > over. AFAIK, major-version WAL changes are fine; it was the previous > claim that we could do it in a minor version that I was unsure about. OK,

Re: doc: add missing "id" attributes to extension packaging page

2023-01-17 Thread Karl O. Pinc
On Tue, 17 Jan 2023 06:57:23 +0100 Brar Piening wrote: > On 17.01.2023 at 02:05, Karl O. Pinc wrote: > > Or maybe the right way is to set a mode at the very top, > > the first apply-templates call, and not mess with the > > built-in templates at all. (You'd write your own > > "postgres-mode"

Re: Logical replication timeout problem

2023-01-17 Thread Ashutosh Bapat
On Tue, Jan 17, 2023 at 3:34 PM Amit Kapila wrote: > > > > I am a bit worried about the indirections that the wrappers and hooks > > create. Output plugins call OutputPluginUpdateProgress() in callbacks > > but I don't see why ReorderBufferProcessTXN() needs a callback to > > call

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-17 Thread Amit Kapila
On Tue, Jan 17, 2023 at 4:30 PM Takamichi Osumi (Fujitsu) wrote: > > On Saturday, January 14, 2023 3:27 PM vignesh C wrote: > > > 2) I'm not sure if this will add any extra coverage as the altering value of > > min_apply_delay is already tested in the regression, if so this test can be > >

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-17 Thread Hayato Kuroda (Fujitsu)
Dear hackers, > At present, in the latter case, > the apply worker exits without applying the suspended transaction > after ALTER SUBSCRIPTION DISABLE command for the subscription. > Meanwhile, there is no "disabling" command for physical replication, > but I checked the behavior about what

[PATCH] Constify proclist.h

2023-01-17 Thread Aleksander Alekseev
Hi hackers, This is a follow-up to [1] and c8ad4d81. > Additionally Bharath pointed out that there are other pieces of code > that we may want to change in a similar fashion, > proclist.h/proclist_types.h as one example. I didn't do this yet > because I would like to know the community opinion

Re: Fix database creation during installchecks for ICU cluster

2023-01-17 Thread vignesh C
On Tue, 29 Nov 2022 at 20:24, Nazir Bilal Yavuz wrote: > > Hi, > > Thanks for the patch! > > > On 10/29/22 12:54, Marina Polyakova wrote: > > > > 1) The ECPG tests fail because they use the SQL_ASCII encoding [2], > > the database template0 uses the ICU locale provider and SQL_ASCII is > > not

  1   2   >