Re: logical replication empty transactions

On Mon, Mar 28, 2022 at 9:22 PM houzj.f...@fujitsu.com wrote: > > On Monday, March 28, 2022 3:08 PM Amit Kapila wrote: > > > > On Fri, Mar 25, 2022 at 12:50 PM houzj.f...@fujitsu.com > > wrote: > > > > > > Attach the new version patch with this change. > > > > > > > Few comments: > > Thanks for

Re: Accommodate startup process in a separate ProcState array slot instead of in MaxBackends slots.

On Tue, Mar 29, 2022 at 12:47 AM Robert Haas wrote: > > On Sat, Mar 26, 2022 at 2:23 AM Bharath Rupireddy > wrote: > > Thanks. It makes sense to put the comment in SInvalShmemSize. > > Attaching v2 patch. Please review it. > > How about this version, which I have edited lightly for grammar? Than

Re: shared-memory based stats collector - v66

Hi, On 2022-03-25 17:24:18 +0900, Kyotaro Horiguchi wrote: > I'd like to dump out my humble thoughts about other AFIXMEs.. Thanks! Please have another look at the code in https://github.com/anarazel/postgres/tree/shmstat I just pushed a revised version with a lot of [a]fixmes removed. Most imp

Re: shared-memory based stats collector - v67

Hi, On 2022-03-23 16:38:33 +0900, Kyotaro Horiguchi wrote: > At Tue, 22 Mar 2022 11:56:40 +0900 (JST), Kyotaro Horiguchi > wrote in > > Docs.. Yeah I'll try it. > > This is the first cut, based on the earlier patchset. Thanks! > I didn't mention pgstat_force_next_flush() since I think it i

Re: Add LZ4 compression in pg_dump

On Sat, Mar 26, 2022 at 01:14:41AM -0500, Justin Pryzby wrote: > See 0001 and the manpage. > > + 'pg_dump: compression is not supported by tar archive > format'); > > When I submitted a patch to support zstd, I spent awhile trying to make > compression work with tar, but it's a sig

Re: TRAP: FailedAssertion("HaveRegisteredOrActiveSnapshot()", File: "toast_internals.c", Line: 670, PID: 19403)

At Mon, 28 Mar 2022 18:36:46 +0900 (JST), Kyotaro Horiguchi wrote in > Inserting EnsurePortalSnapshotExists() to RunFromStore fixes this but > I'm not sure where is the right place to do this.. Then, I found that portal->holdSnapshot is that. I came up with the attached. It does the follows:

Re: Allow file inclusion in pg_hba and pg_ident files

Hi, On Tue, Mar 29, 2022 at 10:21:36AM +0900, Michael Paquier wrote: > On Mon, Mar 28, 2022 at 04:33:30PM +0800, Julien Rouhaud wrote: > > Ok, v5 attached without the TAP tests and updated sysviews tests. > > The update of the query related to pg_hba_file_rules in the regression > tests was indep

RE: logical replication empty transactions

On Tuesday, March 29, 2022 3:20 PM Masahiko Sawada wrote: > > Some comments: Thanks for the comments! > > + if (skipped_xact && > + SyncRepRequested() && > + ((volatile WalSndCtlData *) > WalSndCtl)->sync_standbys_defined) > + { > + WalSnd

Re: TRAP: FailedAssertion("HaveRegisteredOrActiveSnapshot()", File: "toast_internals.c", Line: 670, PID: 19403)

At Tue, 29 Mar 2022 17:06:21 +0900 (JST), Kyotaro Horiguchi wrote in > At Mon, 28 Mar 2022 18:36:46 +0900 (JST), Kyotaro Horiguchi > wrote in > Then, I found that portal->holdSnapshot is that. I came up with the > attached. It does the follows: > > 1. Teach PlannedStmtRequiresSnapshot() to

Re: logical replication empty transactions

On Tue, Mar 29, 2022 at 2:05 PM houzj.f...@fujitsu.com wrote: > > Attach the new version patch which addressed the > above comments and slightly adjusted some code comments. > The patch looks good to me. One minor suggestion is to change the function name ProcessPendingWritesAndTimeOut() to Proce

RE: logical replication empty transactions

On Tuesday, March 29, 2022 5:12 PM Amit Kapila wrote: > > On Tue, Mar 29, 2022 at 2:05 PM houzj.f...@fujitsu.com > wrote: > > > > Attach the new version patch which addressed the above comments and > > slightly adjusted some code comments. > > > > The patch looks good to me. One minor suggestio

RE: logical replication empty transactions

On Tue, Mar 29, 2022 5:15 PM Hou, Zhijie/侯 志杰 wrote: > > Thanks for the comment. > Attach the new version patch with this change. > Hi, I did a performance test for this patch to see if it affects performance when publishing empty transactions, based on the v32 patch. In this test, I use sync

Re: SQL/JSON: functions

Yes, it seems it was the ccache - I had enabled the 'ccache_failure_remove' flag last night and run a build, which failed, but the builds after that went green. Regards On Tue, 29 Mar 2022 at 06:41, Andres Freund wrote: > On 2022-03-28 19:25:51 -0400, Tom Lane wrote: > > ... even more baffling:

Enables to call Unregister*XactCallback() in Call*XactCallback()

Hi hackers, It's a natural requirement to unregister the callback for transaction or subtransaction when the callback is invoked, so we don't have to unregister the callback somewhere. If it's not allowed to do it in CallXactCallback() or CallSubXactCallback(), we must find the right place and han

Generic JSON API

Hi Hackers! Postgres Pro team would like to present a result of several years' development - a Generic JSON (GSON) API, announced in 2020 as "The Grand Unification". Finally, it is ready and awaiting review. For the sake of compatibility with SQL standard we need one JSON data type, which is inter

Re: Column Filtering in Logical Replication

On Sun, Mar 20, 2022 at 4:53 PM Tomas Vondra wrote: > > On 3/20/22 07:23, Amit Kapila wrote: > > On Sun, Mar 20, 2022 at 8:41 AM Amit Kapila wrote: > >> > >> On Fri, Mar 18, 2022 at 10:42 PM Tomas Vondra > >> wrote: > >> > >>> So the question is why those two sync workers never complete - I gues

Re: Add parameter jit_warn_above_fraction

Hi, On Mon, Mar 28, 2022 at 10:11:16PM +0200, Magnus Hagander wrote: > On Tue, Mar 22, 2022 at 12:50 AM Andres Freund wrote: > > > > This fails on cfbot, due to compiler warnings: > > https://cirrus-ci.com/task/5127667648299008?logs=mingw_cross_warning#L390 > > > Huh. That's annoying. I forgot in

Re: Frontend error logging style

> On 27 Mar 2022, at 22:19, Tom Lane wrote: > > Here's a rebase up to today's HEAD. I've fixed the merge problems, > but there may be some stray new error calls that could be converted > to use pg_fatal() and haven't been. I don't want to do a full > fresh scan of the code until we're about rea

Re: Identify missing publications from publisher while create/alter subscription.

On Tue, Mar 29, 2022 at 11:01 AM Amit Kapila wrote: > > On Sat, Mar 26, 2022 at 7:53 PM vignesh C wrote: > > > > The patch was not applying on HEAD, attached patch which is rebased on > > top of HEAD. > > > > IIUC, this patch provides an option that allows us to give an error if > while creating/

Re: TRAP: FailedAssertion("HaveRegisteredOrActiveSnapshot()", File: "toast_internals.c", Line: 670, PID: 19403)

On Tue, 29 Mar 2022 at 11:10, Kyotaro Horiguchi wrote: > > At Tue, 29 Mar 2022 17:06:21 +0900 (JST), Kyotaro Horiguchi > wrote in > > At Mon, 28 Mar 2022 18:36:46 +0900 (JST), Kyotaro Horiguchi > > wrote in > > Then, I found that portal->holdSnapshot is that. I came up with the > > attached.

Re: Column Filtering in Logical Replication

On 3/29/22 12:00, Amit Kapila wrote: > On Sun, Mar 20, 2022 at 4:53 PM Tomas Vondra > wrote: >> >> On 3/20/22 07:23, Amit Kapila wrote: >>> On Sun, Mar 20, 2022 at 8:41 AM Amit Kapila wrote: On Fri, Mar 18, 2022 at 10:42 PM Tomas Vondra wrote: > So the question is why

Re: TRAP: FailedAssertion("HaveRegisteredOrActiveSnapshot()", File: "toast_internals.c", Line: 670, PID: 19403)

Op 29-03-2022 om 12:50 schreef Matthias van de Meent: On Tue, 29 Mar 2022 at 11:10, Kyotaro Horiguchi wrote: At Tue, 29 Mar 2022 17:06:21 +0900 (JST), Kyotaro Horiguchi wrote in At Mon, 28 Mar 2022 18:36:46 +0900 (JST), Kyotaro Horiguchi wrote in Then, I found that portal->holdSnapshot is

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

On 2022-Mar-29, Kyotaro Horiguchi wrote: > > That's going to result in a call to > > XLogReadBufferForRedoExtended() which will call > > XLogReadBufferExtended() which will do smgrcreate(smgr, forknum, true) > > which will in turn call TablespaceCreateDbspace() to fill in all the > > missing direc

Re: Column Filtering in Logical Replication

On Tue, Mar 29, 2022 at 4:33 PM Tomas Vondra wrote: > > On 3/29/22 12:00, Amit Kapila wrote: > >> > >> Thanks, I'll take a look later. > >> > > > > This is still failing [1][2]. > > > > [1] - > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=florican&dt=2022-03-28%2005%3A16%3A53 > > [2]

Re: [PATCH] Full support for index LP_DEAD hint bits on standby

Hello, Greg. > I'm seeing a recovery test failure. Not sure if this represents an > actual bug or just a test that needs to be adjusted for the new > behaviour. Thanks for notifying me. It is a failure of a test added in the patch. It is a little hard to make it stable (because it depends on minR

Re: [PATCH] Full support for index LP_DEAD hint bits on standby

Hello, Peter. Thanks for your review! > I doubt that the patch's use of pg_memory_barrier() in places like > _bt_killitems() is correct. There is no way to know for sure if this > novel new lockless algorithm is correct or not, since it isn't > explained anywhere. The memory barrier is used only

Re: Add index scan progress to pg_stat_progress_vacuum

Sorry for the late reply. > additional complexity and a possible lag of progress updates. So if we > go with the current approach, I think we need to make sure enough (and > not too many) hash table entries. The hash table can be set 4 times the size of max_worker_processes which should give mor

Re: [PATCH] Enable SSL library detection via PQsslAttribute

Pushed with a few small tweaks to make it match project style, thanks! -- Daniel Gustafsson https://vmware.com/

Re: Add index scan progress to pg_stat_progress_vacuum

> I think that's an absolute no-go. Adding locking to progress reporting, > particularly a single central lwlock, is going to *vastly* increase the > overhead incurred by progress reporting. Sorry for the late reply. The usage of the shared memory will be limited to PARALLEL maintenance operation

Re: unlogged sequences

Patch rebased over some conflicts, and some tests simplified. On 24.03.22 14:10, Peter Eisentraut wrote: Here is an updated patch that now also includes SET LOGGED/UNLOGGED support.  So this version addresses all known issues and open problems. On 28.02.22 10:56, Peter Eisentraut wrote: reba

Re: speed up a logical replica setup

On 22.03.22 02:25, Andres Freund wrote: On 2022-02-21 09:09:12 -0300, Euler Taveira wrote: A new tool called pg_subscriber does this conversion and is tightly integrated with Postgres. Given that this has been submitted just before the last CF and is a patch of nontrivial size, has't made sign

Re: Column Filtering in Logical Replication

On 3/29/22 13:47, Amit Kapila wrote: > On Tue, Mar 29, 2022 at 4:33 PM Tomas Vondra > wrote: >> >> On 3/29/22 12:00, Amit Kapila wrote: Thanks, I'll take a look later. >>> >>> This is still failing [1][2]. >>> >>> [1] - >>> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

On Tue, Mar 29, 2022 at 7:37 AM Alvaro Herrera wrote: > I think we should revert this patch and do it again using the other > approach: create a stub directory during recovery that can be deleted > later. I'm fine with that approach, but I'd like to ask that we proceed expeditiously, because I ha

Re: refactoring basebackup.c (zstd workers)

On Mon, Mar 28, 2022 at 8:11 PM Tom Lane wrote: > I suspect Robert wrote it that way intentionally --- but if so, > I agree it could do with more than zero commentary. Well, the point is, we stop advancing kwend when we get to the end of the keyword, and *vend when we get to the end of the value.

Jumble Query with COERCE_SQL_SYNTAX

Good day. v14 introduced the way to get original text for some kind of expressions using new 'funcformat' - COERCE_SQL_SYNTAX: - EXTRACT(part from timestamp) - (text IS [form] NORMALIZED) and others. Mentioned EXTRACT and NORMALIZED statements has parts, that are not usual arguments but some kind

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi hackers, > I took another look at the patchset. Personally I don't think it will get much > better than it is now. I'm inclined to change the status of the CF entry to > "Ready for Committer" unless anyone disagrees. > > About v25-0002-Use-64-bit-format-to-output-XIDs.patch: > > I don't see th

Re: Add LZ4 compression in pg_dump

On Tue, Mar 29, 2022 at 1:03 AM Michael Paquier wrote: > > Then, if people are willing to adopt the syntax that the > > backup_compression.c/h stuff supports as a project standard (+1 from > > me) we can go the other way and rename that stuff to be more generic, > > taking backup out of the name.

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

On 2022-Mar-29, Robert Haas wrote: > I'm fine with that approach, but I'd like to ask that we proceed > expeditiously, because I have another patch that I want to commit that > touches this area. I can commit to helping with whatever we decide to > do here, but I don't want to keep that patch on i

Re: Accommodate startup process in a separate ProcState array slot instead of in MaxBackends slots.

On Tue, Mar 29, 2022 at 3:21 AM Bharath Rupireddy wrote: > Thanks. LGTM. Committed. -- Robert Haas EDB: http://www.enterprisedb.com

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

On Tue, Mar 29, 2022 at 9:28 AM Alvaro Herrera wrote: > OK, this is a bug that's been open for years. A fix can be committed > after the feature freeze anyway. +1 -- Robert Haas EDB: http://www.enterprisedb.com

Re: Add parameter jit_warn_above_fraction

On Tue, Mar 29, 2022 at 6:09 AM Julien Rouhaud wrote: > The last remaining thing is whether logging at WARNING level is the correct > choice. I'm personally fine with it, because the people who are going to use > it will probably use the same approach as for log_min_duration_statements: > enable

Re: Document atthasmissing default optimization avoids verification table scan

On Sun, Mar 27, 2022 at 11:12 PM David G. Johnston wrote: > > On Sun, Mar 27, 2022 at 11:17 AM James Coleman wrote: >> >> Hmm, I didn't realize that was project policy, > > > Guideline/Rule of Thumb is probably a better concept. Ah, OK, thanks. >> >> but I'm a bit >> surprised given that the s

Correct docs re: rewriting indexes when table rewrite is skipped

Back in 367bc42 (for 9.2!) we "avoid[ed] index rebuild[ing] for no-rewrite ALTER TABLE .. ALTER TYPE." However the docs still claim that "a table rewrite is not needed; but any indexes on the affected columns must still be rebuilt." I've attached a simple patch to update the docs to match the curr

Re: pgsql: Add 'basebackup_to_shell' contrib module.

On Fri, Mar 25, 2022 at 5:52 PM Thomas Munro wrote: > On Sat, Mar 26, 2022 at 9:55 AM Thomas Munro wrote: > > https://api.cirrus-ci.com/v1/artifact/task/5637156969381888/log/contrib/basebackup_to_shell/tmp_check/log/regress_log_001_basic > > This line doesn't look too healthy: > > pg_basebackup:

Restructure ALTER TABLE notes to clarify table rewrites and verification scans

Over in the "Document atthasmissing default optimization avoids verification table scan" thread David Johnston (who I've cc'd) suggested that my goals might be better implemented with a simple restructuring of the Notes section of the ALTER TABLE docs. I think this is also along the lines of Tom La

Re: Frontend error logging style

On 27.03.22 22:19, Tom Lane wrote: Here's a rebase up to today's HEAD. I've fixed the merge problems, but there may be some stray new error calls that could be converted to use pg_fatal() and haven't been. I don't want to do a full fresh scan of the code until we're about ready to commit this.

Re: Frontend error logging style

On 29.03.22 12:13, Daniel Gustafsson wrote: Most of these should probably be addressed separately from Tom's patch. @@ -508,24 +502,15 @@ writefile(char *path, char **lines) if (fclose(out_file)) - { - pg_log_error("could not write file \"%s\": %m", path); -

Re: Identify missing publications from publisher while create/alter subscription.

On Tue, Mar 29, 2022 at 11:02 AM Amit Kapila wrote: > > On Sat, Mar 26, 2022 at 7:53 PM vignesh C wrote: > > > > The patch was not applying on HEAD, attached patch which is rebased on > > top of HEAD. > > > > IIUC, this patch provides an option that allows us to give an error if > while creating/

Re: Identify missing publications from publisher while create/alter subscription.

On Tue, Mar 29, 2022 at 4:12 PM Amit Kapila wrote: > > On Tue, Mar 29, 2022 at 11:01 AM Amit Kapila wrote: > > > > On Sat, Mar 26, 2022 at 7:53 PM vignesh C wrote: > > > > > > The patch was not applying on HEAD, attached patch which is rebased on > > > top of HEAD. > > > > > > > IIUC, this patch

Re: Add header support to text format and matching feature

On 25.03.22 05:21, Greg Stark wrote: Great to see the first of the two patches committed. It looks like the second patch got some feedback from Peter in February and has been marked "Waiting on author" ever since. Remi, will you have a chance to look at this this month? Peter, are these commen

Re: pg14 psql broke \d datname.nspname.relname

On Fri, Mar 25, 2022 at 3:42 PM Mark Dilger wrote: > I think your change is fine, so I've rolled it into this next patch. OK, cool. Here are some more comments. In describe.c, why are the various describeWhatever functions returning true when validateSQLNamePattern returns false? It seems to me

Re: [PATCH] Enable SSL library detection via PQsslAttribute

On Tue, 2022-03-29 at 14:08 +0200, Daniel Gustafsson wrote: > Pushed with a few small tweaks to make it match project style, thanks! Thank you! --Jacob

Re: Correct docs re: rewriting indexes when table rewrite is skipped

On Tue, 29 Mar 2022 at 16:04, James Coleman wrote: > > Back in 367bc42 (for 9.2!) we "avoid[ed] index rebuild[ing] for > no-rewrite ALTER TABLE > .. ALTER TYPE." However the docs still claim that "a table rewrite is > not needed; but any indexes on the affected columns must still be > rebuilt." A

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

On Mon, Mar 28, 2022 at 3:08 PM Robert Haas wrote: > smgrcreate() as we would for most WAL records or whether it should be > adopting the new system introduced by > 49d9cfc68bf4e0d32a948fe72d5a0ef7f464944e. I wrote about this concern > over here: > > http://postgr.es/m/CA+TgmoYcUPL+WOJL2ZzhH=zmrhj

Re: make MaxBackends available in _PG_init

On Sat, Mar 26, 2022 at 1:23 PM Andres Freund wrote: > > And indeed, any third party code that previously needed to access what > > MaxBackends is supposed to store should already be using that formula, and > > the new GetMaxBackends() doesn't do anything about it. > > It couldn't rely on MaxBacke

Re: Corruption during WAL replay

Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Fri, Mar 25, 2022 at 10:34 AM Tom Lane wrote: > > I dunno. Compatibility and speed concerns aside, that seems like an awful > > lot of bits to be expending on every page compared to the value. > > I dunno either, but over on the TDE t

Re: [PATCH] Full support for index LP_DEAD hint bits on standby

UPD: > I was thinking it is safe to have additional hint bits > on primary, but it seems like no. Oh, sorry for the mistake, it is about standby of course. > BTW I am wondering if it is possible > to achieve the same situation by pg_rewind and standby promotion… Looks like it is impossible, bec

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

On Sun, Mar 27, 2022 at 11:24 PM Peter Geoghegan wrote: > Attached is v12. My current goal is to commit all 3 patches before > feature freeze. Note that this does not include the more complicated > patch including with previous revisions of the patch series (the > page-level freezing work that app

Re: Corruption during WAL replay

On Tue, Mar 29, 2022 at 12:34 PM Stephen Frost wrote: > Anyhow, this whole thread has struck me as a good reason to polish those > patches off and add on top of them an extended checksum ability, first, > independent of TDE, and remove the dependency of those patches from the > TDE effort and inst

Re: GSoC: Improve PostgreSQL Regression Test Coverage

Greetings, * Christos Maris (christos.c.ma...@gmail.com) wrote: > This is Christos Maris , and > I'd like to declare my interest in the GSoC project mentioned above. > I'm an experienced Fullstack Developer, but still an open-source beginner. > That, com

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On 29.03.22 15:09, Aleksander Alekseev wrote: Hi hackers, I took another look at the patchset. Personally I don't think it will get much better than it is now. I'm inclined to change the status of the CF entry to "Ready for Committer" unless anyone disagrees. About v25-0002-Use-64-bit-format

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

On 2022-03-29 11:55:05 -0400, Robert Haas wrote: > That commit having been reverted, I committed v6 instead. Let's see > what breaks... It fails in CI (for the mirror of the postgres repo on github): https://cirrus-ci.com/task/6279465603956736?logs=test_bin#L121 tap test log: https://api.cirrus-c

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

On Tue, Mar 29, 2022 at 1:35 PM Andres Freund wrote: > # Running: createdb -T foobar2 foobar6 -S wal_log > createdb: error: too many command-line arguments (first is "wal_log") > Try "createdb --help" for more information. > not ok 31 - createdb -T foobar2 foobar6 -S wal_log exit code 0 > > Looks

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

Andres Freund writes: > Looks like there's some problem with commandline parsing? That test script is expecting glibc-like laxness of switch parsing. Put the switches before the non-switch arguments. regards, tom lane

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

On Tue, Mar 29, 2022 at 1:53 PM Tom Lane wrote: > Andres Freund writes: > > Looks like there's some problem with commandline parsing? > > That test script is expecting glibc-like laxness of switch > parsing. Put the switches before the non-switch arguments. I just did that. :-) -- Robert Haas

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

Robert Haas writes: > On Tue, Mar 29, 2022 at 1:53 PM Tom Lane wrote: >> That test script is expecting glibc-like laxness of switch >> parsing. Put the switches before the non-switch arguments. > I just did that. :-) Yup, you pushed while I was typing. FWIW, I don't think it's "Windows" enfor

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

On Tue, Mar 29, 2022 at 2:17 PM Tom Lane wrote: > Robert Haas writes: > > On Tue, Mar 29, 2022 at 1:53 PM Tom Lane wrote: > >> That test script is expecting glibc-like laxness of switch > >> parsing. Put the switches before the non-switch arguments. > > > I just did that. :-) > > Yup, you pushe

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

Robert Haas writes: > On Tue, Mar 29, 2022 at 2:17 PM Tom Lane wrote: >> it's some random algorithm that they probably feel at liberty >> to change. > I guess that characterization surprises me. The man page for > getopt_long() says this, and has for a long time at least on systems > I've used:

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

On Tue, Mar 29, 2022 at 10:03 AM Robert Haas wrote: > I can understand this version of the commit message. Woohoo! I like > understanding things. That's good news. > I think the header comments for FreezeMultiXactId() focus way too much > on what the caller is supposed to do and not nearly enoug

Higher level questions around shared memory stats

Hi, Separate from the minutia in [1] I'd like to discuss a few questions of more general interest. I'll post another question or two later. 1) How to react to corrupted statsfiles? In HEAD we stop reading stats at the point we detect the stats file to be corrupted. The contents of the currently

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

On Tue, Mar 29, 2022 at 2:37 PM Tom Lane wrote: > Robert Haas writes: > > On Tue, Mar 29, 2022 at 2:17 PM Tom Lane wrote: > >> it's some random algorithm that they probably feel at liberty > >> to change. > > > I guess that characterization surprises me. The man page for > > getopt_long() says t

Re: pgsql: Add 'basebackup_to_shell' contrib module.

On Wed, Mar 30, 2022 at 3:08 AM Robert Haas wrote: > On Fri, Mar 25, 2022 at 5:52 PM Thomas Munro wrote: > > On Sat, Mar 26, 2022 at 9:55 AM Thomas Munro wrote: > > This line doesn't look too healthy: > > > > pg_basebackup: error: backup failed: ERROR: shell command "type con > > > "C:cirruscon

Re: Add parameter jit_warn_above_fraction

On Wed, 30 Mar 2022 at 02:38, Robert Haas wrote: > I think WARNING is fine. After all, the parameter is called > "jit_warn_above_fraction". I had a think about this patch. I guess it's a little similar to checkpoint_warning. The good thing about the checkpoint_warning is that in the LOG message

Re: Higher level questions around shared memory stats

On Tue, Mar 29, 2022 at 3:17 PM Andres Freund wrote: > 1) How to react to corrupted statsfiles? > > IMO it'd make more sense to just throw away all stats > in that case. That seems reasonable to me. I think there's some downside, in that stats are important, and having some of them might be bette

Re: CREATE TABLE ( .. STORAGE ..)

On Wed, 2 Feb 2022 at 11:13, Teodor Sigaev wrote: > > Hi! > > > Are they both set to name or ColId? Although they are the same. > > > > Thank you, fixed, that was just an oversight. > > > 2 For ColumnDef new member storage_name, did you miss the function > > _copyColumnDef() _equalColumnDef()? >

Re: pgsql: Add 'basebackup_to_shell' contrib module.

On Wed, Mar 30, 2022 at 8:30 AM Thomas Munro wrote: > On Wed, Mar 30, 2022 at 3:08 AM Robert Haas wrote: > > Here's a new version, hopefully rectifying that deficiency. I also add > > a second patch here, documenting basebackup_to_shell.required_role, > > because Joe Conway observed elsewhere tha

Re: Add parameter jit_warn_above_fraction

On Tue, Mar 29, 2022 at 1:06 PM David Rowley wrote: > That means that even if the total execution time of a plan was a true > reflection of the total estimated plan cost, then the fraction of time > spent (as is measured by jit_warn_above_fraction) doing JIT would > entirely depend on the number o

Re: pg_stat_reset_single_*_counters vs pg_stat_database.stats_reset

Hi, On 2022-03-24 13:12:24 +0100, Tomas Vondra wrote: > I agree it should have been documented, but I still stand behind the > current behavior. I'm not willing to die on this hill, but I think the > reasoning was/is sound. > > Firstly, calculating transactions/second, reads/second just from by >

Re: Commitfest Update

On Thu, Mar 17, 2022 at 02:07:16PM -0400, Greg Stark wrote: > So far this commitfest these 10 patches have been marked committed. > That leaves us with 175 "Needs Review" and 28 "Ready for Comitter" so > quite a ways to go ... If it were me, I'd move these out of the way somehow; WOA/RWF or move t

Re: Higher level questions around shared memory stats

Hi, On 2022-03-29 16:24:02 -0400, Robert Haas wrote: > On Tue, Mar 29, 2022 at 3:17 PM Andres Freund wrote: > > 1) How to react to corrupted statsfiles? > > > > IMO it'd make more sense to just throw away all stats > > in that case. > > That seems reasonable to me. I think there's some downside,

Re: Commitfest Update

On Tue, Mar 29, 2022 at 4:47 PM Justin Pryzby wrote: > If it were me, I'd move these out of the way somehow; WOA/RWF or move to June: > > https://commitfest.postgresql.org/37/3291/ Add PGDLLIMPORT to all direct or > indirect GUC variables I plan to do this yet, but it seemed best to leave it unt

Re: pg_stat_reset_single_*_counters vs pg_stat_database.stats_reset

On Tue, Mar 29, 2022 at 1:37 PM Andres Freund wrote: > > Secondly, to do anything really meaningful you need to calculate deltas, > > and be able to detect if some of the stats were reset for the particular > > interval. And the stat_reset timestamp was designed to be a simple way > > to detect t

Re: pgsql: Add 'basebackup_to_shell' contrib module.

On Tue, Mar 29, 2022 at 4:36 PM Thomas Munro wrote: > It failed: > > https://cirrus-ci.com/task/5567070686412800 > https://api.cirrus-ci.com/v1/artifact/task/5567070686412800/log/contrib/basebackup_to_shell/tmp_check/log/001_basic_primary.log > https://api.cirrus-ci.com/v1/artifact/task/5567070686

Re: Extend compatibility of PostgreSQL::Test::Cluster

On 1/21/22 09:59, Andrew Dunstan wrote: > On 1/21/22 02:47, Michael Paquier wrote: >> On Tue, Jan 18, 2022 at 06:35:39PM -0500, Andrew Dunstan wrote: >>> Here's a version that does that and removes some recent bitrot. >> I have been looking at the full set of features of Cluster.pm and the >> req

Re: Add parameter jit_warn_above_fraction

Peter Geoghegan writes: > On Tue, Mar 29, 2022 at 1:06 PM David Rowley wrote: >> That means that even if the total execution time of a plan was a true >> reflection of the total estimated plan cost, then the fraction of time >> spent (as is measured by jit_warn_above_fraction) doing JIT would >>

Re: Window Function "Run Conditions"

Hi, On 2022-03-29 15:11:52 +1300, David Rowley wrote: > One thing which I'm not sure about with the patch is how I'm handling > the evaluation of the runcondition in nodeWindowAgg.c. Instead of > having ExecQual() evaluate an OpExpr such as "row_number() over (...) > <= 10", I'm replacing the Win

remove reset_shared()

Hi hackers, Is there any reason to keep reset_shared() around anymore? It is now just a wrapper function for CreateSharedMemoryAndSemaphores(), and AFAICT the information in the comments is already covered by comments in the shared memory code. I think it's arguable that the name of the function

Re: Frontend error logging style

> On 29 Mar 2022, at 16:38, Peter Eisentraut > wrote: > > On 29.03.22 12:13, Daniel Gustafsson wrote: > > Most of these should probably be addressed separately from Tom's patch. Yeah, I think so too. I'll await input from Tom on how he wants to do, but I'm happy to take these to a new thread.

Re: Add parameter jit_warn_above_fraction

On Tue, Mar 29, 2022 at 3:04 PM Tom Lane wrote: > I think David's questions are sufficiently cogent and difficult > that we should not add jit_warn_above_fraction at this time. +1 -- Peter Geoghegan

Re: pgsql: Add 'basebackup_to_shell' contrib module.

Hi, On 2022-03-29 17:19:44 -0400, Robert Haas wrote: > On Tue, Mar 29, 2022 at 4:36 PM Thomas Munro wrote: > > It failed: > > > > https://cirrus-ci.com/task/5567070686412800 > > https://api.cirrus-ci.com/v1/artifact/task/5567070686412800/log/contrib/basebackup_to_shell/tmp_check/log/001_basic_pri

Re: pgsql: Add 'basebackup_to_shell' contrib module.

On 3/29/22 17:19, Robert Haas wrote: > On Tue, Mar 29, 2022 at 4:36 PM Thomas Munro wrote: >> It failed: >> >> https://cirrus-ci.com/task/5567070686412800 >> https://api.cirrus-ci.com/v1/artifact/task/5567070686412800/log/contrib/basebackup_to_shell/tmp_check/log/001_basic_primary.log >> https:/

Re: Frontend error logging style

Daniel Gustafsson writes: > On 29 Mar 2022, at 16:38, Peter Eisentraut > wrote: >> Most of these should probably be addressed separately from Tom's patch. > Yeah, I think so too. Agreed. I tried to confine my patch to mechanical changes, except for changing places where the detail/hint featur

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

Thanks for taking a look! On Thu, Mar 24, 2022 at 01:17:01PM +1300, Thomas Munro wrote: > /* we're only handling directories here, skip if it's not ours */ > -if (lstat(path, &statbuf) == 0 && !S_ISDIR(statbuf.st_mode)) > +if (lstat(path, &statbuf) != 0) > +ereport(ERROR, > +

Re: Add parameter jit_warn_above_fraction

Hi, On 2022-03-30 09:05:44 +1300, David Rowley wrote: > I really believe that the main problem here is that JIT only enables > when the *total* plan cost reaches a certain threshold. Yes, that is/was a clear design mistake. It wasn't quite as bad back when it was written - partitioning blows the

Re: [PATCH] Expose port->authn_id to extensions and triggers

On Sat, 2022-03-26 at 11:36 -0700, Andres Freund wrote: > > I also note that exposing it as a GUC means we have zero control over > > who/what can read it. Maybe that's not a problem, but it needs to be > > thought about before we go down that path. > > Yes, I think that's a fair concern. I like

Re: Add parameter jit_warn_above_fraction

On Wed, 30 Mar 2022 at 12:16, Andres Freund wrote: > > I did propose a patch to address this in [1]. It does need more work > > and I do plan to come back to it for v16. > > FWIW, that doesn't seem quite right - won't it stop JITing e.g. on the inner > side of a nested loop, just because it's chea

Re: pg_stat_reset_single_*_counters vs pg_stat_database.stats_reset

Hi, On 2022-03-29 14:14:05 -0700, David G. Johnston wrote: > On Tue, Mar 29, 2022 at 1:37 PM Andres Freund wrote: > > > > Secondly, to do anything really meaningful you need to calculate deltas, > > > and be able to detect if some of the stats were reset for the particular > > > interval. And th

Re: Add parameter jit_warn_above_fraction

David Rowley writes: > On Wed, 30 Mar 2022 at 12:16, Andres Freund wrote: >> FWIW, that doesn't seem quite right - won't it stop JITing e.g. on the inner >> side of a nested loop, just because it's cheap, even though that's where the >> bulk of the benefits comes from? > Yeah, I think the total

Re: Temporary tables versus wraparound... again

On Mon, 28 Mar 2022 at 16:30, Andres Freund wrote: > > > Make ON COMMIT DELETE ROWS reset relfrozenxmin and other table stats > > like normal truncate. Otherwise even typical short-lived transactions > > using temporary tables can easily cause them to reach relfrozenxid. > > Might be w

Re: [PATCH] Expose port->authn_id to extensions and triggers

Hi, On 2022-03-29 23:38:29 +, Jacob Champion wrote: > On Sat, 2022-03-26 at 11:36 -0700, Andres Freund wrote: > > > I also note that exposing it as a GUC means we have zero control over > > > who/what can read it. Maybe that's not a problem, but it needs to be > > > thought about before we go

  1   2   >