Re: Improve description of XLOG_RUNNING_XACTS

2022-07-27 Thread Masahiko Sawada
On Thu, Jul 28, 2022 at 3:24 PM Kyotaro Horiguchi wrote: > > At Thu, 28 Jul 2022 09:56:33 +0530, Ashutosh Bapat > wrote in > > Thanks Masahiko for the updated patch. It looks good to me. > > > > I wonder whether the logic should be, similar > > to ProcArrayApplyRecoveryInfo() > > if (xlrec->sub

Re: Refactoring postgres_fdw/connection.c

2022-07-27 Thread Fujii Masao
On 2022/07/27 10:36, Kyotaro Horiguchi wrote: At Tue, 26 Jul 2022 18:33:04 +0900, Fujii Masao wrote in I'm not sure the two are similar with each other. The new function pgfdw_exec_pre_commit() looks like a merger of two isolated code paths intended to share a seven-line codelet. I feel t

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-27 Thread Masahiko Sawada
() an On Thu, Jul 28, 2022 at 12:21 PM Amit Kapila wrote: > > On Thu, Jul 28, 2022 at 7:18 AM Masahiko Sawada wrote: > > > > On Wed, Jul 27, 2022 at 8:33 PM Amit Kapila wrote: > > > > > > > > I have changed accordingly in the attached > > > and apart from that slightly modified the comments an

Re: Improve description of XLOG_RUNNING_XACTS

2022-07-27 Thread Kyotaro Horiguchi
At Thu, 28 Jul 2022 09:56:33 +0530, Ashutosh Bapat wrote in > Thanks Masahiko for the updated patch. It looks good to me. > > I wonder whether the logic should be, similar > to ProcArrayApplyRecoveryInfo() > if (xlrec->subxid_overflow) > ... > else if (xlrec->subxcnt > 0) > ... > > But you ma

How to get accurate backup end time when it is taken from synchronous standby ?

2022-07-27 Thread Harinath Kanchu
Greetings, When we take backups from a synchronous standby replica, how can we get the accurate timestamp of the backup end time ? (As backup history files are not generated on standbys) For example: this is a part of control file after a backup (created using wal-g by calling pg_startbackup a

Re: Fix annotations nextFullXid

2022-07-27 Thread Fujii Masao
On 2022/07/27 1:29, Zhang Mingli wrote: Thanks! Pushed. Thanks! Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: Handle infinite recursion in logical replication setup

2022-07-27 Thread Peter Smith
Hi Vignesh. FYI the v39* patch fails to apply [1]. Can you please rebase it? [1] === Applying patches on top of PostgreSQL commit ID 5f858dd3bebd1f3845aef2bff7f4345bfb7b74b3 === === applying patch ./v39-0001-Check-and-throw-an-error-if-publication-tables-w.patch patching file doc/src/sgml/ref/al

Re: Documentation about PL transforms

2022-07-27 Thread Pavel Stehule
čt 28. 7. 2022 v 4:06 odesílatel napsal: > On 2022-07-13 00:26, Pavel Stehule wrote: > > > 1. I am not sure if get_call_trftypes is a good name - the prefix > > get_call > > is used when some runtime data is processed. > > I guess I hadn't caught on that the prefix carried that meaning. > To me,

RE: Perform streaming logical transactions by background workers and parallel apply

2022-07-27 Thread kuroda.hay...@fujitsu.com
Dear Wang, I found further comments about the test code. 11. src/test/regress/sql/subscription.sql ``` -- fail - streaming must be boolean CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false, streaming = foo); ``` The comment i

Re: Official Windows Installer and Documentation

2022-07-27 Thread David G. Johnston
On Wed, Jul 27, 2022 at 8:22 PM Tom Lane wrote: > I wrote: > > If EDB isn't adequately filling in the documentation for the behavior > > of their packaging, that's on them. > > Having now looked more closely at the pg_upgrade documentation, > I don't think this is exactly EDB's fault; it's text t

Re: Improve description of XLOG_RUNNING_XACTS

2022-07-27 Thread Ashutosh Bapat
Thanks Masahiko for the updated patch. It looks good to me. I wonder whether the logic should be, similar to ProcArrayApplyRecoveryInfo() if (xlrec->subxid_overflow) ... else if (xlrec->subxcnt > 0) ... But you may ignore it. -- Best Wishes, Ashutosh On Thu, Jul 28, 2022 at 7:41 AM Masahiko S

Re: [Patch] ALTER SYSTEM READ ONLY

2022-07-27 Thread Amul Sul
Hi, On Thu, Jul 28, 2022 at 4:05 AM Jacob Champion wrote: > > On Fri, Apr 8, 2022 at 7:27 AM Amul Sul wrote: > > Attached is rebase version for the latest maste head(#891624f0ec). > > Hi Amul, > > I'm going through past CF triage emails today; I noticed that this > patch dropped out of the commi

Re: Official Windows Installer and Documentation

2022-07-27 Thread Tom Lane
I wrote: > If EDB isn't adequately filling in the documentation for the behavior > of their packaging, that's on them. Having now looked more closely at the pg_upgrade documentation, I don't think this is exactly EDB's fault; it's text that should never have been there to begin with. ISTM we need

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-27 Thread Amit Kapila
On Thu, Jul 28, 2022 at 7:18 AM Masahiko Sawada wrote: > > On Wed, Jul 27, 2022 at 8:33 PM Amit Kapila wrote: > > > > > I have changed accordingly in the attached > > and apart from that slightly modified the comments and commit message. > > Do let me know what you think of the attached? > > It

Re: Skip partition tuple routing with constant partition key

2022-07-27 Thread David Rowley
On Thu, 28 Jul 2022 at 00:50, Amit Langote wrote: > So, in a way the caching scheme works for > LIST partitioning only if the same value appears consecutively in the > input set, whereas it does not for *a set of* values belonging to the > same partition appearing consecutively. Maybe that's a re

Re: Official Windows Installer and Documentation

2022-07-27 Thread Julien Rouhaud
On Wed, Jul 27, 2022 at 07:31:35PM -0700, David G. Johnston wrote: > > Ultimately we do our users the best service if when they operate an > installation using defaults that they have documentation showing how to > perform something like an upgrade that works with those defaults. I don't really ag

Re: Official Windows Installer and Documentation

2022-07-27 Thread Tom Lane
"David G. Johnston" writes: > On Wed, Jul 27, 2022 at 6:42 PM Julien Rouhaud wrote: >> Note that there's no "official" Windows installer, Yeah, that. > Our technical definition aside, the fact is our users consider the sole EDB > installer to be official. > If the ultimate solution is to update

RE: Support load balancing in libpq

2022-07-27 Thread kuroda.hay...@fujitsu.com
Dear Jelte, > With plain Postgres this assumption is probably correct. But the main reason > I'm interested in this patch was because I would like to be able to load > balance across the workers in a Citus cluster. These workers are all > primaries. > Similar usage would likely be possible with B

Re: Official Windows Installer and Documentation

2022-07-27 Thread David G. Johnston
On Wednesday, July 27, 2022, Julien Rouhaud wrote: > On Wed, Jul 27, 2022 at 07:02:51PM -0700, David G. Johnston wrote: > > > > In the end the problem is ours and cannot be simply assigned to a > > third-party. So let's resolve it here (on this list, whatever the > > solution) where representati

Re: Official Windows Installer and Documentation

2022-07-27 Thread Julien Rouhaud
On Wed, Jul 27, 2022 at 07:02:51PM -0700, David G. Johnston wrote: > > In the end the problem is ours and cannot be simply assigned to a > third-party. So let's resolve it here (on this list, whatever the > solution) where representatives from all parties are present. We could amend the pg_upgrad

Re: fairywren hung in pg_basebackup tests

2022-07-27 Thread Thomas Munro
On Thu, Jul 28, 2022 at 12:41 PM Thomas Munro wrote: > I thought about putting it at the top, but don't we really only need > to make an extra system call if MinGW's stat() told us it saw a > directory? And what if you asked to look through symlinks? I thought > about putting it near the S_ISLNK

Re: Improve description of XLOG_RUNNING_XACTS

2022-07-27 Thread Masahiko Sawada
On Thu, Jul 21, 2022 at 10:13 PM Ashutosh Bapat wrote: > > Hi > > On Thu, Jul 21, 2022 at 6:44 AM Masahiko Sawada wrote: > > > > Hi, > > > > I realized that standby_desc_running_xacts() in standbydesc.c doesn't > > describe subtransaction XIDs. I've attached the patch to improve the > > descripti

RE: [Commitfest 2022-07] Patch Triage: Waiting on Author

2022-07-27 Thread houzj.f...@fujitsu.com
On Wednesday, July 27, 2022 3:27 AM Jacob Champion wrote: > > - Parallel INSERT SELECT take 2 > https://commitfest.postgresql.org/38/3143/ > > There was a lot of discussion early on in this patchset's life, and > then it got caught in a rebase loop without review in August 2021. The > t

Re: Documentation about PL transforms

2022-07-27 Thread chap
On 2022-07-13 00:26, Pavel Stehule wrote: 1. I am not sure if get_call_trftypes is a good name - the prefix get_call is used when some runtime data is processed. I guess I hadn't caught on that the prefix carried that meaning. To me, it appeared to be a prefix used to avoid being specific to

Re: Official Windows Installer and Documentation

2022-07-27 Thread David G. Johnston
On Wed, Jul 27, 2022 at 6:42 PM Julien Rouhaud wrote: > Hi, > > On Wed, Jul 27, 2022 at 11:36:11PM +0200, Thomas Kellerer wrote: > > David G. Johnston schrieb am 27.07.2022 um 21:21: > > > And then there is the issue of file ownership. > > > > > > Assuming we want better documentation for this sp

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-27 Thread Masahiko Sawada
On Wed, Jul 27, 2022 at 8:33 PM Amit Kapila wrote: > > On Mon, Jul 25, 2022 at 11:26 AM Masahiko Sawada > wrote: > > > > On Mon, Jul 25, 2022 at 10:45 AM Masahiko Sawada > > wrote: > > > > I've attached the patch for REl15 that I forgot. > > > > I feel the place to remember running xacts infor

Re: collect_corrupt_items_vacuum.patch

2022-07-27 Thread Robert Haas
On Wed, Jul 27, 2022 at 5:56 PM Tom Lane wrote: > Maybe we need a different function for pg_visibility to call? > If we want ComputeXidHorizons to serve both these purposes, then it > has to always deliver exactly the right answer, which seems like > a definition that will be hard and expensive to

Re: Official Windows Installer and Documentation

2022-07-27 Thread Julien Rouhaud
Hi, On Wed, Jul 27, 2022 at 11:36:11PM +0200, Thomas Kellerer wrote: > David G. Johnston schrieb am 27.07.2022 um 21:21: > > And then there is the issue of file ownership. > > > > Assuming we want better documentation for this specific issue for > > back-patching what would that look like? > > >

Re: Introduce wait_for_subscription_sync for TAP tests

2022-07-27 Thread Masahiko Sawada
On Wed, Jul 27, 2022 at 8:54 PM Amit Kapila wrote: > > On Tue, Jul 26, 2022 at 1:12 PM Masahiko Sawada wrote: > > > > On Tue, Jul 26, 2022 at 2:01 PM Amit Kapila wrote: > > > > > > 2. > > > +# wait for the replication to catchup if required. > > > +if (defined($publisher)) > > > +{ >

Re: Introduce wait_for_subscription_sync for TAP tests

2022-07-27 Thread Masahiko Sawada
On Wed, Jul 27, 2022 at 7:08 PM shiy.f...@fujitsu.com wrote: > > On Tue, Jul 26, 2022 3:42 PM Masahiko Sawada wrote: > > > > I've attached an updated patch as well as a patch to remove duplicated > > waits in 007_ddl.pl. > > > > Thanks for your patch. Here are some comments. Thank you for the co

Re: fairywren hung in pg_basebackup tests

2022-07-27 Thread Thomas Munro
On Thu, Jul 28, 2022 at 3:21 AM Andrew Dunstan wrote: > On 2022-07-27 We 10:58, Tom Lane wrote: > > Andrew Dunstan writes: > >> The alternative I thought of would be to switch msys to using our > >> dirent.c. Probably not too hard, but certainly more work than reverting. Thanks for figuring this

Re: Proposal: add a debug message about using geqo

2022-07-27 Thread David G. Johnston
On Fri, Jul 22, 2022 at 1:20 PM Jacob Champion wrote: > On Wed, Jun 1, 2022 at 11:09 PM KAWAMOTO Masaya > wrote: > > That sounds a nice idea. But I don't think that postgres shows in the > > EXPLAIN output why the plan is selected. Would it be appropriate to > > show that GEQO is used in EXPLAIN

Re: Unprivileged user can induce crash by using an SUSET param in PGOPTIONS

2022-07-27 Thread Tom Lane
I wrote: > =?utf-8?Q?Dagfinn_Ilmari_Manns=C3=A5ker?= writes: >> Thanks! Just one minor nitpick: setting an %ENV entry to `undef` >> doesn't unset the environment variable, it sets it to the empty string. >> To unset a variable it needs to be deleted from %ENV, i.e. `delete >> $ENV{PGUSER};`. > A

Re: [Patch] ALTER SYSTEM READ ONLY

2022-07-27 Thread Jacob Champion
On Fri, Apr 8, 2022 at 7:27 AM Amul Sul wrote: > Attached is rebase version for the latest maste head(#891624f0ec). Hi Amul, I'm going through past CF triage emails today; I noticed that this patch dropped out of the commitfest when you withdrew it in January, but it hasn't been added back with

Re: collect_corrupt_items_vacuum.patch

2022-07-27 Thread Tom Lane
Robert Haas writes: > In reality, the oldest all-visible XID cannot move backward, but > ComputeXidHorizons() lets it move backward, because it's intended for > use by a caller who wants to mark pages all-visible, and it's only > concerned with making sure that the value is old enough to be safe.

Re: collect_corrupt_items_vacuum.patch

2022-07-27 Thread Robert Haas
On Mon, Apr 4, 2022 at 4:51 AM Daniel Shelepanov wrote: > I’ve been working on this > [https://www.postgresql.org/message-id/flat/cfcca574-6967-c5ab-7dc3-2c82b6723b99%40mail.ru] > bug. Finally, I’ve come up with the patch you can find attached. Basically > what is does is rises a PROC_IN_VACUUM

Re: Official Windows Installer and Documentation

2022-07-27 Thread Thomas Kellerer
David G. Johnston schrieb am 27.07.2022 um 21:21: And then there is the issue of file ownership. Assuming we want better documentation for this specific issue for back-patching what would that look like? Going forward should our installer be creating the postgres user for consistency with other

Re: [PoC] Reducing planning time when tables have many partitions

2022-07-27 Thread David Rowley
On Wed, 27 Jul 2022 at 18:07, Yuya Watari wrote: > I agree that introducing a Bitmapset field may solve this problem. I > will try this approach in addition to previous ones. I've attached a very half-done patch that might help you get started on this. There are still 2 failing regression tests w

Re: ExecRTCheckPerms() and many prunable partitions

2022-07-27 Thread Tom Lane
... One more thing: maybe we should rethink where to put extraUpdatedCols. Between the facts that it's not used for actual permissions checks, and that it's calculated by the rewriter not parser, it doesn't seem like it really belongs in RelPermissionInfo. Should we keep it in RangeTblEntry? Shou

Re: ExecRTCheckPerms() and many prunable partitions

2022-07-27 Thread Tom Lane
Amit Langote writes: > [ v16 patches ] I took a quick look at this ... I think that the notion behind MergeRelPermissionInfos, ie that a single RelPermissionInfo can represent *all* the checks for a given table OID, is fundamentally wrong. For example, when merging a view into an outer query th

Re: out of date comment in commit_ts.c

2022-07-27 Thread Nathan Bossart
On Tue, Jul 26, 2022 at 10:33:43AM -0700, Nathan Bossart wrote: > IIUC the ability for callers to request WAL record generation is no longer > possible as of 08aa89b [0]. Should the second sentence be removed? Here's a patch. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com diff --

Re: pgsql: Remove the restriction that the relmap must be 512 bytes.

2022-07-27 Thread Tom Lane
Robert Haas writes: > In short, I think Alvaro's idea is unprincipled but solves the problem > of making it clear that a new initdb is required. Your idea is > principled but does not solve any problem. [ shrug... ] Fair enough. regards, tom lane

Re: pgsql: Remove the restriction that the relmap must be 512 bytes.

2022-07-27 Thread Robert Haas
On Wed, Jul 27, 2022 at 3:17 PM Tom Lane wrote: > Alvaro Herrera writes: > >> Hmm, but that doesn't actually produce nice behavior. It just goes > >> into an infinite loop, like this: > >> So now I'm back to being unsure what to do here. > > > I vote to go for the catversion bump for now. > > Wha

Re: Allow pageinspect's bt_page_stats function to return a set of rows instead of a single row

2022-07-27 Thread Tom Lane
Hamid Akhtar writes: > That attached patch is based on the master branch. It makes the following > changes to the pageinspect contrib module: > - Updates bt_page_stats_internal function to accept 3 arguments instead of > 2. > - The function now uses SRF macros to return a set rather than a single

Official Windows Installer and Documentation

2022-07-27 Thread David G. Johnston
Hey, Just interacted with a frustrated user on Slack trying to upgrade from v13 to v14 on Windows. Our official download page for the Windows installer claims the core documentation as its official reference - can someone responsible for this area please suggest and test some changes to make this

Re: pgsql: Remove the restriction that the relmap must be 512 bytes.

2022-07-27 Thread Tom Lane
Alvaro Herrera writes: >> Hmm, but that doesn't actually produce nice behavior. It just goes >> into an infinite loop, like this: >> So now I'm back to being unsure what to do here. > I vote to go for the catversion bump for now. What this is showing us is that any form of corruption in the re

Re: Allow foreign keys to reference a superset of unique columns

2022-07-27 Thread Tom Lane
Kaiting Chen writes: > I'd like to propose a change to PostgreSQL to allow the creation of a foreign > key constraint referencing a superset of uniquely constrained columns. TBH, I think this is a fundamentally bad idea and should be rejected outright. It fuzzes the semantics of the FK relations

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2022-07-27 Thread Alvaro Herrera
Okay, I think I'm done with this. Here's v27 for the master branch, where I fixed some comments as well as thinkos in the test script. The ones on older branches aren't materially different, they just have tonnes of conflicts resolved. I'll get this pushed tomorrow morning. I have run it through

Re: pgsql: Remove the restriction that the relmap must be 512 bytes.

2022-07-27 Thread Alvaro Herrera
On 2022-Jul-27, Robert Haas wrote: > Hmm, but that doesn't actually produce nice behavior. It just goes > into an infinite loop, like this: > 2022-07-27 14:21:12.869 EDT [32853] FATAL: relation mapping file > "global/pg_filenode.map" contains invalid data This seems almost identical what happen

small windows psqlrc re-wording

2022-07-27 Thread Robert Treat
Howdy folks, The attached patch tweaks the wording around finding the psqlrc file on windows, with the primary goal of removing the generally incorrect statement that windows has no concept of a home directory. Robert Treat https://xzilla.net windows-psqlrc.patch Description: Binary data

Re: pgsql: Remove the restriction that the relmap must be 512 bytes.

2022-07-27 Thread Robert Haas
On Wed, Jul 27, 2022 at 2:13 PM Robert Haas wrote: > On Wed, Jul 27, 2022 at 1:42 PM Tom Lane wrote: > > If there's a magic number, then I'd (a) change that and (b) adjust > > whatever comments led you to think you shouldn't. Bumping catversion > > is a good fallback choice when there's not any

Re: Hash index build performance tweak from sorting

2022-07-27 Thread Tom Lane
Simon Riggs writes: > [ hash_sort_by_hash.v2.patch ] The cfbot says this no longer applies --- probably sideswiped by Korotkov's sorting-related commits last night. regards, tom lane

Re: pgsql: Remove the restriction that the relmap must be 512 bytes.

2022-07-27 Thread Robert Haas
On Wed, Jul 27, 2022 at 1:42 PM Tom Lane wrote: > If there's a magic number, then I'd (a) change that and (b) adjust > whatever comments led you to think you shouldn't. Bumping catversion > is a good fallback choice when there's not any more-proximate version > indicator, but here there is. Mayb

Re: making relfilenodes 56 bits

2022-07-27 Thread Robert Haas
On Wed, Jul 27, 2022 at 12:37 PM Dilip Kumar wrote: > Just realised that this should have been BufferTagsEqual instead of > BufferTagEqual > > I will modify this and send an updated patch tomorrow. I changed it and committed. What was formerly 0002 will need minor rebasing. -- Robert Haas EDB

Re: Use -fvisibility=hidden for shared libraries

2022-07-27 Thread Tom Lane
Andres Freund writes: > On 2022-07-18 00:05:16 -0700, Andres Freund wrote: >> Given that that's just about all compilers we support using configure, >> perhaps >> we should just move that out of the compiler specific section? Doesn't look >> like there's much precedent for that so far... > Here'

Re: pgsql: Remove the restriction that the relmap must be 512 bytes.

2022-07-27 Thread Tom Lane
Robert Haas writes: > On Wed, Jul 27, 2022 at 1:19 PM Alvaro Herrera > wrote: >> Another thing that seems to have happened here is that catversion ought >> to have been touched and wasn't. > Hmm, interesting. I didn't think about bumping catversion because I > didn't change anything in the cata

Re: pgsql: Remove the restriction that the relmap must be 512 bytes.

2022-07-27 Thread Robert Haas
On Wed, Jul 27, 2022 at 1:19 PM Alvaro Herrera wrote: > Another thing that seems to have happened here is that catversion ought > to have been touched and wasn't. Trying to start a cluster that was > initdb'd with the previous code enters an infinite loop that dies each > time with > > 2022-07-27

Re: pgsql: Remove the restriction that the relmap must be 512 bytes.

2022-07-27 Thread Alvaro Herrera
On 2022-Jul-26, Robert Haas wrote: > Remove the restriction that the relmap must be 512 bytes. > > Instead of relying on the ability to atomically overwrite the > entire relmap file in one shot, write a new one and durably > rename it into place. Removing the struct padding and the > calculation

Re: Reducing planning time on tables with many indexes

2022-07-27 Thread Tom Lane
I wrote: > Unfortunately, as things stand today, the planner needs more than the > right to look at the indexes' schemas, because it makes physical accesses > to btree indexes to find out their tree height (and I think there are some > comparable behaviors in other AMs). I've never particularly ca

Re: Reducing planning time on tables with many indexes

2022-07-27 Thread Tom Lane
David Geier writes: > We tracked down the root cause of this slowdown to lock contention in > 'get_relation_info()'. The index lock of every single index of every single > table used in that query is acquired. We attempted a fix by pre-filtering > out all indexes that anyways cannot be used with a

Re: making relfilenodes 56 bits

2022-07-27 Thread Dilip Kumar
On Wed, 27 Jul 2022 at 9:49 PM, Dilip Kumar wrote: > On Wed, Jul 27, 2022 at 12:07 AM Robert Haas > wrote: > > > > On Tue, Jul 26, 2022 at 2:07 AM Dilip Kumar > wrote: > > > I have thought about it while doing so but I am not sure whether it is > > > a good idea or not, because before my change

Re: generic plans and "initial" pruning

2022-07-27 Thread Robert Haas
On Tue, Jul 26, 2022 at 11:01 PM Amit Langote wrote: > Needed to be rebased again, over 2d04277121f this time. 0001 adds es_part_prune_result but does not use it, so maybe the introduction of that field should be deferred until it's needed for something. I wonder whether it's really necessary to

Re: Request for assistance to backport CVE-2022-1552 fixes to 9.6 and 9.4

2022-07-27 Thread Roberto C . Sánchez
Hello pgsql-hackers, Is there anyone willing to review the patches that I prepared? I'd have substatntially more confidence in the patches with a review from an upstream developer who is familiar with the code. Regards, -Roberto On Mon, Jul 04, 2022 at 06:06:58PM -0400, Roberto C. Sánchez wrot

Re: [Commitfest 2022-07] Patch Triage: Waiting on Author

2022-07-27 Thread Robert Haas
On Tue, Jul 26, 2022 at 7:47 PM James Coleman wrote: > These are both mine, and I'd hoped to work on them this CF, but I've > been sufficiently busy that that hasn't happened. > > I'd like to just move these to the next CF. Well, if we mark them returned with feedback now, and you get time to wor

Re: fairywren hung in pg_basebackup tests

2022-07-27 Thread Andrew Dunstan
On 2022-07-27 We 10:58, Tom Lane wrote: > Andrew Dunstan writes: >> The alternative I thought of would be to switch msys to using our >> dirent.c. Probably not too hard, but certainly more work than reverting. > If you ask me, the shortest-path general-purpose fix is to insert > > #if MSYS >

Re: autoprewarm worker failing to load

2022-07-27 Thread Tom Lane
Robins Tharakan writes: > 089480c077056 seems to have broken pg_prewarm. Ugh ... sure would be nice if contrib/pg_prewarm had some regression tests. > Checking pg_prewarm.so the function 'autoprewarm_main' visibility > switched from GLOBAL to LOCAL. Per [1], using PGDLLEXPORT > makes it GLOBAL a

Re: fairywren hung in pg_basebackup tests

2022-07-27 Thread Tom Lane
Andrew Dunstan writes: > The alternative I thought of would be to switch msys to using our > dirent.c. Probably not too hard, but certainly more work than reverting. If you ask me, the shortest-path general-purpose fix is to insert #if MSYS if (pgwin32_is_junction(path)) retu

autoprewarm worker failing to load

2022-07-27 Thread Robins Tharakan
Hi, 089480c077056 seems to have broken pg_prewarm. When pg_prewarm is added to shared_preload_libraries, each new connection results in thousands of errors such as this: 2022-07-27 04:25:14.325 UTC [2903955] LOG: background worker "autoprewarm leader" (PID 2904146) exited with exit code 1 2022-0

Re: fairywren hung in pg_basebackup tests

2022-07-27 Thread Andrew Dunstan
On 2022-07-27 We 10:24, Alvaro Herrera wrote: > On 2022-Jul-27, Andrew Dunstan wrote: > >> The msys dirent.h doesn't have a d_type field at all in a struct dirent. >> I can see a number of ways of dealing with this, but the simplest seems >> to be just to revert 5344723755, at least for msys, alo

Re: fairywren hung in pg_basebackup tests

2022-07-27 Thread Alvaro Herrera
On 2022-Jul-27, Andrew Dunstan wrote: > The msys dirent.h doesn't have a d_type field at all in a struct dirent. > I can see a number of ways of dealing with this, but the simplest seems > to be just to revert 5344723755, at least for msys, along with a comment > about why it's necessary. Hmm, wh

Re: fairywren hung in pg_basebackup tests

2022-07-27 Thread Andrew Dunstan
On 2022-07-27 We 09:47, Andrew Dunstan wrote: > On 2022-07-26 Tu 18:31, Thomas Munro wrote: >> On Tue, Jul 26, 2022 at 4:03 AM Andrew Dunstan wrote: >>> On 2022-07-25 Mo 11:24, Thomas Munro wrote: On Tue, Jul 26, 2022 at 3:08 AM Tom Lane wrote: > Hmm ... an alternative theory is that t

Re: fairywren hung in pg_basebackup tests

2022-07-27 Thread Andrew Dunstan
On 2022-07-26 Tu 18:31, Thomas Munro wrote: > On Tue, Jul 26, 2022 at 4:03 AM Andrew Dunstan wrote: >> On 2022-07-25 Mo 11:24, Thomas Munro wrote: >>> On Tue, Jul 26, 2022 at 3:08 AM Tom Lane wrote: Hmm ... an alternative theory is that the test is fine, and what it's telling us is th

Re: Trying to add more tests to gistbuild.c

2022-07-27 Thread Aleksander Alekseev
Hi Matheus, Many thanks for hacking on increasing the code coverage! I noticed that this patch was stuck in "Needs Review" state for some time and decided to take a look. > With these new tests the coverage went from 45.3% to 85.5%, but I have some > doubts: > - Does this test make sense? > - Wo

Re: Skip partition tuple routing with constant partition key

2022-07-27 Thread Amit Langote
On Wed, Jul 27, 2022 at 7:28 AM David Rowley wrote: > On Sat, 23 Jul 2022 at 01:23, Amit Langote wrote: > > + /* > > +* The Datum has changed. Zero the number of times > > we've > > +* found last_found_datum_index in a row. > > +

Re: [PoC] Reducing planning time on tables with many indexes

2022-07-27 Thread David Geier
Sorry, by accident I sent this one out twice. -- David Geier (ServiceNow) On Wed, Jul 27, 2022 at 2:42 PM David Geier wrote: > Hi hackers, > > We came across a slowdown in planning, where queries use tables with many > indexes. In setups with wide tables it is not uncommon to have easily > 10-1

[PoC] Reducing planning time on tables with many indexes

2022-07-27 Thread David Geier
Hi hackers, We came across a slowdown in planning, where queries use tables with many indexes. In setups with wide tables it is not uncommon to have easily 10-100 indexes on a single table. The slowdown is already visible in serial workloads with just a handful of indexes, but gets drastically amp

RE: Perform streaming logical transactions by background workers and parallel apply

2022-07-27 Thread kuroda.hay...@fujitsu.com
Dear Wang-san, Hi, I'm also interested in the patch and I started to review this. Followings are comments about 0001. 1. terminology In your patch a new worker "apply background worker" has been introduced, but I thought it might be confused because PostgreSQL has already the worker "background

Reducing planning time on tables with many indexes

2022-07-27 Thread David Geier
Hi hackers, We came across a slowdown in planning, where queries use tables with many indexes. In setups with wide tables it is not uncommon to have easily 10-100 indexes on a single table. The slowdown is already visible in serial workloads with just a handful of indexes, but gets drastically amp

Re: Refactor to make use of a common function for GetSubscriptionRelations and GetSubscriptionNotReadyRelations.

2022-07-27 Thread vignesh C
On Wed, Jul 27, 2022 at 4:22 PM Michael Paquier wrote: > > On Wed, Jul 27, 2022 at 08:47:46AM +0530, vignesh C wrote: > > I feel this would be an overkill, I did not make any changes for this. > > Agreed. Using an extra layer of wrappers for that seems a bit too > much, so I have applied your v5

Re: Non-replayable WAL records through overflows and >MaxAllocSize lengths

2022-07-27 Thread Matthias van de Meent
On Wed, 27 Jul 2022 at 11:09, Michael Paquier wrote: > > On Tue, Jul 26, 2022 at 06:58:02PM +0200, Matthias van de Meent wrote: > > - Retained the check in XLogRegisterData, so that we check against > > integer overflows in the registerdata code instead of only an assert > > in XLogRecordAssemble

Re: Introduce wait_for_subscription_sync for TAP tests

2022-07-27 Thread Amit Kapila
On Tue, Jul 26, 2022 at 1:12 PM Masahiko Sawada wrote: > > On Tue, Jul 26, 2022 at 2:01 PM Amit Kapila wrote: > > > > 2. > > +# wait for the replication to catchup if required. > > +if (defined($publisher)) > > +{ > > + croak 'subscription name must be specified' unless defined($subna

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-27 Thread Amit Kapila
On Mon, Jul 25, 2022 at 11:26 AM Masahiko Sawada wrote: > > On Mon, Jul 25, 2022 at 10:45 AM Masahiko Sawada > wrote: > > I've attached the patch for REl15 that I forgot. > I feel the place to remember running xacts information in SnapBuildProcessRunningXacts is not appropriate. Because in case

Re: Refactor to make use of a common function for GetSubscriptionRelations and GetSubscriptionNotReadyRelations.

2022-07-27 Thread Michael Paquier
On Wed, Jul 27, 2022 at 08:47:46AM +0530, vignesh C wrote: > I feel this would be an overkill, I did not make any changes for this. Agreed. Using an extra layer of wrappers for that seems a bit too much, so I have applied your v5 with a slight tweak to the comment. -- Michael signature.asc Desc

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2022-07-27 Thread Melih Mutlu
Hi Amit, I updated the patch in order to prevent the problems that might be caused by using different replication slots for syncing a table. As suggested in previous emails, replication slot names are stored in the catalog. So slot names can be reached later and it is ensured that same replication

RE: Introduce wait_for_subscription_sync for TAP tests

2022-07-27 Thread shiy.f...@fujitsu.com
On Tue, Jul 26, 2022 3:42 PM Masahiko Sawada wrote: > > I've attached an updated patch as well as a patch to remove duplicated > waits in 007_ddl.pl. > Thanks for your patch. Here are some comments. 1. I think some comments need to be changed in the patch. For example: # Also wait for initial

Re: making relfilenodes 56 bits

2022-07-27 Thread vignesh C
On Tue, Jul 26, 2022 at 1:32 PM Dilip Kumar wrote: > > On Thu, Jul 21, 2022 at 9:53 AM Thomas Munro wrote: > > > > On Wed, Jul 20, 2022 at 11:27 PM Dilip Kumar wrote: > > > [v10 patch set] > > > > Hi Dilip, I'm experimenting with these patches and will hopefully have > > more to say soon, but I

Re: [PATCH] Simple code cleanup in tuplesort.c.

2022-07-27 Thread Richard Guo
On Wed, Jul 27, 2022 at 5:10 PM Xing Guo wrote: > The bounded heap sorting status flag is set twice in sort_bounded_heap() > and tuplesort_performsort(). This patch helps remove one of them. > +1. Looks good to me. Thanks Richard

[PATCH] Simple code cleanup in tuplesort.c.

2022-07-27 Thread Xing Guo
Hi hackers, The bounded heap sorting status flag is set twice in sort_bounded_heap() and tuplesort_performsort(). This patch helps remove one of them. Best Regards, Xing diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c index d90a1aebdf..84dc0d07f9 100644 --- a/

Re: Non-replayable WAL records through overflows and >MaxAllocSize lengths

2022-07-27 Thread Michael Paquier
On Tue, Jul 26, 2022 at 06:58:02PM +0200, Matthias van de Meent wrote: > - Retained the check in XLogRegisterData, so that we check against > integer overflows in the registerdata code instead of only an assert > in XLogRecordAssemble where it might be too late. Why? The record has not been inser

Re: [PATCH] Compression dictionaries for JSONB

2022-07-27 Thread Matthias van de Meent
On Wed, 27 Jul 2022 at 09:36, Simon Riggs wrote: > > On Sun, 17 Jul 2022 at 19:15, Nikita Malakhov wrote: > > > For using in special Toaster for JSON datatype compression dictionaries > > seem to be very valuable addition, but now I > > have to agree that this feature in current state is competi

RE: Perform streaming logical transactions by background workers and parallel apply

2022-07-27 Thread houzj.f...@fujitsu.com
On Tuesday, July 26, 2022 5:34 PM Dilip Kumar wrote: > On Tue, Jul 26, 2022 at 2:30 PM Dilip Kumar wrote: > > > > On Fri, Jul 22, 2022 at 8:27 AM wangw.f...@fujitsu.com > > wrote: > > > > > > On Tues, Jul 19, 2022 at 10:29 AM I wrote: > > > > Attach the news patches. > > > > > > Not able to appl

Re: making relfilenodes 56 bits

2022-07-27 Thread Dilip Kumar
On Wed, Jul 27, 2022 at 1:24 PM Ashutosh Sharma wrote: > > Some more comments: Note: Please don't top post. > == > > Shouldn't we retry for the new relfilenumber if > "ShmemVariableCache->nextRelFileNumber > MAX_RELFILENUMBER". There can be a > cases where some of the tables are dropped by the

Re: Perform streaming logical transactions by background workers and parallel apply

2022-07-27 Thread Peter Smith
Here are some review comments for the patch v19-0004: == 1. doc/src/sgml/ref/create_subscription.sgml @@ -244,6 +244,11 @@ CREATE SUBSCRIPTION subscription_nameparallel mode is disregarded when retrying; + instead the transaction will be applied using on + mode.

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

2022-07-27 Thread Amit Langote
On Fri, Jul 22, 2022 at 2:49 PM Amit Langote wrote: > On Fri, Jul 22, 2022 at 1:13 PM David Rowley wrote: > > BTW, I was working on code inside llvm_compile_expr() a few days ago > > and I thought I'd gotten the new evaluation steps I was adding correct > > as it worked fine with jit_above_cost=0

RE: Perform streaming logical transactions by background workers and parallel apply

2022-07-27 Thread houzj.f...@fujitsu.com
On Wednesday, July 27, 2022 1:29 PM Dilip Kumar wrote: > > On Wed, Jul 27, 2022 at 10:06 AM Amit Kapila > wrote: > > > > On Tue, Jul 26, 2022 at 2:30 PM Dilip Kumar wrote: > > > > > > On Fri, Jul 22, 2022 at 8:27 AM wangw.f...@fujitsu.com > > > wrote: > > > > > > > > On Tues, Jul 19, 2022 at 1

Re: making relfilenodes 56 bits

2022-07-27 Thread Ashutosh Sharma
Some more comments: == Shouldn't we retry for the new relfilenumber if "ShmemVariableCache->nextRelFileNumber > MAX_RELFILENUMBER". There can be a cases where some of the tables are dropped by the user and relfilenumber of those tables can be reused for which we would need to find the relfilenumb

Re: [PATCH] Compression dictionaries for JSONB

2022-07-27 Thread Simon Riggs
On Sun, 17 Jul 2022 at 19:15, Nikita Malakhov wrote: > we suggest that as an improvement compression should be put inside the > Toaster as an option, > thus the Toaster could have maximum benefits from knowledge of data internal > structure (and in future use JSON Schema). Very much agreed. >

Re: Slow standby snapshot

2022-07-27 Thread Kyotaro Horiguchi
At Tue, 26 Jul 2022 23:09:16 +0500, Andrey Borodin wrote in > > > > On 20 Jul 2022, at 02:12, Michail Nikolaev > > wrote: > > > >> I've looked into v5. > > Thanks! > > > > Patch is updated accordingly your remarks. > > The patch seems Ready for Committer from my POV. + * is s updated dur