Re: Avoid streaming the transaction which are skipped (in corner cases)

2022-11-25 Thread Amit Kapila
On Fri, Nov 25, 2022 at 5:38 PM Amit Kapila wrote: > > On Fri, Nov 25, 2022 at 1:35 PM Dilip Kumar wrote: > > > > During DecodeCommit() for skipping a transaction we use ReadRecPtr to > > check whether to skip this transaction or not. Whereas in > > ReorderBufferCanStartStreaming() we use

Re: Avoid streaming the transaction which are skipped (in corner cases)

2022-11-25 Thread Dilip Kumar
On Fri, Nov 25, 2022 at 4:04 PM Ashutosh Bapat wrote: > > Excellent catch. We were looking at this code last week and wondered > the purpose of this abort. Probably we should have some macro or > function to decided whether to skip a transaction based on log record. > That will avoid using

Re: Collation version tracking for macOS

2022-11-25 Thread Thomas Munro
On Thu, Nov 24, 2022 at 5:48 PM Thomas Munro wrote: > On Thu, Nov 24, 2022 at 3:07 PM Jeff Davis wrote: > > I'd vote for 1 on the grounds that it's easier to document and > > understand a single collation version, which comes straight from > > ucol_getVersion(). This approach makes it a separate

Re: [DOCS] Stats views and functions not in order?

2022-11-25 Thread David G. Johnston
On Wed, Nov 23, 2022 at 1:36 AM Peter Smith wrote: > On Thu, Nov 17, 2022 at 8:46 AM David G. Johnston > wrote: > > > Also, make it so each view ends up being its own separate page. > > > > I did not do this. AFAIK those views of chapter 54 get rendered to > separate pages only because they are

Re: Small miscellaneous fixes

2022-11-25 Thread Michael Paquier
On Fri, Nov 25, 2022 at 01:15:40PM +0100, Peter Eisentraut wrote: > Is this something you want to follow up on, since you were involved in that > patch? Is the redundant assignment simply to be deleted, or do you want to > check the original patch again for context? Most of the changes of this

Re: Patch: Global Unique Index

2022-11-25 Thread David Zhang
Hi Bruce, Thank you for helping review the patches in such detail. On 2022-11-25 9:48 a.m., Bruce Momjian wrote: Looking at the patch, I am unclear how the the patch prevents concurrent duplicate value insertion during the partitioned index checking. I am actually not sure how that can be

Re: MSVC vs Perl

2022-11-25 Thread Andres Freund
Hi, On 2022-11-25 18:48:26 -0500, Andrew Dunstan wrote: > I could download the installer from ActiveState, but they want me to > sign up for an account before I do that, which I'm not happy about. And > while I think our use probably comes within their license terms, IANAL > and I'm not dead

Re: MSVC vs Perl

2022-11-25 Thread Andrew Dunstan
On 2022-11-25 Fr 18:48, Andrew Dunstan wrote: > For various reasons (see below) it's preferable to build on Windows with > Strawberry Perl. This works OK if you're building with Msys2, I upgraded > the instance on the machine that runs fairywren and drongo today, and > fairywren seems fine. Not

MSVC vs Perl

2022-11-25 Thread Andrew Dunstan
For various reasons (see below) it's preferable to build on Windows with Strawberry Perl. This works OK if you're building with Msys2, I upgraded the instance on the machine that runs fairywren and drongo today, and fairywren seems fine. Not so drongo, however. First it encountered a build error,

Re: Making autovacuum logs indicate if insert-based threshold was the triggering condition

2022-11-25 Thread Peter Geoghegan
On Sat, Aug 6, 2022 at 1:03 PM Peter Geoghegan wrote: > Attached patch teaches autovacuum.c to pass the information down to > lazyvacuum.c, which includes the information in the autovacuum log. > The approach I've taken is very similar to the existing approach with > anti-wraparound autovacuum.

Re: Introduce a new view for checkpointer related stats

2022-11-25 Thread Andres Freund
Hi, On 2022-11-23 11:39:43 +0530, Bharath Rupireddy wrote: > On Wed, Nov 23, 2022 at 2:23 AM Andres Freund wrote: > > > > On 2022-11-22 18:08:28 +0530, Bharath Rupireddy wrote: > > > > > > CREATE VIEW pg_stat_bgwriter AS > > > SELECT > > > -pg_stat_get_bgwriter_timed_checkpoints()

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2022-11-25 Thread Peter Geoghegan
On Mon, Oct 24, 2022 at 9:00 AM Peter Geoghegan wrote: > Maybe it could be broken out into a separate "autovacuum triggered by: > " line, seen only in the autovacuum log instrumentation (and not in > the similar report output by a manual VACUUM VERBOSE). When we still > end up "escalating" to an

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2022-11-25 Thread Andres Freund
Hi, On 2022-11-22 23:43:29 -0600, Justin Pryzby wrote: > I think there may be a problem/deficiency with hint bits: > > |postgres=# DROP TABLE u2; CREATE TABLE u2 AS SELECT > generate_series(1,99)a; SELECT pg_stat_reset_shared('io'); explain > (analyze,buffers) SELECT * FROM u2; > |... > |

Re: Support logical replication of DDLs

2022-11-25 Thread Zheng Li
Hello, Thanks for the feedback. > I have been following this patch for a long time. > Recently, I started to try to test it. I found several bugs > here and want to give you feedback. > > 1. CREATE TABLE LIKE > I found that this case may be repication incorrectly. >You can run the

Re: [PATCH] Simple code cleanup in tuplesort.c.

2022-11-25 Thread Cary Huang
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: not tested Spec compliant: not tested Documentation:not tested Removing "state->status = TSS_SORTEDINMEM" should be fine as it is

Re: Use fadvise in wal replay

2022-11-25 Thread Pavel Borisov
On Sat, 26 Nov 2022 at 01:10, Pavel Borisov wrote: > > Hi, hackers! > > On Sun, 13 Nov 2022 at 02:02, Andrey Borodin wrote: > > > > On Sun, Aug 7, 2022 at 9:41 AM Andrey Borodin wrote: > > > > > > > Hi everyone. The patch is 16 lines, looks harmless and with proven > > benefits. I'm moving this

Small miscellaneus fixes (Part II)

2022-11-25 Thread Ranier Vilela
Hi. There another assorted fixes to the head branch. 1. Avoid useless pointer increment (src/backend/utils/activity/pgstat_shmem.c) The pointer *p, is used in creation dsa memory, not p + MAXALIGN(pgstat_dsa_init_size()). 2. Discard result unused (src/backend/access/transam/xlogrecovery.c)

Re: Use fadvise in wal replay

2022-11-25 Thread Pavel Borisov
Hi, hackers! On Sun, 13 Nov 2022 at 02:02, Andrey Borodin wrote: > > On Sun, Aug 7, 2022 at 9:41 AM Andrey Borodin wrote: > > > > Hi everyone. The patch is 16 lines, looks harmless and with proven > benefits. I'm moving this into RfC. As I've written up in the thread we can not gain much from

Re: Report roles in pg_upgrade pg_ prefix check

2022-11-25 Thread Bruce Momjian
On Thu, Nov 24, 2022 at 12:31:09PM +0100, Daniel Gustafsson wrote: > Looking at a recent pg_upgrade thread I happened to notice that the check for > roles with a pg_ prefix only reports the error, not the roles it found. Other > similar checks where the user is expected to alter the old cluster

Re: Fix for visibility check on 14.5 fails on tpcc with high concurrency

2022-11-25 Thread Alvaro Herrera
On 2022-Nov-25, Dimos Stamatakis wrote: > So does this mean there is no race condition in this case and that > this error is redundant? No, it means I believe a bug exists but that I haven't spent enough time on it to understand what it is. Please do not top-post.

Rethinking the implementation of ts_headline()

2022-11-25 Thread Tom Lane
After further contemplation of bug #17691 [1], I've concluded that what I did in commit c9b0c678d was largely misguided. For one thing, the new hlCover() algorithm no longer finds shortest-possible cover strings: if your query is "x & y" and the text is like "... x ... x ... y ...", then the

Re: Lockless queue of waiters in LWLock

2022-11-25 Thread Pavel Borisov
Hi, hackers! In the measurements above in the thread, I've been using LIFO wake queue in a primary lockless patch (and it was attached as the previous versions of a patch) and an "inverted wake queue" (in faсt FIFO) as the alternative benchmarking option. I think using the latter is more fair and

Re: Patch: Global Unique Index

2022-11-25 Thread Bruce Momjian
On Mon, Nov 21, 2022 at 12:33:30PM +, Simon Riggs wrote: > On Thu, 17 Nov 2022 at 22:01, Cary Huang wrote: > > > > Patch: Global Unique Index > > Let me start by expressing severe doubt on the usefulness of such a > feature, but also salute your efforts to contribute. > > > In other words,

Re: BUG #17434: CREATE/DROP DATABASE can be executed in the same transaction with other commands

2022-11-25 Thread Tom Lane
Israel Barth Rubio writes: > It would be great if we can back-patch this to all supported versions, > as the issue itself is currently affecting them all. In my mind, this is waiting for Peter to opine on whether it satisfies his concern. I'm also looking for input on whether to reject if

Re: postgres_fdw: batch inserts vs. before row triggers

2022-11-25 Thread Tom Lane
Etsuro Fujita writes: > I have committed the patch. Apologies for not having paid attention to this thread, but ... I don't think the committed patch is acceptable at all, at least not in the back branches, because it creates a severe ABI break. Specifically, by adding a field to ResultRelInfo

Re: checking rd_rules in RelationBuildDesc

2022-11-25 Thread Ted Yu
On Fri, Nov 25, 2022 at 8:17 AM Tom Lane wrote: > Ted Yu writes: > > I wonder if we should check relation->rd_rules after the call > > to RelationBuildRuleLock(). > > That patch is both pointless and wrong. There is some > value in updating relhasrules in the catalog, so that future > relcache

Re: Bug in row_number() optimization

2022-11-25 Thread Tom Lane
Sergey Shinderuk writes: > What about user-defined operators? I created my own <= operator for int8 > which returns true on null input, and put it in a btree operator class. > Admittedly, it's weird that (null <= 1) evaluates to true. But does it > violate the contract of the btree operator

Re: checking rd_rules in RelationBuildDesc

2022-11-25 Thread Tom Lane
Ted Yu writes: > I wonder if we should check relation->rd_rules after the call > to RelationBuildRuleLock(). That patch is both pointless and wrong. There is some value in updating relhasrules in the catalog, so that future relcache loads don't uselessly call RelationBuildRuleLock; but we

Re: Non-decimal integer literals

2022-11-25 Thread Peter Eisentraut
On 24.11.22 10:13, David Rowley wrote: On Thu, 24 Nov 2022 at 21:35, Peter Eisentraut wrote: My code follows the style used for parsing the decimal integers. Keeping that consistent is valuable I think. I think the proposed change makes the code significantly harder to understand. Also, what

Re: Bug in row_number() optimization

2022-11-25 Thread Sergey Shinderuk
On 25.11.2022 15:46, Richard Guo wrote: Considering the 'Filter' is a strict function, marking it as NULL would do.  I think this is why this patch works. What about user-defined operators? I created my own <= operator for int8 which returns true on null input, and put it in a btree operator

checking rd_rules in RelationBuildDesc

2022-11-25 Thread Ted Yu
Hi, (In light of commit 7b2ccc5e03bf16d1e1bbabca25298108c839ec52) In RelationBuildDesc(), we have: if (relation->rd_rel->relhasrules) RelationBuildRuleLock(relation); I wonder if we should check relation->rd_rules after the call to RelationBuildRuleLock(). Your comment

Re: Fix for visibility check on 14.5 fails on tpcc with high concurrency

2022-11-25 Thread Dimos Stamatakis
So does this mean there is no race condition in this case and that this error is redundant? Thanks, Dimos [ServiceNow] From: Alvaro Herrera Date: Thursday, 24. November 2022 at 19:24 To: Dimos Stamatakis Cc: Peter Geoghegan , simon.ri...@enterprisedb.com , pgsql-hackers@lists.postgresql.org

Re: BUG #17434: CREATE/DROP DATABASE can be executed in the same transaction with other commands

2022-11-25 Thread Israel Barth Rubio
> Attached is a draft patch along the lines I speculated about above. > It breaks backwards compatibility in that PreventInTransactionBlock > commands will now be rejected if they're a non-first command in a > pipeline. I think that's okay, and arguably desirable, for HEAD > but I'm pretty

Re: [DOCS] Stats views and functions not in order?

2022-11-25 Thread David G. Johnston
On Fri, Nov 25, 2022 at 5:09 AM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > On 23.11.22 09:36, Peter Smith wrote: > > v6-0005-Cleanup-view-name-hyperlinks-for-Tables-28.1-and-.patch > v6-0006-Remove-all-stats-views-from-the-ToC-of-28.2.patch > > I wasn't sure yet whether these

Re: Bug in MERGE's test for tables with rules

2022-11-25 Thread Dean Rasheed
On Wed, 23 Nov 2022 at 15:32, Tom Lane wrote: > > Not quite sure the added test case is worth the cycles. > No, probably not, for such a trivial change. Pushed to HEAD and 15, without the test. Thanks for looking! Regards, Dean

Re: Bug in row_number() optimization

2022-11-25 Thread Richard Guo
On Fri, Nov 25, 2022 at 11:26 AM David Rowley wrote: > There are two different pass-through modes that the WindowAgg can move > into when it detects that the run condition is no longer true: > > 1) WINDOWAGG_PASSTHROUGH and > 2) WINDOWAGG_PASSTHROUGH_STRICT > > #2 is used when the WindowAgg is

Re: xml2: add test for coverage

2022-11-25 Thread Peter Eisentraut
On 23.08.22 03:38, Dong Wook Lee wrote: I made a small patch for xml2 to improve test coverage. However, there was a problem using the functions below. - xpath_number - xpath_bool - xpath_nodeset - xpath_list Do you have any advice on how to use this function correctly? It would also be good

Re: [PATCH] Add initial xid/mxid/mxoff to initdb

2022-11-25 Thread Peter Eisentraut
On 05.05.22 17:47, Maxim Orlov wrote: During work on 64-bit XID patch [1] we found handy to have initdb options to set initial xid/mxid/mxoff values to arbitrary non default values. It helps test different scenarios: related to wraparound, pg_upgrade from 32-bit XID to 64-bit XID, etc. We

Re: Small miscellaneous fixes

2022-11-25 Thread Peter Eisentraut
On 04.10.22 06:18, Michael Paquier wrote: On Mon, Oct 03, 2022 at 08:05:57AM -0300, Ranier Vilela wrote: Em seg., 3 de out. de 2022 às 05:01, Masahiko Sawada escreveu: On Fri, Sep 30, 2022 at 9:08 AM Ranier Vilela wrote: 1. Avoid useless reassigning var _logsegno

Re: Does pg_rman support PG15?

2022-11-25 Thread Daniel Gustafsson
> On 24 Nov 2022, at 17:31, T Adachi wrote: > To the developers working on pg_rman, are there any plans to support PG15 > and when might pg_rman source be released? > The latest version of pg_rman, V1.3.14, appears to be incompatible with PG15. > (In PG15, pg_start_backup()/pg_stop_backup() have

Re: [DOCS] Stats views and functions not in order?

2022-11-25 Thread Peter Eisentraut
On 23.11.22 09:36, Peter Smith wrote: PSA new patches. Now there are 6 of them. If some of the earlier patches are agreeable can those ones please be committed? (because I think this patch may be susceptible to needing a big rebase if anything in those tables changes). I have committed

Re: Avoid streaming the transaction which are skipped (in corner cases)

2022-11-25 Thread Amit Kapila
On Fri, Nov 25, 2022 at 1:35 PM Dilip Kumar wrote: > > During DecodeCommit() for skipping a transaction we use ReadRecPtr to > check whether to skip this transaction or not. Whereas in > ReorderBufferCanStartStreaming() we use EndRecPtr to check whether to > stream or not. Generally it will not

Re: Avoid distributing new catalog snapshot again for the transaction being decoded.

2022-11-25 Thread Ashutosh Bapat
Hi Hou, Thanks for the patch. With a simple condition, we can eliminate the need to queueing snapshot change in the current transaction and then applying it. Saves some memory and computation. This looks useful. When the queue snapshot change is processed in ReorderBufferProcessTXN(), we call

Re: Avoid LWLockWaitForVar() for currently held WAL insertion lock in WaitXLogInsertionsToFinish()

2022-11-25 Thread Bharath Rupireddy
On Fri, Nov 25, 2022 at 12:16 AM Andres Freund wrote: > > Hi, > > On 2022-11-24 18:13:10 +0530, Bharath Rupireddy wrote: > > With that said, here's a small improvement I can think of, that is, to > > avoid calling LWLockWaitForVar() for the WAL insertion lock the caller > > of

Re: Support logical replication of DDLs

2022-11-25 Thread vignesh C
On Fri, 25 Nov 2022 at 11:47, vignesh C wrote: > > On Sun, 20 Nov 2022 at 09:29, vignesh C wrote: > > > > On Fri, 11 Nov 2022 at 11:03, Peter Smith wrote: > > > > > > On Fri, Nov 11, 2022 at 4:17 PM Peter Smith wrote: > > > > > > > > On Fri, Nov 11, 2022 at 4:09 PM Peter Smith > > > > wrote:

Re: Logical Replication Custom Column Expression

2022-11-25 Thread Stavros Koureas
Yes, if the property is on the subscription side then it should be applied for all the tables that the connected publication is exposing. So if the property is enabled you should be sure that this origin column exists to all of the tables that the publication is exposing... Sure this is the

Re: Avoid streaming the transaction which are skipped (in corner cases)

2022-11-25 Thread Ashutosh Bapat
Excellent catch. We were looking at this code last week and wondered the purpose of this abort. Probably we should have some macro or function to decided whether to skip a transaction based on log record. That will avoid using different values in different places. On Fri, Nov 25, 2022 at 1:35 PM

Re: Minimal logical decoding on standbys

2022-11-25 Thread Drouvot, Bertrand
Hi, On 9/30/22 2:11 PM, Drouvot, Bertrand wrote: Hi, On 7/6/22 3:30 PM, Drouvot, Bertrand wrote: Hi, On 10/28/21 11:07 PM, Andres Freund wrote: Hi, On 2021-10-28 16:24:22 -0400, Robert Haas wrote: On Wed, Oct 27, 2021 at 2:56 AM Drouvot, Bertrand wrote: So you have in mind to check for

Re: PGDOCS - Logical replication GUCs - added some xrefs

2022-11-25 Thread Peter Eisentraut
Your patch moves the description of the subscriber-related configuration parameters from config.sgml to logical-replication.sgml. But config.sgml is supposed to contain *all* configuration parameters. If we're going to start splitting this up and moving things around then we'd need a more

Re: Logical Replication Custom Column Expression

2022-11-25 Thread Ashutosh Bapat
On Wed, Nov 23, 2022 at 4:54 AM Peter Smith wrote: > > On Wed, Nov 23, 2022 at 7:38 AM Stavros Koureas > wrote: > > > > Reading more carefully what you described, I think you are interested in > > getting something you call origin from publishers, probably some metadata > > from the

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-25 Thread John Naylor
On Thu, Nov 24, 2022 at 9:54 PM Masahiko Sawada wrote: > > [v11] There is one more thing that just now occurred to me: In expanding the use of size classes, that makes rebasing and reworking the shared memory piece more work than it should be. That's important because there are still some open

Re: Support logical replication of DDLs

2022-11-25 Thread li jie
Hi Developer, I have been following this patch for a long time. Recently, I started to try to test it. I found several bugs here and want to give you feedback. 1. CREATE TABLE LIKE I found that this case may be repication incorrectly. You can run the following SQL statement: ```

Re: [BUG] FailedAssertion in SnapBuildPurgeOlderTxn

2022-11-25 Thread Maxim Orlov
On Fri, 25 Nov 2022 at 09:40, Amit Kapila wrote: > On Thu, Nov 24, 2022 at 4:43 PM Amit Kapila > wrote: > > > > On Thu, Nov 24, 2022 at 1:48 PM Masahiko Sawada > wrote: > > > > > > On Wed, Nov 23, 2022 at 12:00 PM Amit Kapila > wrote: > > > > > > Agreed not to have a test case for this. > > >

Re: postgres_fdw: batch inserts vs. before row triggers

2022-11-25 Thread Etsuro Fujita
On Thu, Nov 24, 2022 at 8:19 PM Etsuro Fujita wrote: > Here is an updated patch. In the attached, I added an assertion to > ExecInsert(). Also, I tweaked comments and test cases a little bit, > for consistency. Also, I noticed a copy-and-pasteo in a comment in > ExecBatchInsert(), so I fixed

Avoid distributing new catalog snapshot again for the transaction being decoded.

2022-11-25 Thread houzj.f...@fujitsu.com
Hi, When doing some other related work, I noticed that when decoding the COMMIT record via SnapBuildCommitTxn()-> SnapBuildDistributeNewCatalogSnapshot() we will add a new snapshot to all transactions including the one being decoded(just committed one). But since we've already done required

Re: CREATE UNLOGGED TABLE seq faults when debug_discard_caches=1

2022-11-25 Thread Jakub Wartak
Hi hackers, Mr Lane, thank you for backporting this also to version 13. It seems to be occuring in the wild (without debug_discard_caches) for real user too when doing a lot of "CREATE INDEX i ON unlogged_table_truncated_after_crash (x,y)" which sometimes (rarely) results in SIGSEGV11. I've

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-25 Thread Aleksander Alekseev
Hi hackers, > I'm wondering whether the safest way to handle this is by creating a > new TAM called "heap64", so that all storage changes happens there. > Many current users see stability as one of the greatest strengths of > Postgres, so while I very much support this move, I wonder if this >

Avoid streaming the transaction which are skipped (in corner cases)

2022-11-25 Thread Dilip Kumar
During DecodeCommit() for skipping a transaction we use ReadRecPtr to check whether to skip this transaction or not. Whereas in ReorderBufferCanStartStreaming() we use EndRecPtr to check whether to stream or not. Generally it will not create a problem but if the commit record itself is adding

Re: Remove a unused argument from qual_is_pushdown_safe

2022-11-25 Thread Richard Guo
On Fri, Nov 25, 2022 at 2:27 PM Yugo NAGATA wrote: > I found that qual_is_pushdown_safe() has an argument "subquery" > that is not used in the function. This argument has not been > referred to since the commit 964c0d0f80e485dd3a4073e073ddfd9bfdda90b2. > > I think we can remove this if there is

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-25 Thread John Naylor
On Thu, Nov 24, 2022 at 9:54 PM Masahiko Sawada wrote: > > So it seems that there are two candidates of rt_node structure: (1) > all nodes except for node256 are variable-size nodes and use pointer > tagging, and (2) node32 and node128 are variable-sized nodes and do > not use pointer tagging