Re: POC: GROUP BY optimization

2024-01-14 Thread Alexander Korotkov
On Mon, Jan 15, 2024 at 8:42 AM Richard Guo wrote: > On Mon, Jan 15, 2024 at 8:20 AM Alexander Korotkov > wrote: >> >> Thank you for providing the test case relevant for this code change. >> The revised patch incorporating this change is attached. Now the >> patchset looks good to me. I'm

Re: In-placre persistance change of a relation

2024-01-14 Thread Kyotaro Horiguchi
At Tue, 9 Jan 2024 15:07:20 +0530, vignesh C wrote in > CFBot shows compilation issues at [1] with: Thanks! The reason for those errors was that I didn't consider Meson at the time. Additionally, the signature change of reindex_index() caused the build failure. I fixed both issues. While

Re: Add PQsendSyncMessage() to libpq

2024-01-14 Thread Michael Paquier
On Wed, Jan 10, 2024 at 03:40:36PM +0900, Michael Paquier wrote: > Hence, as a whole, wouldn't it be more consistent if the new > PQsendPipelineSync() and the existing PQpipelineSync() call an > internal static routine (PQPipelineSyncInternal?) that can switch > between both modes? Let's just

Re: pgbnech: allow to cancel queries during benchmark

2024-01-14 Thread Tatsuo Ishii
> On Wed, 6 Sep 2023 20:13:34 +0900 > Yugo NAGATA wrote: > >> I attached the updated patch v3. The changes since the previous >> patch includes the following; >> >> I removed the unnecessary condition (&& false) that you >> pointed out in [1]. >> >> The test was rewritten by using IPC::Run

Fix race condition in InvalidatePossiblyObsoleteSlot()

2024-01-14 Thread Bertrand Drouvot
Hi hackers, While working on [1], we discovered (thanks Alexander for the testing) that an conflicting active logical slot on a standby could be "terminated" without leading to an "obsolete" message (see [2]). Indeed, in case of an active slot we proceed in 2 steps in

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-14 Thread Masahiko Sawada
On Thu, Jan 11, 2024 at 10:24 AM Sutou Kouhei wrote: > > Hi, > > In > "Re: Make COPY format extendable: Extract COPY TO format implementations" > on Wed, 10 Jan 2024 16:53:48 +0900, > Masahiko Sawada wrote: > > >> Interesting. But I feel that it introduces another (a bit) > >> tricky

Re: Oom on temp (un-analyzed table caused by JIT) V16.1

2024-01-14 Thread Pavel Stehule
Hi po 15. 1. 2024 v 7:24 odesílatel Kirk Wolak napsal: > Daniel, > You have a commit [1] that MIGHT fix this. > I have a script that recreates the problem, using random data in pg_temp. > And a nested cursor. > > It took me a few days to reduce this from actual code that was > experiencing

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-01-14 Thread Masahiko Sawada
On Mon, Jan 15, 2024 at 8:21 AM Alexander Korotkov wrote: > > On Sun, Jan 14, 2024 at 10:35 PM Masahiko Sawada > wrote: > > Thank you for updating the patch. Here are two comments: > > > > --- > > + if (cstate->opts.save_error_to != COPY_SAVE_ERROR_TO_UNSPECIFIED && > > +

Re: POC: GROUP BY optimization

2024-01-14 Thread Richard Guo
On Mon, Jan 15, 2024 at 8:20 AM Alexander Korotkov wrote: > Thank you for providing the test case relevant for this code change. > The revised patch incorporating this change is attached. Now the > patchset looks good to me. I'm going to push it if there are no > objections. Seems I'm late

Re: Show WAL write and fsync stats in pg_stat_io

2024-01-14 Thread Michael Paquier
On Fri, Jan 12, 2024 at 04:23:26PM +0300, Nazir Bilal Yavuz wrote: > On Thu, 11 Jan 2024 at 17:28, Melanie Plageman > wrote: >> Even if we made a separate view for WAL I/O stats, we would still have >> this issue of variable sized I/O vs block sized I/O and would probably >> end up solving it

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-14 Thread Sutou Kouhei
Hi, If there are no more comments for the current design, I'll start implementing this feature with the following approaches for "Discussing" items: > 3.1 Should we use one function(internal) for COPY TO/FROM > handlers or two function(internal)s (one is for COPY TO > handler and another

Oom on temp (un-analyzed table caused by JIT) V16.1

2024-01-14 Thread Kirk Wolak
Daniel, You have a commit [1] that MIGHT fix this. I have a script that recreates the problem, using random data in pg_temp. And a nested cursor. It took me a few days to reduce this from actual code that was experiencing this. If I turn off JIT, the problem goes away. (if I don't FETCH the

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-14 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Fri, 12 Jan 2024 14:40:41 +0800, Junwang Zhao wrote: >> Could you clarify what should we discuss? We should require >> that COPY TO/FROM handlers should use PostgreSQL's memory >> context for all internal

Commitfest 2024-01 second week update

2024-01-14 Thread vignesh C
Hi, Here's a quick status report after the second week: Status summary: status | w1 | w2 -+---+- Needs review:| 238 | 213 Waiting on Author: |44 | 46 Ready for Committer: |27 | 27

Re: An inefficient query caused by unnecessary PlaceHolderVar

2024-01-14 Thread Richard Guo
Updated this patch over 29f114b6ff, which indicates that we should apply the same rules for PHVs. Thanks Richard v3-0001-Avoid-unnecessary-PlaceHolderVars-for-Vars-PHVs.patch Description: Binary data

Re: BRIN indexes vs. SK_SEARCHARRAY (and preprocessing scan keys)

2024-01-14 Thread vignesh C
On Mon, 15 Jan 2024 at 04:45, Tomas Vondra wrote: > > On 1/14/24 12:18, vignesh C wrote: > > On Fri, 14 Jul 2023 at 20:17, Tomas Vondra > > wrote: > >> > >> On 7/9/23 23:44, Tomas Vondra wrote: > >>> ... > > Yes, my previous message was mostly about backwards compatibility, and > > this

Re: the s_lock_stuck on perform_spin_delay

2024-01-14 Thread Andy Fan
Robert Haas writes: > On Wed, Jan 10, 2024 at 10:17 PM Andy Fan wrote: >> fixed in v2. > > Timing the spinlock wait seems like a separate patch from the new sanity > checks. Yes, a separate patch would be better, so removed it from v4. > I suspect that the new sanity checks should only be

Re: Compile warnings in dbcommands.c building with meson

2024-01-14 Thread jian he
Hi. one more feedback. I tested the original repo setup, but it does not generate a warning on my local setup. meson setup --reconfigure ${BUILD} \ -Dprefix=${PG_PREFIX} \ -Dpgport=5463 \ -Dplpython=enabled \ -Dcassert=true \ -Dtap_tests=enabled \ -Dicu=enabled \ -Ddebug=true \ -Dnls=disabled

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-14 Thread Michael Paquier
On Sun, Jan 14, 2024 at 11:08:39PM -0500, Tom Lane wrote: > Michael Paquier writes: >> While thinking about that, a second idea came into my mind: a >> superuser-settable developer GUC to disable such WAL records to be >> generated within certain areas of the test. This requires a small >>

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-14 Thread Tom Lane
Michael Paquier writes: > While thinking about that, a second idea came into my mind: a > superuser-settable developer GUC to disable such WAL records to be > generated within certain areas of the test. This requires a small > implementation, but nothing really huge, while being portable >

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-14 Thread Michael Paquier
On Fri, Jan 12, 2024 at 01:46:08PM +, Bertrand Drouvot wrote: > 1) Michael's proposal up-thread (means tweak the test with a retry logic, > retrying > things if such a standby snapshot is found). > > 2) Don't report a test error for active slots in case its catalog_xmin > advanced. > > I'd

Re: Documentation to upgrade logical replication cluster

2024-01-14 Thread Peter Smith
Hi Vignesh, here are some review comments for patch v2-0001. == doc/src/sgml/ref/pgupgrade.sgml 1. + +Upgrade logical replication clusters + + + Refer logical replication upgrade section + for details on upgrading logical replication clusters. + + + + This renders

Re: Recovering from detoast-related catcache invalidations

2024-01-14 Thread Xiaoran Wang
This is an interesting idea. Although some catalog tables are not in catcaches, such as pg_depend, when scanning them, if there is any SharedInvalidationMessage, the CatalogSnapshot will be invalidated and recreated ("RelationInvalidatesSnapshotsOnly" in syscache.c) Maybe during the system_scan,

Re: ALTER ROLE documentation improvement

2024-01-14 Thread Nathan Bossart
On Sun, Jan 14, 2024 at 04:17:41PM +0530, vignesh C wrote: > The attached v3 version patch has the changes for the same. LGTM. I'll wait a little while longer for additional feedback, but if none materializes, I'll commit this soon. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: doc: add LITERAL tag to RETURNING

2024-01-14 Thread torikoshia
On 2024-01-12 20:56, Alvaro Herrera wrote: On 2024-Jan-12, Ashutosh Bapat wrote: On Fri, Jan 12, 2024 at 11:27 AM torikoshia wrote: > > RETURNING is usually tagged with appropriate tags, such as , > but not in the 'query' section of COPY. The patch looks good. Good catch, pushed. It

Re: On login trigger: take three

2024-01-14 Thread Alexander Korotkov
Hi, Alexander! On Sat, Jan 13, 2024 at 6:00 PM Alexander Lakhin wrote: > I've discovered one more instability in the event_trigger_login test. > Please look for example at case [1]: > ok 213 + event_trigger 28946 ms > not ok 214 - event_trigger_login

Re: POC: GROUP BY optimization

2024-01-14 Thread Alexander Korotkov
On Sun, Jan 14, 2024 at 2:14 PM Andrei Lepikhov wrote: > On 13/1/2024 22:00, Alexander Korotkov wrote: > > On Sat, Jan 13, 2024 at 11:09 AM Andrei Lepikhov > > wrote: > >> On 11/1/2024 18:30, Alexander Korotkov wrote: > >>> On Tue, Jan 9, 2024 at 1:14 PM Pavel Borisov > >>> wrote: > > Hmm,

Re: Built-in CTYPE provider

2024-01-14 Thread Michael Paquier
On Fri, Jan 12, 2024 at 01:13:04PM -0500, Robert Haas wrote: > On Fri, Jan 12, 2024 at 1:00 PM Daniel Verite wrote: >> ISTM that in general the behavior of old psql vs new server does >> not weight much against choosing optimal catalog changes. > > +1. +1. There is a good amount of effort put

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-01-14 Thread Alexander Korotkov
On Sun, Jan 14, 2024 at 10:35 PM Masahiko Sawada wrote: > Thank you for updating the patch. Here are two comments: > > --- > + if (cstate->opts.save_error_to != COPY_SAVE_ERROR_TO_UNSPECIFIED && > + cstate->num_errors > 0) > + ereport(WARNING, > + errmsg("%zd rows were

Re: Printing backtrace of postgres processes

2024-01-14 Thread Maciek Sakrejda
The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: tested, passed Spec compliant: not tested Documentation:tested, passed I'm not sure if this actually still needs review, but it's

Re: BRIN indexes vs. SK_SEARCHARRAY (and preprocessing scan keys)

2024-01-14 Thread Tomas Vondra
On 1/14/24 12:18, vignesh C wrote: > On Fri, 14 Jul 2023 at 20:17, Tomas Vondra > wrote: >> >> On 7/9/23 23:44, Tomas Vondra wrote: >>> ... > Yes, my previous message was mostly about backwards compatibility, and > this may seem a bit like an argument against it. But that message was

Re: Custom explain options

2024-01-14 Thread Tomas Vondra
On 1/13/24 17:13, Konstantin Knizhnik wrote: > > On 13/01/2024 4:51 pm, Tomas Vondra wrote: >> >> On 1/12/24 20:30, Konstantin Knizhnik wrote: >>> On 12/01/2024 7:03 pm, Tomas Vondra wrote: On 10/21/23 14:16, Konstantin Knizhnik wrote: > Hi hackers, > > EXPLAIN statement has a

Re: plperl and perl 5.38

2024-01-14 Thread Christoph Berg
Re: Andrew Dunstan > > +WARNING: could not determine encoding for locale "C.utf8": codeset is > > "ANSI_X3.4-1968" > > I can't reproduce this on my Ubuntu 22.04 ARM64 instance with perl 5.38.2 > installed via perlbrew, nor on a fresh Debian unstable with it's perl > 5.38.2. In both instances my

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-01-14 Thread Masahiko Sawada
On Sun, Jan 14, 2024 at 10:30 AM Alexander Korotkov wrote: > > Hi! > > I think this is a demanding and long-waited feature. The thread is > pretty long, but mostly it was disputes about how to save the errors. > The present patch includes basic infrastructure and ability to ignore > errors, thus

Re: Recovering from detoast-related catcache invalidations

2024-01-14 Thread Noah Misch
On Fri, Jan 12, 2024 at 03:47:13PM -0500, Tom Lane wrote: > I wrote: > > This is uncomfortably much in bed with the tuple table slot code, > > perhaps, but I don't see a way to do it more cleanly unless we want > > to add some new provisions to that API. Andres, do you have any > > thoughts about

Re: A failure in t/038_save_logical_slots_shutdown.pl

2024-01-14 Thread Bharath Rupireddy
On Sat, Jan 13, 2024 at 4:43 PM Amit Kapila wrote: > > > > The current test tries to ensure that > > > during shutdown after we shutdown walsender and ensures that it sends > > > all the wal records and receipts an ack for the same, there is no > > > other WAL except shutdown_checkpoint.

Re: plperl and perl 5.38

2024-01-14 Thread Andrew Dunstan
On 2024-01-12 Fr 05:14, Christoph Berg wrote: Perl 5.38 has landed in Debian unstable, and plperl doesn't like it: diff -U3 /home/myon/projects/postgresql/pg/postgresql/src/pl/plperl/expected/plperl_elog_1.out

Re: Tidy fill hstv array (src/backend/access/heap/pruneheap.c)

2024-01-14 Thread John Naylor
On Sat, Jan 13, 2024 at 9:36 PM Ranier Vilela wrote: > > Em ter., 9 de jan. de 2024 às 06:31, John Naylor > escreveu: >> This just moves an operation from one place to the other, while >> obliterating the explanatory comment, so I don't see an advantage. > > Well, I think that is precisely the

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-01-14 Thread John Naylor
On Fri, Jan 12, 2024 at 3:49 PM Masahiko Sawada wrote: > > On Thu, Jan 11, 2024 at 9:28 AM Masahiko Sawada wrote: > > So I agree to remove both max_bytes and num_items from the control > > object.Also, as you mentioned, we can remove the tidstore control > > object itself. TidStoreGetHandle()

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-01-14 Thread Cédric Villemain
Hi Palak, there is currently even more interest in your patch as it should help building tests for on-going development around cache/read management/effects. Do you expect to be able to follow-up in the coming future ? Thank you, Cédric On 04/01/2024 00:15, Jim Nasby wrote: On 1/3/24

Re: POC: GROUP BY optimization

2024-01-14 Thread Andrei Lepikhov
On 13/1/2024 22:00, Alexander Korotkov wrote: On Sat, Jan 13, 2024 at 11:09 AM Andrei Lepikhov wrote: On 11/1/2024 18:30, Alexander Korotkov wrote: On Tue, Jan 9, 2024 at 1:14 PM Pavel Borisov wrote: Hmm, I don't see this old code in these patches. Resend 0002-* because of trailing spaces.

Re: Improvements in pg_dump/pg_restore toc format and performances

2024-01-14 Thread vignesh C
On Fri, 10 Nov 2023 at 23:20, Nathan Bossart wrote: > > On Tue, Oct 03, 2023 at 03:17:57PM +0530, vignesh C wrote: > > Few comments: > > Pierre, do you plan to submit a new revision of this patch set for the > November commitfest? If not, the commitfest entry may be marked as >

Re: Document efficient self-joins / UPDATE LIMIT techniques.

2024-01-14 Thread vignesh C
On Tue, 31 Oct 2023 at 23:42, Corey Huinker wrote: >> >> >> I think the SQL statements should end with semicolons. Our SQL examples >> are usually written like that. > > > ok > > >> >> >> Our general style with CTEs seems to be (according to >>

Re: Build versionless .so for Android

2024-01-14 Thread Matthias Kuhn
What I try to do is packaging an app with androiddeployqt which fails with an error: The bundled library lib/libpq.so.5 doesn't end with .so. Android only supports versionless libraries ending with the .so suffix. This error was introduced in response to this issue which contains hints about the

Re: BRIN indexes vs. SK_SEARCHARRAY (and preprocessing scan keys)

2024-01-14 Thread vignesh C
On Fri, 14 Jul 2023 at 20:17, Tomas Vondra wrote: > > On 7/9/23 23:44, Tomas Vondra wrote: > > ... > >>> Yes, my previous message was mostly about backwards compatibility, and > >>> this may seem a bit like an argument against it. But that message was > >>> more a question "If we do this, is it

Re: Add test module for Table Access Method

2024-01-14 Thread vignesh C
On Thu, 28 Sept 2023 at 10:23, Michael Paquier wrote: > > On Sat, Jun 03, 2023 at 07:42:36PM -0400, Fabrízio de Royes Mello wrote: > > So in order to improve things a bit in this area I'm proposing to add a > > test module for Table Access Method similar what we already have for Index > > Access

Re: Add connection active, idle time to pg_stat_activity

2024-01-14 Thread vignesh C
On Wed, 25 Oct 2023 at 19:06, Andrei Zubkov wrote: > > Hi Aleksander, > > On Wed, 2023-10-25 at 16:17 +0300, Aleksander Alekseev wrote: > > On top of that not sure if I see the patch on the November commitfest > > [1]. Please make sure it's there so that cfbot will check the patch. > > Yes, this

Re: Add last_commit_lsn to pg_stat_database

2024-01-14 Thread vignesh C
On Sat, 10 Jun 2023 at 07:57, James Coleman wrote: > > I've previously noted in "Add last commit LSN to > pg_last_committed_xact()" [1] that it's not possible to monitor how > many bytes of WAL behind a logical replication slot is (computing such > is obviously trivial for physical slots) because

Re: There should be a way to use the force flag when restoring databases

2024-01-14 Thread vignesh C
On Wed, 20 Sept 2023 at 17:27, Daniel Gustafsson wrote: > > > On 20 Sep 2023, at 11:24, Peter Eisentraut wrote: > > > > On 06.08.23 21:39, Ahmed Ibrahim wrote: > >> I have addressed the pg version compatibility with the FORCE option in > >> drop. Here is the last version of the patch > > > >

Re: ALTER ROLE documentation improvement

2024-01-14 Thread vignesh C
On Tue, 26 Sept 2023 at 04:38, Nathan Bossart wrote: > > On Fri, Sep 15, 2023 at 02:25:38PM -0700, Yurii Rashkovskii wrote: > > Thank you for the feedback. I've updated the glossary and updated the > > terminology to be consistent. Please see the new patch attached. > > Thanks for the new version