Add progress reporting to pg_verifybackup

2023-01-05 Thread Masahiko Sawada
Hi all, I've attached the simple patch to add the progress reporting option to pg_verifybackup. The progress information is displayed with --progress option only during the checksum verification, which is the most time consuming task. It cannot be used together with --quiet option. Feedback is

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-05 Thread Dilip Kumar
On Fri, Jan 6, 2023 at 12:59 PM Dilip Kumar wrote: > > > > 3. > > > +else if (shmq_res == SHM_MQ_WOULD_BLOCK) > > > +{ > > > +/* Replay the changes from the file, if any. */ > > > +if (pa_has_spooled_message_pending()) > > > +{ > > > +

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-05 Thread Dilip Kumar
On Fri, Jan 6, 2023 at 12:05 PM Amit Kapila wrote: > > > Yeah, I also don't think sending extra eight bytes with stream_start > message is worth it. But it is fine to mention the same in the > comments. Right. > > 2. > > > > + * XXX Additionally, we also stop the worker if the leader apply

Re: Logical replication timeout problem

2023-01-05 Thread Ashutosh Bapat
Hi Wang, Thanks for working on this. One of our customer faced a similar situation when running BDR with PostgreSQL. I tested your patch and it solves the problem. Please find some review comments below On Tue, Nov 8, 2022 at 8:34 AM wangw.f...@fujitsu.com wrote: > > > Attach the patch. > +/*

Re: Rework of collation code, extensibility

2023-01-05 Thread Jeff Davis
On Wed, 2023-01-04 at 22:46 +0100, Peter Eisentraut wrote: > It combines some refactoring that was previously posted with partial > support for multiple ICU libraries with partial support for some new > hooks.  Shouldn't those be three separate threads? Originally they felt more separate to me,

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2023-01-05 Thread Japin Li
On Mon, 19 Dec 2022 at 22:40, Maxim Orlov wrote: > Hi! > > As a result of discussion in the thread [0], Robert Haas proposed to focus > on making SLRU 64 bit, as a first step towards 64 bit XIDs. > Here is the patch set. > > In overall, code of this patch set is based on the existing code from

Re: moving extraUpdatedCols out of RangeTblEntry (into ModifyTable)

2023-01-05 Thread Amit Langote
On Fri, Jan 6, 2023 at 3:33 PM Tom Lane wrote: > Amit Langote writes: > > BTW, you wrote in the commit message: > > (At present it seems that we don't enforce that for partitioning > > either, which is likely wrong to some degree or other; but the case > > clearly needs to be handled

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-05 Thread Amit Kapila
On Fri, Jan 6, 2023 at 11:24 AM Dilip Kumar wrote: > > On Fri, Jan 6, 2023 at 9:37 AM houzj.f...@fujitsu.com > wrote: > > > > On Thursday, January 5, 2023 7:54 PM Dilip Kumar > > wrote: > > Thanks, I have started another thread[1] > > > > Attach the parallel apply patch set here again. I

Re: moving extraUpdatedCols out of RangeTblEntry (into ModifyTable)

2023-01-05 Thread Tom Lane
Amit Langote writes: > BTW, you wrote in the commit message: > (At present it seems that we don't enforce that for partitioning > either, which is likely wrong to some degree or other; but the case > clearly needs to be handled with traditional inheritance.) > Maybe I'm missing

Re: moving extraUpdatedCols out of RangeTblEntry (into ModifyTable)

2023-01-05 Thread Amit Langote
On Fri, Jan 6, 2023 at 12:28 AM Tom Lane wrote: > Amit Langote writes: > > On Thu, Jan 5, 2023 at 4:59 AM Tom Lane wrote: > >> I've not looked into what it'd take to back-patch this. We can't > >> add a field to ResultRelInfo in released branches (cf 4b3e37993), > >> but we might be able to

Re: Moving forward with TDE

2023-01-05 Thread vignesh C
On Fri, 4 Nov 2022 at 03:36, David Christensen wrote: > > > Unless somebody in the community remembers open questions/issues with > > TDE that were never addressed I suggest simply iterating with our > > usual testing/reviewing process. For now I'm going to change the > > status of the CF entry

Re: refactoring relation extension and BufferAlloc(), faster COPY

2023-01-05 Thread vignesh C
On Sat, 29 Oct 2022 at 08:24, Andres Freund wrote: > > The patches here aren't fully polished (as will be evident). But they should > be more than good enough to discuss whether this is a sane direction. The patch does not apply on top of HEAD as in [1], please post a rebased patch: === Applying

Re: Add a new pg_walinspect function to extract FPIs from WAL records

2023-01-05 Thread Bharath Rupireddy
On Thu, Jan 5, 2023 at 6:51 PM Bharath Rupireddy wrote: > > > I'm also wondering if it would make sense to extend the test coverage of it > > (and pg_waldump) to "validate" that both > > extracted images are the same and matches the one modified right after the > > checkpoint. > > > > What do

RE: Force streaming every change in logical decoding

2023-01-05 Thread shiy.f...@fujitsu.com
On Thu, Dec 22, 2022 3:17 PM Masahiko Sawada wrote: > > I think > > instead of adding new tests with this patch, it may be better to > > change some of the existing tests related to streaming to use this > > parameter as that will clearly show one of the purposes of this patch. > > +1. I think

Re: A new strategy for pull-up correlated ANY_SUBLINK

2023-01-05 Thread vignesh C
On Sun, 13 Nov 2022 at 04:15, Tom Lane wrote: > > Andy Fan writes: > > In the past we pull-up the ANY-sublink with 2 steps, the first step is to > > pull up the sublink as a subquery, and the next step is to pull up the > > subquery if it is allowed. The benefits of this method are obvious, > >

Re: making relfilenodes 56 bits

2023-01-05 Thread Dilip Kumar
On Wed, Jan 4, 2023 at 5:45 PM vignesh C wrote: > > On Fri, 21 Oct 2022 at 11:31, Michael Paquier wrote: > > > > On Thu, Sep 29, 2022 at 09:23:38PM -0400, Tom Lane wrote: > > > Hmmm ... I'd tend to do SELECT COUNT(*) FROM. But can't we provide > > > any actual checks on the sanity of the

Re: Generating code for query jumbling through gen_node_support.pl

2023-01-05 Thread vignesh C
On Wed, 7 Dec 2022 at 13:27, Michael Paquier wrote: > > Hi all, > > This thread is a follow-up of the recent discussion about query > jumbling with DDL statements, where the conclusion was that we'd want > to generate all this code automatically for all the nodes: >

Re: Force streaming every change in logical decoding

2023-01-05 Thread vignesh C
On Mon, 26 Dec 2022 at 14:04, Amit Kapila wrote: > > On Sat, Dec 24, 2022 at 3:28 PM Dilip Kumar wrote: > > > > On Sat, Dec 24, 2022 at 8:56 AM Amit Kapila wrote: > > > > > > > > > > > > > OK, I removed the modification in 022_twophase_cascade.pl and combine > > > > > the two patches. > > > >

Re: Add a new pg_walinspect function to extract FPIs from WAL records

2023-01-05 Thread vignesh C
On Thu, 5 Jan 2023 at 18:52, Bharath Rupireddy wrote: > > On Wed, Jan 4, 2023 at 8:19 PM Drouvot, Bertrand > wrote: > > > > I think it makes sense to somehow align the pg_walinspect functions with > > the pg_waldump "features". > > And since [1] added FPI "extraction" then +1 for the proposed

Re: [PATCH] Expand character set for ltree labels

2023-01-05 Thread vignesh C
On Wed, 4 Jan 2023 at 00:27, Garen Torikian wrote: > > Sure. Rebased onto HEAD. > There is one more merge conflict, please post a rebased patch: === Applying patches on top of PostgreSQL commit ID eb5ad4ff05fd382ac98cab60b82f7fd6ce4cfeb8 === === applying patch

Re: TAP output format in pg_regress

2023-01-05 Thread vignesh C
On Tue, 3 Jan 2023 at 16:01, vignesh C wrote: > > On Tue, 29 Nov 2022 at 00:57, Daniel Gustafsson wrote: > > > > > On 28 Nov 2022, at 20:02, Nikolay Shaplov wrote: > > > > > From my reviewer's point of view patch is ready for commit. > > > > > > Thank you for your patience :-) > > > > Thanks

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-05 Thread Dilip Kumar
On Fri, Jan 6, 2023 at 9:37 AM houzj.f...@fujitsu.com wrote: > > On Thursday, January 5, 2023 7:54 PM Dilip Kumar > wrote: > Thanks, I have started another thread[1] > > Attach the parallel apply patch set here again. I didn't change the patch set, > attach it here just to let the CFbot keep

Re: Notify downstream to discard the streamed transaction which was aborted due to crash.

2023-01-05 Thread Dilip Kumar
On Fri, Jan 6, 2023 at 9:25 AM houzj.f...@fujitsu.com wrote: > > Hi, > > When developing another feature, I find an existing bug which was reported > from Dilip[1]. > > Currently, it's possible that we only send a streaming block without sending a > end of stream message(stream abort) when

RE: Notify downstream to discard the streamed transaction which was aborted due to crash.

2023-01-05 Thread houzj.f...@fujitsu.com
On Friday, January 6, 2023 1:15 PM Amit Kapila wrote: > > On Fri, Jan 6, 2023 at 9:25 AM houzj.f...@fujitsu.com > wrote: > > > > > > To fix it, One idea is to send a stream abort message when we are > > cleaning up crashed transaction on publisher(e.g. in > > ReorderBufferAbortOld()). And here

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-05 Thread Nathan Bossart
I found some additional places that should remove the last-start time from the hash table. I've added those in v14. On Fri, Jan 06, 2023 at 10:30:18AM +0530, Amit Kapila wrote: > On Thu, Jan 5, 2023 at 10:49 PM Nathan Bossart > wrote: >> On Thu, Jan 05, 2023 at 11:34:37AM +0530, Amit Kapila

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2023-01-05 Thread Andrey Borodin
On Mon, Dec 19, 2022 at 6:41 AM Maxim Orlov wrote: > > As always, reviews and opinions are very welcome. Hi! I think that 64-bit xids are a very important feature and I want to help advance it. That's why I want to try to understand a patch better. Do I get it right that the proposed v51

Re: Logical replication - schema change not invalidating the relation cache

2023-01-05 Thread vignesh C
On Fri, 6 Jan 2023 at 04:32, Tom Lane wrote: > > vignesh C writes: > > On Thu, 5 Jan 2023 at 03:17, Tom Lane wrote: > >> The bigger picture here though is that in examples such as the one > >> you gave at the top of the thread, it's not very clear to me that > >> there's *any* principled

Re: Notify downstream to discard the streamed transaction which was aborted due to crash.

2023-01-05 Thread Amit Kapila
On Fri, Jan 6, 2023 at 9:25 AM houzj.f...@fujitsu.com wrote: > > > To fix it, One idea is to send a stream abort message when we are cleaning up > crashed transaction on publisher(e.g. in ReorderBufferAbortOld()). And here is > a tiny patch which changes the same. I have confirmed that the bug is

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-05 Thread Amit Kapila
On Thu, Jan 5, 2023 at 10:49 PM Nathan Bossart wrote: > > On Thu, Jan 05, 2023 at 11:34:37AM +0530, Amit Kapila wrote: > > On Thu, Jan 5, 2023 at 10:16 AM Nathan Bossart > > wrote: > >> In v12, I moved the restart for two_phase mode to the end of > >> process_syncing_tables_for_apply() so that

Re: Infinite Interval

2023-01-05 Thread jian he
On Fri, Jan 6, 2023 at 6:54 AM Joseph Koshakow wrote: > Jian, > > I incorporated your changes and updated interval.out and ran > pgindent. Looks like some of the error messages have changed and we > have some issues with parsing "+infinity" after rebasing. > > - Joe > Looks like some of the

Re: Schema variables - new implementation for Postgres 15 (typo)

2023-01-05 Thread Julien Rouhaud
Hi, On Fri, Dec 23, 2022 at 08:38:43AM +0100, Pavel Stehule wrote: > > I am sending an updated patch, fixing the mentioned issue. Big thanks for > testing, and checking. There were multiple reviews in the last weeks which reported some issues, but unless I'm missing something I don't see any

Re: pg_ftruncate hardcodes length=0 but only under windows

2023-01-05 Thread Thomas Munro
On Fri, Jan 6, 2023 at 4:16 PM Justin Pryzby wrote: > - ret = ftruncate(fd, 0); > + ret = ftruncate(fd, length); Oops. Right. Thanks, pushed.

Notify downstream to discard the streamed transaction which was aborted due to crash.

2023-01-05 Thread houzj.f...@fujitsu.com
Hi, When developing another feature, I find an existing bug which was reported from Dilip[1]. Currently, it's possible that we only send a streaming block without sending a end of stream message(stream abort) when decoding and streaming a transaction that was aborted due to crash because we

Re: Minimal logical decoding on standbys

2023-01-05 Thread Andres Freund
Hi, Thomas, there's one point at the bottom wrt ConditionVariables that'd be interesting for you to comment on. On 2023-01-05 16:15:39 -0500, Robert Haas wrote: > On Tue, Jan 3, 2023 at 2:42 AM Drouvot, Bertrand > wrote: > > Please find attached v36, tiny rebase due to 1de58df4fe. > > 0001

pg_ftruncate hardcodes length=0 but only under windows

2023-01-05 Thread Justin Pryzby
57faaf376 added pg_truncate(const char *path, off_t length), but "length" is ignored under WIN32 and the file is unconditionally truncated to 0. There's no live bug, since the only caller passes 0: | src/backend/storage/smgr/md.c: ret = pg_truncate(path, 0); But I guess extension users could

Re: Add index scan progress to pg_stat_progress_vacuum

2023-01-05 Thread Imseih (AWS), Sami
>Similar to above three cases, vacuum can bypass index vacuuming if >there are almost zero TIDs. Should we set indexes_total to 0 in this >case too? If so, I think we can set both indexes_total and >indexes_completed at the beginning of the index vacuuming/cleanup and >reset

Re: Improve tab completion for ALTER FUNCTION/PROCEDURE/ROUTINE

2023-01-05 Thread vignesh C
On Thu, 5 Jan 2023 at 18:22, Dean Rasheed wrote: > > On Tue, 6 Dec 2022 at 19:12, vignesh C wrote: > > > > On Tue, 6 Dec 2022 at 20:42, Melih Mutlu wrote: > > > > > > Also one little suggestion: > > > > > >> + if (ends_with(prev_wd, ')')) > > >> + COMPLETE_WITH(Alter_routine_options, "CALLED ON

Re: Infinite Interval

2023-01-05 Thread Joseph Koshakow
Jian, I incorporated your changes and updated interval.out and ran pgindent. Looks like some of the error messages have changed and we have some issues with parsing "+infinity" after rebasing. - Joe From 4bf672f9079322cffde635dff2078582fca55f09 Mon Sep 17 00:00:00 2001 From: Joseph Koshakow

Re: BUG: Postgres 14 + vacuum_defer_cleanup_age + FOR UPDATE + UPDATE

2023-01-05 Thread Peter Geoghegan
On Thu, Jan 5, 2023 at 3:27 PM Peter Geoghegan wrote: > This particular error message is from the hardening added to Postgres > 15 in commit e7428a99. So it's not surprising that Michail didn't see > the same error on 14. Reproduced this on HEAD locally (no docker), without any difficulty.

Re: fix and document CLUSTER privileges

2023-01-05 Thread Nathan Bossart
Apparently I forgot to run all the tests before posting v4. Here is a new version of the patch that should pass all tests. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com diff --git a/doc/src/sgml/ref/cluster.sgml b/doc/src/sgml/ref/cluster.sgml index 145101e6a5..749b410e16 100644

Re: [PATCH] Enable using llvm jitlink as an alternative llvm jit linker of old Rtdyld.

2023-01-05 Thread Alex Fan
There is discussion in https://github.com/riscv-non-isa/riscv-toolchain-conventions/issues/13 to change the abi default, but not much attention for some time. The consensus seems to be set the abi and extension explicitly. > I recommend proposing a patch for adding such an API to LLVM. I would

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-05 Thread Nathan Bossart
On Thu, Jan 05, 2023 at 09:29:24AM -0800, Nathan Bossart wrote: > On Thu, Jan 05, 2023 at 10:57:58AM +0530, Amit Kapila wrote: >> True, if we want we can use dshash for this. > > I'll look into this. Here is an attempt at using dshash. This is quite a bit cleaner since we don't need garbage

Re: A varint implementation for PG?

2023-01-05 Thread Robert Haas
Andres asked me off-list if I could take another look at this. So here's a bit of review: - The header comment at the top of the file gives some examples of how the encoding works, and then basically says, oh wait, there's also a sign bit at the end, so all those examples are actually wrong.

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-05 Thread David Rowley
On Thu, 5 Jan 2023 at 19:14, Ankit Kumar Pandey wrote: > We are already doing something like I mentioned. > > Consider this example: > > explain SELECT rank() OVER (ORDER BY a), count(*) OVER (ORDER BY a,b) > FROM abcd; > QUERY PLAN >

Re: BUG: Postgres 14 + vacuum_defer_cleanup_age + FOR UPDATE + UPDATE

2023-01-05 Thread Peter Geoghegan
On Thu, Jan 5, 2023 at 1:49 PM Matthias van de Meent wrote: > client 29 script 0 aborted in command 2 query 0: ERROR: heap tid from index > tuple (111,1) points past end of heap page line pointer array at offset 262 > of block 1 in index "something_is_wrong_here_pkey" This particular error

Re: Generate pg_stat_get_xact*() functions with Macros

2023-01-05 Thread Andres Freund
Hi, On 2023-01-05 15:19:54 -0500, Corey Huinker wrote: > It does get me wondering, however, if we reordered the three typedefs to > group like-typed registers together, we could make them an array with the > names becoming defined constant index values (or keeping them via a union), > then the

Re: [PATCH] Enable using llvm jitlink as an alternative llvm jit linker of old Rtdyld.

2023-01-05 Thread Alex Fan
> why should we do this only for riscv? I originally considered riscv because I only have amd64 and riscv hardware for testing. But afaik, JITLink currently supports arm64 and x86-64 (ELF & MacOS), riscv (both 64bit and 32bit, ELF). i386 seems also supported. No support for ppc or mips exist yet.

Re: Logical replication - schema change not invalidating the relation cache

2023-01-05 Thread Tom Lane
vignesh C writes: > On Thu, 5 Jan 2023 at 03:17, Tom Lane wrote: >> The bigger picture here though is that in examples such as the one >> you gave at the top of the thread, it's not very clear to me that >> there's *any* principled behavior. If the connection between publisher >> and subscriber

Re: postgres_fdw: using TABLESAMPLE to collect remote sample

2023-01-05 Thread Tom Lane
Tomas Vondra writes: > There are no callers outside postgresAcquireSampleRowsFunc, so what > about renaming them like this? > deparseAnalyzeTuplesSql > -> deparseAnalyzeInfoSql > postgresCountTuplesForForeignTable > -> postgresGetAnalyzeInfoForForeignTable WFM.

Re: BUG: Postgres 14 + vacuum_defer_cleanup_age + FOR UPDATE + UPDATE

2023-01-05 Thread Matthias van de Meent
On Thu, 5 Jan 2023 at 14:12, Michail Nikolaev wrote: > > Hello, hackers. > > It seems like PG 14 works incorrectly with vacuum_defer_cleanup_age > (or just not cleared rows, not sure) and SELECT FOR UPDATE + UPDATE. > I am not certain, but hot_standby_feedback probably able to cause the > same

Re: postgres_fdw: using TABLESAMPLE to collect remote sample

2023-01-05 Thread Tomas Vondra
On 1/5/23 22:05, Tom Lane wrote: > Tomas Vondra writes: >> The second patch adds the relkind check, so that we only issue >> TABLESAMPLE on valid relation types (tables, matviews). But I'm not sure >> we actually need it - the example presented earlier was foreign table >> pointing to a

Re: Cygwin cleanup

2023-01-05 Thread Thomas Munro
On Fri, Jan 6, 2023 at 1:22 AM Thomas Munro wrote: > https://cirrus-ci.com/task/5142810819559424 [still running at time of writing] > > Gotta run, but I'll check again in the morning to see if that does better... Yes! Two successful runs with all TAP tests so far. So it looks like we can

Re: BUG: Postgres 14 + vacuum_defer_cleanup_age + FOR UPDATE + UPDATE

2023-01-05 Thread Michail Nikolaev
Hello, Andres. I apologize for the direct ping, but I think your snapshot scalability work in PG14 could be related to the issue. The TransactionIdRetreatedBy implementation looks correct... But with txid_current=212195 I see errors like "could not access status of transaction 58643736"... So,

Re: Add tracking of backend memory allocated to pg_stat_activity

2023-01-05 Thread Reid Thompson
On Thu, 2023-01-05 at 14:13 -0600, Justin Pryzby wrote: > > I suggest to close the associated CF entry. Closed with status of Withdrawn. If that is invalid, please advise. > (Also, the people who participated in this thread may want to be > included in the other thread going forward.)

Re: ExecRTCheckPerms() and many prunable partitions (checkAsUser)

2023-01-05 Thread Tom Lane
Justin Pryzby writes: > On Wed, Dec 21, 2022 at 01:44:11PM -0600, Justin Pryzby wrote: >> Alvaro could you comment on this ? I believe Alvaro's on vacation for a few days more. regards, tom lane

Re: Minimal logical decoding on standbys

2023-01-05 Thread Robert Haas
On Tue, Jan 3, 2023 at 2:42 AM Drouvot, Bertrand wrote: > Please find attached v36, tiny rebase due to 1de58df4fe. 0001 looks committable to me now, though we probably shouldn't do that unless we're pretty confident about shipping enough of the rest of this to accomplish something useful. --

Re: ExecRTCheckPerms() and many prunable partitions (checkAsUser)

2023-01-05 Thread Justin Pryzby
On Wed, Dec 21, 2022 at 01:44:11PM -0600, Justin Pryzby wrote: > Alvaro could you comment on this ? I added here so it's not forgotten. https://commitfest.postgresql.org/42/4107/

Re: postgres_fdw: using TABLESAMPLE to collect remote sample

2023-01-05 Thread Tom Lane
Tomas Vondra writes: > The second patch adds the relkind check, so that we only issue > TABLESAMPLE on valid relation types (tables, matviews). But I'm not sure > we actually need it - the example presented earlier was foreign table > pointing to a sequence. But that actually works fine even

Re: postgres_fdw: using TABLESAMPLE to collect remote sample

2023-01-05 Thread Tomas Vondra
Hi, here's two minor postgres_fdw patches, addressing the two issues discussed last week. 1) 0001-Fix-stale-comment-in-postgres_fdw.patch The first one cleans up the comment referencing the sample rate adjustment. While doing that, I realized without the adjustment we should never get

Re: Generate pg_stat_get_xact*() functions with Macros

2023-01-05 Thread Corey Huinker
On Thu, Jan 5, 2023 at 8:50 AM Drouvot, Bertrand < bertranddrouvot...@gmail.com> wrote: > Hi hackers, > > Please find attached a patch proposal to $SUBJECT. > > This is the same kind of work that has been done in 83a1a1b566 and > 8018ffbf58 but this time for the > pg_stat_get_xact*() functions

Re: Add tracking of backend memory allocated to pg_stat_activity

2023-01-05 Thread Justin Pryzby
On Thu, Jan 05, 2023 at 01:58:33PM -0500, Reid Thompson wrote: > On Tue, 2023-01-03 at 16:25 +0530, vignesh C wrote: > > ... > > The patch does not apply on top of HEAD as in [1], please post a > > rebased patch: > >... > > Regards, > > Vignesh > > Per conversation in thread listed below,

Re: fixing CREATEROLE

2023-01-05 Thread Robert Haas
On Tue, Jan 3, 2023 at 3:11 PM Robert Haas wrote: > Committed and back-patched 0001 with fixes for the issues that you pointed > out. > > Here's a trivial rebase of the rest of the patch set. I committed 0001 and 0002 after improving the commit messages a bit. Here's the remaining two patches

Re: Add tracking of backend memory allocated to pg_stat_activity

2023-01-05 Thread Reid Thompson
On Tue, 2023-01-03 at 16:25 +0530, vignesh C wrote: > ... > The patch does not apply on top of HEAD as in [1], please post a > rebased patch: >... > Regards, > Vignesh Per conversation in thread listed below, patches have been submitted to the "Add the ability to limit the amount of memory that

Re: pgsql: Delay commit status checks until freezing executes.

2023-01-05 Thread Peter Geoghegan
On Wed, Jan 4, 2023 at 10:59 PM Amit Kapila wrote: > You are an extremely valuable person for this project and I wish that > you continue working with the same enthusiasm. Thank you, Amit. Knowing that my efforts are appreciated by colleagues does make it easier to persevere. -- Peter Geoghegan

ATTACH PARTITION seems to ignore column generation status

2023-01-05 Thread Tom Lane
This does not seem good: regression=# create table pp (a int, b int) partition by range(a); CREATE TABLE regression=# create table cc (a int generated always as (b+1) stored, b int); CREATE TABLE regression=# alter table pp attach partition cc for values from ('1') to ('10'); ALTER TABLE

Re: Add the ability to limit the amount of memory that can be allocated to backends.

2023-01-05 Thread Reid Thompson
On Tue, 2023-01-03 at 16:22 +0530, vignesh C wrote: > > The patch does not apply on top of HEAD as in [1], please post a > rebased patch: > ... > Regards, > Vignesh > Attached is rebased patch, with some updates related to committed changes. Thanks, Reid -- Reid Thompson Senior Software

Support for dumping extended statistics

2023-01-05 Thread Hari krishna Maddileti
Hi Team, In order to restore dumped extended statistics (stxdndistinct, stxddependencies, stxdmcv) we need to provide input functions to parse pg_distinct/pg_dependency/pg_mcv_list strings. Today we get the ERROR "cannot accept a value of type pg_ndistinct/pg_dependencies/pg_mcv_list" when we

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-05 Thread Matthias van de Meent
On Thu, 5 Jan 2023 at 02:21, I wrote: > > On Tue, 3 Jan 2023 at 21:30, Peter Geoghegan wrote: > > > > Attached is v14. > > [PATCH v14 3/3] Finish removing aggressive mode VACUUM. > > I've not completed a review for this patch - I'll continue on that > tomorrow: This is that. > @@ -2152,10

Re: daitch_mokotoff module

2023-01-05 Thread Alvaro Herrera
On 2023-Jan-05, Dag Lem wrote: > Is there anything else I should do here, to avoid the status being > incorrectly stuck at "Waiting for Author" again. Just mark it Needs Review for now. I'll be back from vacation on Jan 11th and can have a look then (or somebody else can, perhaps.) -- Álvaro

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-05 Thread Nathan Bossart
On Thu, Jan 05, 2023 at 10:57:58AM +0530, Amit Kapila wrote: > True, if we want we can use dshash for this. I'll look into this. > The garbage collection > mechanism used in the patch seems odd to me as that will remove/add > entries to the hash table even when the corresponding subscription is

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-05 Thread Nathan Bossart
On Thu, Jan 05, 2023 at 11:34:37AM +0530, Amit Kapila wrote: > On Thu, Jan 5, 2023 at 10:16 AM Nathan Bossart > wrote: >> In v12, I moved the restart for two_phase mode to the end of >> process_syncing_tables_for_apply() so that we don't need to rely on another >> iteration of the loop. > >

Re: fix and document CLUSTER privileges

2023-01-05 Thread Nathan Bossart
On Thu, Jan 05, 2023 at 02:38:58PM +0100, Gilles Darold wrote: > This is a bit confusing, this commitfest entry patch is also included in an > other commitfest entry [1] into patch v3-0001-fix-maintain-privs.patch with > some additional conditions. > > Committers should be aware that this

Re: Infinite Interval

2023-01-05 Thread Joseph Koshakow
On Thu, Jan 5, 2023 at 5:20 AM jian he wrote: > > > > On Wed, Jan 4, 2023 at 10:13 PM jian he wrote: >> >> >> >> I don't know how to generate an interval.out file. Personally I just write the .out files manually. I think it especially helps as a way to double-check that the results are what you

Re: moving extraUpdatedCols out of RangeTblEntry (into ModifyTable)

2023-01-05 Thread Tom Lane
Amit Langote writes: > On Thu, Jan 5, 2023 at 4:59 AM Tom Lane wrote: >> Here's a draft patch that does it like that. This seems like a win >> for more reasons than just pruning, because I was able to integrate >> the calculation into runtime setup of the expressions, so that we >> aren't doing

Re: Allow tailoring of ICU locales with custom rules

2023-01-05 Thread Peter Eisentraut
Patch needed a rebase; no functionality changes. On 14.12.22 10:26, Peter Eisentraut wrote: This patch exposes the ICU facility to add custom collation rules to a standard collation.  This would allow users to customize any ICU collation to whatever they want.  A very simple example from the

Re: moving extraUpdatedCols out of RangeTblEntry (into ModifyTable)

2023-01-05 Thread Amit Langote
On Thu, Jan 5, 2023 at 4:59 AM Tom Lane wrote: > I wrote: > > After further thought: maybe we should get radical and postpone this > > work all the way to executor startup. The downside of that is having > > to do it over again on each execution of a prepared plan. But the > > upside is that

Re: heapgettup refactoring

2023-01-05 Thread Peter Eisentraut
On 03.01.23 21:39, Melanie Plageman wrote: On 30.11.22 23:34, Melanie Plageman wrote: I have attached a patchset with only the code changes contained in the previous patch 0003. I have broken the refactoring down into many smaller pieces for ease of review. To keep this moving along a bit, I

Generate pg_stat_get_xact*() functions with Macros

2023-01-05 Thread Drouvot, Bertrand
Hi hackers, Please find attached a patch proposal to $SUBJECT. This is the same kind of work that has been done in 83a1a1b566 and 8018ffbf58 but this time for the pg_stat_get_xact*() functions (as suggested by Andres in [1]). The function names remain the same, but some fields have to be

Re: fix and document CLUSTER privileges

2023-01-05 Thread Gilles Darold
Le 05/01/2023 à 06:12, Nathan Bossart a écrit : On Wed, Jan 04, 2023 at 11:27:05PM +0100, Gilles Darold wrote: Got it, this is patch add_cluster_skip_messages.patch . IMHO this patch should be part of this commitfest as it is directly based on this one. You could create a second patch here that

Re: Ignore dropped columns when checking the column list in logical replication

2023-01-05 Thread Amit Kapila
On Wed, Jan 4, 2023 at 12:28 PM shiy.f...@fujitsu.com wrote: > > I tried to fix them in the attached patch. > Don't we need a similar handling for generated columns? We don't send those to the subscriber side, see checks in proto.c. -- With Regards, Amit Kapila.

Re: Simplify standby state machine a bit in WaitForWALToBecomeAvailable()

2023-01-05 Thread Bharath Rupireddy
On Wed, Jan 4, 2023 at 12:03 AM Nathan Bossart wrote: > > On Tue, Jan 03, 2023 at 02:53:10PM +0530, Bharath Rupireddy wrote: > > In summary: > > the flow when the standby is in crash recovery is pg_wal -> [archive > > -> pg_wal -> stream] -> [archive -> pg_wal -> stream] -> [] -> [] ... > > the

Re: Add a new pg_walinspect function to extract FPIs from WAL records

2023-01-05 Thread Bharath Rupireddy
On Wed, Jan 4, 2023 at 8:19 PM Drouvot, Bertrand wrote: > > I think it makes sense to somehow align the pg_walinspect functions with the > pg_waldump "features". > And since [1] added FPI "extraction" then +1 for the proposed patch in this > thread. > > > [1] > >

Re: MERGE ... WHEN NOT MATCHED BY SOURCE

2023-01-05 Thread Dean Rasheed
On Thu, 5 Jan 2023 at 11:03, Alvaro Herrera wrote: > > I haven't read this patch other than superficially; I suppose the > feature it's introducing is an OK one to have as an extension to the > standard. (I hope the community members that are committee members > will propose this extension to

BUG: Postgres 14 + vacuum_defer_cleanup_age + FOR UPDATE + UPDATE

2023-01-05 Thread Michail Nikolaev
Hello, hackers. It seems like PG 14 works incorrectly with vacuum_defer_cleanup_age (or just not cleared rows, not sure) and SELECT FOR UPDATE + UPDATE. I am not certain, but hot_standby_feedback probably able to cause the same issues. Steps to reproduce: 1) Start Postgres like this:

Re: Improve tab completion for ALTER FUNCTION/PROCEDURE/ROUTINE

2023-01-05 Thread Dean Rasheed
On Tue, 6 Dec 2022 at 19:12, vignesh C wrote: > > On Tue, 6 Dec 2022 at 20:42, Melih Mutlu wrote: > > > > Also one little suggestion: > > > >> + if (ends_with(prev_wd, ')')) > >> + COMPLETE_WITH(Alter_routine_options, "CALLED ON NULL INPUT", > >> + "RETURNS NULL ON NULL INPUT", "STRICT",

Re: Using AF_UNIX sockets always for tests on Windows

2023-01-05 Thread Andrew Dunstan
On 2023-01-04 We 07:13, vignesh C wrote: > On Fri, 2 Dec 2022 at 18:08, Andrew Dunstan wrote: >> >> On 2022-12-01 Th 21:10, Andres Freund wrote: >>> Hi, >>> >>> On 2022-12-01 20:56:18 -0500, Tom Lane wrote: Andres Freund writes: > On 2022-12-01 20:30:36 -0500, Tom Lane wrote: >>

Re: Cygwin cleanup

2023-01-05 Thread Thomas Munro
On Wed, Jan 4, 2023 at 3:25 AM Justin Pryzby wrote: > On Tue, Jan 03, 2023 at 05:54:56PM +0530, vignesh C wrote: > > Is there still some work pending for this thread as Andres had > > committed some part, if so, can you post an updated patch for the > > same. > > Thomas, what's your opinion ?

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-01-05 Thread Alexander Korotkov
On Wed, Jan 4, 2023 at 5:05 PM Alexander Korotkov wrote: > On Wed, Jan 4, 2023 at 3:43 PM Pavel Borisov wrote: > > One more update of a patchset to avoid compiler warnings. > > Thank you for your help. I'm going to provide the revised version of > patch with comments and commit message in the

Re: Logical replication - schema change not invalidating the relation cache

2023-01-05 Thread vignesh C
On Thu, 5 Jan 2023 at 03:17, Tom Lane wrote: > > vignesh C writes: > > [ v3-0001-Fix-for-invalidating-logical-replication-relation.patch ] > > (btw, please don't send multiple patch versions with the same number, > it's very confusing.) Since it was just rebasing on top of HEAD, I did not

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-05 Thread Dilip Kumar
On Thu, Jan 5, 2023 at 5:03 PM houzj.f...@fujitsu.com wrote: > > On Thursday, January 5, 2023 4:22 PM Dilip Kumar > wrote: > > > Thanks for reporting the problem. > > After analyzing the behavior, I think it's a bug on publisher side which > is not directly related to parallel apply. > > I

RE: Perform streaming logical transactions by background workers and parallel apply

2023-01-05 Thread houzj.f...@fujitsu.com
On Thursday, January 5, 2023 4:22 PM Dilip Kumar wrote: > > On Thu, Jan 5, 2023 at 9:07 AM houzj.f...@fujitsu.com > wrote: > > > > On Wednesday, January 4, 2023 9:29 PM Dilip Kumar > wrote: > > > > I think this looks good to me. > > > > Thanks for the comments. > > Attach the new version

Re: MERGE ... WHEN NOT MATCHED BY SOURCE

2023-01-05 Thread Alvaro Herrera
I haven't read this patch other than superficially; I suppose the feature it's introducing is an OK one to have as an extension to the standard. (I hope the community members that are committee members will propose this extension to become part of the standard.) On 2023-Jan-02, Dean Rasheed

Re: How to generate the new expected out file.

2023-01-05 Thread Amit Kapila
On Thu, Jan 5, 2023 at 4:12 PM jian he wrote: > Hi. > > I changed the src/test/regress/sql/interval.sql, How can I generate the new > src/test/regress/expected/interval.out file. > You can run the tests and copy the required changes from src/test/regress/output/interval.out to

How to generate the new expected out file.

2023-01-05 Thread jian he
Hi. I changed the src/test/regress/sql/interval.sql, How can I generate the new src/test/regress/expected/interval.out file.

Issue in MERGE with concurrent UPDATE and MERGE

2023-01-05 Thread Shruthi Gowda
Hi, While I was running some isolation tests for MERGE, I noticed one issue when MERGE tries to UPDATE rows that are concurrently updated by another session. Below is the test case for the same. TEST CASE START = DROP TABLE target; DROP

Re: Infinite Interval

2023-01-05 Thread jian he
On Wed, Jan 4, 2023 at 10:13 PM jian he wrote: > > > On Tue, Jan 3, 2023 at 6:14 AM Joseph Koshakow wrote: > >> I have another patch, this one adds validations to operations that >> return intervals and updated error messages. I tried to give all of the >> error messages meaningful text, but

Re: Allow placeholders in ALTER ROLE w/o superuser

2023-01-05 Thread Alexander Korotkov
On Tue, Jan 3, 2023 at 5:41 PM Pavel Borisov wrote: > On Tue, 3 Jan 2023 at 17:28, Justin Pryzby wrote: > > I should've mentioned that the same things should be removed from > > Makefile, too... > > > Thanks, Justin! > Attached is a new patch accordingly. Thank you. I've pushed my version,

Re: Split index and table statistics into different types of stats

2023-01-05 Thread Drouvot, Bertrand
Hi, On 1/5/23 1:27 AM, Andres Freund wrote: Hi, On 2023-01-03 15:19:18 +0100, Drouvot, Bertrand wrote: diff --git a/src/backend/access/common/relation.c b/src/backend/access/common/relation.c index 4017e175e3..fca166a063 100644 --- a/src/backend/access/common/relation.c +++

Re: daitch_mokotoff module

2023-01-05 Thread Dag Lem
Is there anything else I should do here, to avoid the status being incorrectly stuck at "Waiting for Author" again. Best regards Dag Lem

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-05 Thread Dilip Kumar
On Thu, Jan 5, 2023 at 9:07 AM houzj.f...@fujitsu.com wrote: > > On Wednesday, January 4, 2023 9:29 PM Dilip Kumar > wrote: > > I think this looks good to me. > > Thanks for the comments. > Attach the new version patch set which changed the comments as suggested. Thanks for the updated patch,

  1   2   >