Re: dfmgr additional ABI version fields

2021-11-18 Thread Michael Paquier
On Wed, Oct 13, 2021 at 12:50:38PM -0400, Robert Haas wrote: > I'm not a different vendor, but I do work on different code than you > do, and I like this. Advanced Server accidentally dodges this problem > at present by shipping with a different FUNC_MAX_ARGS value, but this > is much cleaner. I

Re: wait event and archive_command

2021-11-18 Thread Michael Paquier
On Thu, Nov 18, 2021 at 10:04:57AM +0530, Bharath Rupireddy wrote: > Yeah let's not do that. I'm fine with the > wait_event_for_archive_command_v2.patch as is. Switched the patch as RfC, then. -- Michael signature.asc Description: PGP signature

Re: Skipping logical replication transactions on subscriber side

2021-11-18 Thread Greg Nancarrow
On Fri, Nov 19, 2021 at 5:52 PM Amit Kapila wrote: > Why that Assert will hit? We seem to be always passing 'create' as > true so it should create a new entry. I think a similar situation can > happen for functions and it will be probably cleaned in the next > vacuum cycle. > Oops, I missed that

Re: drop tablespace failed when location contains .. on win32

2021-11-18 Thread Michael Paquier
On Wed, Nov 10, 2021 at 05:43:31PM -0500, Tom Lane wrote: > Another thing I happened to notice is that join_path_components > is going out of its way to not generate "foo/./bar", but if > we are fixing canonicalize_path to be able to delete the "./", > that seems like a waste of code now. > > I

A spot of redundant initialization of brin memtuple

2021-11-18 Thread Richard Guo
Happened to notice this when reading around the codes. The BrinMemTuple would be initialized in brin_new_memtuple(), right after being created. So we don't need to initialize it again outside. diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c index

Re: CREATE tab completion

2021-11-18 Thread Michael Paquier
On Fri, Nov 19, 2021 at 04:19:01PM +0900, Ken Kato wrote: > I assume Michael has committed the modified version of the patch. > Therefore, I changed the status to"committed" in Commitfest 2022-01. > https://commitfest.postgresql.org/36/3418/ Thanks, I did not notice that :) -- Michael

Re: Some RELKIND macro refactoring

2021-11-18 Thread Michael Paquier
On Mon, Nov 01, 2021 at 07:28:16AM +0100, Peter Eisentraut wrote: > > Small rebase of this patch set. Regarding 0001, I find the existing code a bit more self-documenting if we keep those checks flagInhAttrs() and guessConstraintInheritance(). So I would rather leave these. I like 0002, which

Re: CREATE tab completion

2021-11-18 Thread Ken Kato
I have fixed (or discarded) that, and the parts for sequences, domains and transforms remained. That looked like good enough on its own, so applied those parts of the patch. -- Michael Thank you very much! I assume Michael has committed the modified version of the patch. Therefore, I changed

Re: Skipping logical replication transactions on subscriber side

2021-11-18 Thread vignesh C
On Fri, Nov 19, 2021 at 12:22 PM Amit Kapila wrote: > > On Fri, Nov 19, 2021 at 11:09 AM vignesh C wrote: > > > > On Fri, Nov 19, 2021 at 9:22 AM Amit Kapila wrote: > > > > > > On Thu, Nov 18, 2021 at 5:10 PM Masahiko Sawada > > > wrote: > > > > > > > > On Thu, Nov 18, 2021 at 5:45 PM

Re: Skipping logical replication transactions on subscriber side

2021-11-18 Thread Amit Kapila
On Fri, Nov 19, 2021 at 11:09 AM vignesh C wrote: > > On Fri, Nov 19, 2021 at 9:22 AM Amit Kapila wrote: > > > > On Thu, Nov 18, 2021 at 5:10 PM Masahiko Sawada > > wrote: > > > > > > On Thu, Nov 18, 2021 at 5:45 PM tanghy.f...@fujitsu.com > > > wrote: > > > > > > > > On Tuesday, November 16,

Re: Skipping logical replication transactions on subscriber side

2021-11-18 Thread Greg Nancarrow
On Fri, Nov 19, 2021 at 4:39 PM vignesh C wrote: > > Since the statistics collector process uses UDP socket, the sequencing > of the messages is not guaranteed. Will there be a problem if > Subscription is dropped and stats collector receives > PGSTAT_MTYPE_SUBSCRIPTIONPURGE first and the

Re: Skipping logical replication transactions on subscriber side

2021-11-18 Thread vignesh C
On Fri, Nov 19, 2021 at 9:22 AM Amit Kapila wrote: > > On Thu, Nov 18, 2021 at 5:10 PM Masahiko Sawada wrote: > > > > On Thu, Nov 18, 2021 at 5:45 PM tanghy.f...@fujitsu.com > > wrote: > > > > > > On Tuesday, November 16, 2021 2:31 PM Masahiko Sawada > > > wrote: > > > > > > > > Right. I've

Re: pg_get_publication_tables() output duplicate relid

2021-11-18 Thread Amit Kapila
On Fri, Nov 19, 2021 at 7:19 AM Amit Langote wrote: > > On Thu, Nov 18, 2021 at 1:53 PM Amit Kapila wrote: > > > > AFAICU, there are actually two problems related to > > pg_publication_tables view that are being discussed: (a) when > > 'publish_via_partition_root' is true then it returns both

Re: row filtering for logical replication

2021-11-18 Thread Peter Smith
On Fri, Nov 19, 2021 at 4:15 PM Greg Nancarrow wrote: > > On Thu, Nov 18, 2021 at 12:33 PM Peter Smith wrote: > > > > PSA new set of v40* patches. > > > > I notice that in the 0001 patch, it adds a "relid" member to the > PublicationRelInfo struct: > > src/include/catalog/pg_publication.h > >

Re: row filtering for logical replication

2021-11-18 Thread Greg Nancarrow
On Thu, Nov 18, 2021 at 12:33 PM Peter Smith wrote: > > PSA new set of v40* patches. > I notice that in the 0001 patch, it adds a "relid" member to the PublicationRelInfo struct: src/include/catalog/pg_publication.h typedef struct PublicationRelInfo { + Oid relid; Relation relation; +

Re: Should we improve "PID XXXX is not a PostgreSQL server process" warning for pg_terminate_backend(<>)?

2021-11-18 Thread Bharath Rupireddy
On Thu, Nov 18, 2021 at 5:01 PM Bharath Rupireddy wrote: > The following is what I made up in my mind after looking at other > existing messages, like [1] and the review comments: > errmsg("cannot send signal to postmaster %d", pid, --> the process > is postmaster but the caller isn't allowed

Re: CREATE tab completion

2021-11-18 Thread Michael Paquier
On Thu, Nov 18, 2021 at 05:20:58PM +0900, Kyotaro Horiguchi wrote: > So we could use COMPLETE_WITH_CS instead so that CREATE SEQUENCE > behavess the same way with the existing behavior of TYPE. Makes sense. Another issue I have noticed with the patch is that it forgets to apply quotes for the

Re: row filtering for logical replication

2021-11-18 Thread Amit Kapila
On Fri, Nov 19, 2021 at 5:35 AM Peter Smith wrote: > > On Mon, Nov 15, 2021 at 9:31 PM Amit Kapila wrote: > > > ... > > > > Few comments on the latest set of patches (v39*) > > === > > 0001* > > 1. > > ObjectAddress > > -publication_add_relation(Oid pubid,

Re: row filtering for logical replication

2021-11-18 Thread Amit Kapila
On Fri, Nov 19, 2021 at 3:16 AM Peter Smith wrote: > > On Thu, Nov 18, 2021 at 4:32 PM Peter Smith wrote: > > > > On Mon, Nov 15, 2021 at 9:31 PM Amit Kapila wrote: > > > > > > On Wed, Nov 10, 2021 at 12:36 PM Peter Smith > > > wrote: > > > > > > > > On Mon, Nov 8, 2021 at 5:53 PM

add missing errdetail for xlogreader allocation failure error

2021-11-18 Thread Bharath Rupireddy
Hi, It seems like some of the XLogReaderAllocate failure check errors are not having errdetail "Failed while allocating a WAL reading processor." but just the errmsg "out of memory". The "out of memory" message without the errdetail is too generic and let's add it for consistency and readability

Re: Skipping logical replication transactions on subscriber side

2021-11-18 Thread Amit Kapila
On Thu, Nov 18, 2021 at 5:10 PM Masahiko Sawada wrote: > > On Thu, Nov 18, 2021 at 5:45 PM tanghy.f...@fujitsu.com > wrote: > > > > On Tuesday, November 16, 2021 2:31 PM Masahiko Sawada > > wrote: > > > > > > Right. I've fixed this issue and attached an updated patch. > > > > > > > > > >

RE: parallel vacuum comments

2021-11-18 Thread houzj.f...@fujitsu.com
On Tues, Nov 16, 2021 1:53 PM Masahiko Sawada wrote: > I've incorporated these comments and attached an updated patch. Thanks for updating the patch. I read the latest patch and have few comments. 1) +/* + * lazy_vacuum_one_index() -- vacuum index relation. ... +IndexBulkDeleteResult *

Re: Teach pg_receivewal to use lz4 compression

2021-11-18 Thread Michael Paquier
On Thu, Nov 18, 2021 at 07:54:37PM +0530, Jeevan Ladhe wrote: > In dir_open_for_write() I observe that we are writing the header > and then calling LZ4F_compressEnd() in case there is an error > while writing the buffer to the file, and the output buffer of > LZ4F_compressEnd() is not written

Re: Skipping logical replication transactions on subscriber side

2021-11-18 Thread Greg Nancarrow
On Tue, Nov 16, 2021 at 5:31 PM Masahiko Sawada wrote: > > Right. I've fixed this issue and attached an updated patch. > A couple of comments for the v23 patch: doc/src/sgml/monitoring.sgml (1) inconsistent decription I think that the following description seems inconsistent with the previous

Re: pg_get_publication_tables() output duplicate relid

2021-11-18 Thread Amit Langote
On Thu, Nov 18, 2021 at 1:53 PM Amit Kapila wrote: > On Wed, Nov 17, 2021 at 11:39 AM Amit Langote wrote: > > What IS problematic is what a subscriber sees in the > > pg_publication_tables view and the problem occurs only in the initial > > sync phase, where the partition is synced duplicatively

Re: row filtering for logical replication

2021-11-18 Thread Peter Smith
On Mon, Nov 15, 2021 at 9:31 PM Amit Kapila wrote: > ... > > Few comments on the latest set of patches (v39*) > === > 0001* > 1. > ObjectAddress > -publication_add_relation(Oid pubid, PublicationRelInfo *targetrel, > +publication_add_relation(Oid pubid,

Re: pg_upgrade parallelism

2021-11-18 Thread Bruce Momjian
On Wed, Nov 17, 2021 at 02:44:52PM -0500, Jaime Casanova wrote: > Hi, > > Currently docs about pg_upgrade says: > > """ > > The --jobs option allows multiple CPU cores to be used > for copying/linking of files and to dump and reload database schemas > in parallel; a good

Re: Showing I/O timings spent reading/writing temp buffers in EXPLAIN

2021-11-18 Thread Melanie Plageman
On Sun, Aug 22, 2021 at 9:47 PM Masahiko Sawada wrote: > > On Thu, Aug 19, 2021 at 10:52 PM Ranier Vilela wrote: > > > > Em qui., 19 de ago. de 2021 às 09:21, Masahiko Sawada > > escreveu: > > > The presentation seems a little confusing, wouldn't it be better? > > > > I/O Timings: shared/local

Re: LogwrtResult contended spinlock

2021-11-18 Thread Alvaro Herrera
Here's a further attempt at this. Sorry it took so long. In this version, I replaced the coupled-in-a-struct representation of Write with two separate global variables. The reason to do this is to cater to Andres' idea to keep them up-to-date separately. Of course, I could kept them together,

Re: Mixing CC and a different CLANG seems like a bad idea

2021-11-18 Thread Tom Lane
Andres Freund writes: > On 2021-11-18 13:39:04 -0500, Tom Lane wrote: >> More, I think (though this ought to be documented in a comment) that >> the policy is to not bother turning on extra -W options in the bitcode >> switches, on the grounds that warning once in the main build is enough. >> I

Re: xlog.c: removing ReadRecPtr and EndRecPtr

2021-11-18 Thread Tom Lane
Robert Haas writes: > On Thu, Nov 18, 2021 at 3:14 PM Tom Lane wrote: >> I'm a little dubious that this test case is valuable enough to >> mess around with a nonstandard postmaster startup protocol, though. > The problem that I have with the present situation is that the test > coverage of

Re: row filtering for logical replication

2021-11-18 Thread Peter Smith
On Thu, Nov 18, 2021 at 4:32 PM Peter Smith wrote: > > On Mon, Nov 15, 2021 at 9:31 PM Amit Kapila wrote: > > > > On Wed, Nov 10, 2021 at 12:36 PM Peter Smith wrote: > > > > > > On Mon, Nov 8, 2021 at 5:53 PM houzj.f...@fujitsu.com > > > wrote: > > > > > > > > 3) v37-0005 > > > > > > > > - no

Re: Mixing CC and a different CLANG seems like a bad idea

2021-11-18 Thread Andres Freund
Hi, On 2021-11-18 16:13:50 -0500, Robert Haas wrote: > On Thu, Nov 18, 2021 at 3:43 PM Andres Freund wrote: > > Hi, > > > > Greetings, > > > > Andres Freund > > Greetings to you too, Andres. :-) Oops I sent the email that I copied text from, rather than the one I wanted to send... Greetings,

Re: Mixing CC and a different CLANG seems like a bad idea

2021-11-18 Thread Andres Freund
Hi, On 2021-11-18 13:39:04 -0500, Tom Lane wrote: > After studying configure's list more closely, that doesn't seem like > a great plan either. There's a lot of idiosyncrasy in the tests, > such as things that only apply to C or to C++. Yea. It seems doable, but not really worth it for now. >

Re: Time to drop plpython2?

2021-11-18 Thread Andrew Dunstan
On 11/16/21 11:26, Andrew Dunstan wrote: > > My other machine with an old python instance is bowerbird. It has python > 3.4 installed but not used, alongside 2.7 which is udsed. I will install > the latest and see if that can be made to work. > > bowerbird is now building with python 3.10

Re: terminate called after throwing an instance of 'std::bad_alloc' (llvmjit)

2021-11-18 Thread Justin Pryzby
On Wed, Nov 10, 2021 at 09:56:44AM -0600, Justin Pryzby wrote: > Thread starting here: > https://www.postgresql.org/message-id/20201001021609.GC8476%40telsasoft.com > > On Fri, Dec 18, 2020 at 05:56:07PM -0600, Justin Pryzby wrote: > > I'm 99% sure the "bad_alloc" is from LLVM. It happened

Re: Mixing CC and a different CLANG seems like a bad idea

2021-11-18 Thread Robert Haas
On Thu, Nov 18, 2021 at 3:43 PM Andres Freund wrote: > Hi, > > Greetings, > > Andres Freund Greetings to you too, Andres. :-) -- Robert Haas EDB: http://www.enterprisedb.com

Re: xlog.c: removing ReadRecPtr and EndRecPtr

2021-11-18 Thread Robert Haas
On Thu, Nov 18, 2021 at 3:14 PM Tom Lane wrote: > Robert Haas writes: > > There's a second place where the patch needs to wait for something > > also, and that one I've crudely kludged with sleep(10). If anybody > > around here who is good at figuring out how to write clever TAP tests > > can

Re: [RFC] building postgres with meson

2021-11-18 Thread Thomas Munro
On Tue, Nov 16, 2021 at 8:23 AM Andres Freund wrote: > On 2021-11-15 14:11:25 -0500, Tom Lane wrote: > > (In fact, unless somebody renews fossa/husky's > > icc license, the three xlc animals will be an outright majority of > > them, because wrasse and anole are the only other active animals with

Re: Mixing CC and a different CLANG seems like a bad idea

2021-11-18 Thread Andres Freund
Hi, On 2021-11-18 12:43:15 -0500, Tom Lane wrote: > Yeah. I'm inclined to think we ought to just bite the bullet and fold > CLANG/CLANGXX into the main list of compiler switch probes, so that we > check every interesting one four times. That sounds fairly horrid, > but as long as you are using

Re: Should rename "startup process" to something else?

2021-11-18 Thread Tom Lane
Alvaro Herrera writes: > If we change the name, and I support the idea that we do, I think a > good name would be "wal replay". I think "recovery" is not great > precisely because in a standby there is likely no crash that we're > recovering from. Fair point. > The word "replay" is at odds

Re: xlog.c: removing ReadRecPtr and EndRecPtr

2021-11-18 Thread Tom Lane
Robert Haas writes: > There's a second place where the patch needs to wait for something > also, and that one I've crudely kludged with sleep(10). If anybody > around here who is good at figuring out how to write clever TAP tests > can tell me how to fix this test to be non-stupid, I will happily

Re: xlog.c: removing ReadRecPtr and EndRecPtr

2021-11-18 Thread Robert Haas
On Thu, Nov 18, 2021 at 2:21 PM Alvaro Herrera wrote: > Would it work to start postmaster directly instad of using pg_ctl, and > then rely on (say) pg_isready? I *think* that pg_isready would also fail, because the documentation says "pg_isready returns 0 to the shell if the server is accepting

Re: xlog.c: removing ReadRecPtr and EndRecPtr

2021-11-18 Thread Alvaro Herrera
On 2021-Nov-18, Robert Haas wrote: > Unfortunately, this test case isn't remotely committable as-is, and I > don't know how to make it so. The main problem is that, although you > can start up a server with nothing in pg_wal, no restore_command, and > no archive command, pg_ctl will not believe

Re: xlog.c: removing ReadRecPtr and EndRecPtr

2021-11-18 Thread Robert Haas
On Thu, Nov 18, 2021 at 7:30 AM Amul Sul wrote: > Somehow with and without patch I am getting the same log. Try applying the attached 0001-dubious-test-cast.patch for you and see if that fails. It does for me. If so, then try applying 0002-fix-the-bug.patch and see if that makes it pass.

Re: Should rename "startup process" to something else?

2021-11-18 Thread Alvaro Herrera
On 2021-Nov-18, Tom Lane wrote: > Justin Pryzby writes: > > On Thu, Nov 18, 2021 at 12:24:14PM -0500, Tom Lane wrote: > >> Personally I think making a glossary entry that explains what the > >> process does would be a better plan than renaming it. > > > Since d3014fff4: > >

Re: Should rename "startup process" to something else?

2021-11-18 Thread Robert Haas
On Thu, Nov 18, 2021 at 12:24 PM Tom Lane wrote: > Being hackers ourselves, I'm not sure we're qualified to opine on > that. I cannot say that I've noticed any questions about it on > the mailing lists, though. What I've noticed when supporting EDB customer is that if, say, there's a problem

Re: Non-superuser subscription owners

2021-11-18 Thread Jeff Davis
On Wed, 2021-11-17 at 16:17 -0800, Mark Dilger wrote: > Some of what I perceive as the screwiness of your argument I must > admin is not your fault. The properties of subscriptions are defined > in ways that don't make sense to me. It would be far more sensible > if connection strings were

Re: Mixing CC and a different CLANG seems like a bad idea

2021-11-18 Thread Tom Lane
I wrote: > Yeah. I'm inclined to think we ought to just bite the bullet and fold > CLANG/CLANGXX into the main list of compiler switch probes, so that we > check every interesting one four times. After studying configure's list more closely, that doesn't seem like a great plan either. There's a

Re: Non-superuser subscription owners

2021-11-18 Thread Jeff Davis
On Wed, 2021-11-17 at 16:17 -0800, Mark Dilger wrote: > You must choose a single role you want the subscription to run > under. I think that was the source of a lot of my confusion: Your patches are creating the notion of a "run as" user by assigning ownership-that-isn't-really-ownership. I

Re: make tuplestore helper function

2021-11-18 Thread Justin Pryzby
On Thu, Nov 18, 2021 at 12:59:03PM -0500, Melanie Plageman wrote: > On Mon, Nov 8, 2021 at 3:13 PM Justin Pryzby wrote: > > On Mon, Nov 08, 2021 at 02:52:28PM -0500, Melanie Plageman wrote: > > > On Tue, Nov 2, 2021 at 4:23 PM Justin Pryzby wrote: > > > > > > > > Several places have a

Re: Should rename "startup process" to something else?

2021-11-18 Thread Tom Lane
Justin Pryzby writes: > On Thu, Nov 18, 2021 at 12:24:14PM -0500, Tom Lane wrote: >> Personally I think making a glossary entry that explains what the >> process does would be a better plan than renaming it. > Since d3014fff4: >

Re: make tuplestore helper function

2021-11-18 Thread Melanie Plageman
On Mon, Nov 8, 2021 at 3:13 PM Justin Pryzby wrote: > > On Mon, Nov 08, 2021 at 02:52:28PM -0500, Melanie Plageman wrote: > > On Tue, Nov 2, 2021 at 4:23 PM Justin Pryzby wrote: > > > > > > Several places have a conditional value for the first argument > > > (randomAccess), > > > but your patch

Re: Should rename "startup process" to something else?

2021-11-18 Thread Justin Pryzby
On Thu, Nov 18, 2021 at 12:24:14PM -0500, Tom Lane wrote: > Robert Haas writes: > > On Thu, Nov 18, 2021 at 11:05 AM Tom Lane wrote: > >> Yeah, given current usage it would be better to call it the "recovery > >> process". However, I'm feeling dubious that it's worth the cost to > >> change.

Re: Should rename "startup process" to something else?

2021-11-18 Thread Thomas Munro
On Tue, Nov 16, 2021 at 3:33 AM Rushabh Lathia wrote: > I think a better name for the process may be “recovery” +1

Re: Mixing CC and a different CLANG seems like a bad idea

2021-11-18 Thread Tom Lane
Andres Freund writes: > On 2021-11-18 11:56:59 -0500, Tom Lane wrote: >> Why did we not simply insist that if you want to use --with-llvm, the >> selected compiler must be clang? I cannot see any benefit of mix-and-match >> here. > It also just seems architecturally wrong: People pressed for

Re: Mixing CC and a different CLANG seems like a bad idea

2021-11-18 Thread Andres Freund
Hi, On 2021-11-18 11:56:59 -0500, Tom Lane wrote: > I noticed that, a week after Michael pushed 9ff47ea41 to silence > -Wcompound-token-split-by-macro warnings, buildfarm member sidewinder > is still spewing them. Investigation shows that it's building with > > configure: using compiler=cc (nb4

Re: Mixing CC and a different CLANG seems like a bad idea

2021-11-18 Thread Tom Lane
=?UTF-8?Q?Mikael_Kjellstr=c3=b6m?= writes: > Hm, actually it's: > CC => "ccache cc", > CXX => "ccache c++", > CLANG => "ccache clang", Right. > want me to change it to: > CC => "ccache clang", > CXX => "ccache c++", > CLANG => "ccache clang", What I actually think is we should get rid of the

Re: Mixing CC and a different CLANG seems like a bad idea

2021-11-18 Thread Mikael Kjellström
On 2021-11-18 17:56, Tom Lane wrote: I noticed that, a week after Michael pushed 9ff47ea41 to silence -Wcompound-token-split-by-macro warnings, buildfarm member sidewinder is still spewing them. Investigation shows that it's building with configure: using compiler=cc (nb4 20200810) 7.5.0

Re: Should rename "startup process" to something else?

2021-11-18 Thread Tom Lane
Robert Haas writes: > On Thu, Nov 18, 2021 at 11:05 AM Tom Lane wrote: >> Yeah, given current usage it would be better to call it the "recovery >> process". However, I'm feeling dubious that it's worth the cost to >> change. The "startup" name is embedded in a lot of places, I think, >> and

Re: Should rename "startup process" to something else?

2021-11-18 Thread Robert Haas
On Thu, Nov 18, 2021 at 11:05 AM Tom Lane wrote: > Yeah, given current usage it would be better to call it the "recovery > process". However, I'm feeling dubious that it's worth the cost to > change. The "startup" name is embedded in a lot of places, I think, > and people are used to it. I

Mixing CC and a different CLANG seems like a bad idea

2021-11-18 Thread Tom Lane
I noticed that, a week after Michael pushed 9ff47ea41 to silence -Wcompound-token-split-by-macro warnings, buildfarm member sidewinder is still spewing them. Investigation shows that it's building with configure: using compiler=cc (nb4 20200810) 7.5.0 configure: using CLANG=ccache clang and the

Re: Printing backtrace of postgres processes

2021-11-18 Thread Justin Pryzby
On Wed, Nov 17, 2021 at 08:12:44PM +0530, vignesh C wrote: > Attached v14 patch has the fixes for the same. Thanks for updating the patch. I cleaned up the docs and comments. I think this could be nearly "Ready". If you like the changes in my "fixup" patch (0002 and 0004), you should be able

Re: Should rename "startup process" to something else?

2021-11-18 Thread Tom Lane
Robert Haas writes: > That's true, but those tasks are very brief. Nobody's going to get too > confused by a "recovery" process that shows up for a few milliseconds > when there's no recovery to be done. Having a "startup" process that > sticks around forever on a standy, though, actually is

Re: Non-superuser subscription owners

2021-11-18 Thread Mark Dilger
> On Nov 18, 2021, at 3:37 AM, Amit Kapila wrote: > >> I have rethought my prior analysis. The problem in the previous patch was >> that the subscription apply workers did not check for a change in ownership >> the way they checked for other changes, instead only picking up the new >>

Re: Non-superuser subscription owners

2021-11-18 Thread Mark Dilger
> On Nov 18, 2021, at 2:50 AM, Amit Kapila wrote: > >> I gave that a slight amount of thought during the design of this patch, but >> didn't think we could refuse to revoke superuser on such a basis, and didn't >> see what we should do with the subscription other than have it continue to

RE: Failed transaction statistics to measure the logical replication progress

2021-11-18 Thread osumi.takami...@fujitsu.com
On Thursday, November 18, 2021 12:26 PM Amit Kapila wrote: > BTW, I think the way you are computing error_count in > pgstat_recv_subworker_error() doesn't seem correct to me because it will > accumulate the counter/bytes for the same error again and again. > You might want to update these

Re: XLogReadRecord() error in XlogReadTwoPhaseData()

2021-11-18 Thread Tom Lane
Andrey Borodin writes: > Let's add more tests that check survival of 2PC through crash recovery? We do > now only one restart. Maybe it worth to do 4 or 8? That seems a little premature when we can't explain the failure we have. Also, buildfarm cycles aren't free.

RE: Failed transaction statistics to measure the logical replication progress

2021-11-18 Thread osumi.takami...@fujitsu.com
On Thursday, November 18, 2021 8:35 PM vignesh C wrote: > On Tue, Nov 16, 2021 at 6:04 PM osumi.takami...@fujitsu.com > wrote: > > > > On Monday, November 15, 2021 9:14 PM I wrote: > > > I've conducted some update for this. > > > (The rebased part is only C code and checked by pgindent) > > I'll

Re: Teach pg_receivewal to use lz4 compression

2021-11-18 Thread Jeevan Ladhe
In dir_open_for_write() I observe that we are writing the header and then calling LZ4F_compressEnd() in case there is an error while writing the buffer to the file, and the output buffer of LZ4F_compressEnd() is not written anywhere. Why should this be necessary? To flush the contents of the

Re: Should rename "startup process" to something else?

2021-11-18 Thread Robert Haas
On Thu, Nov 18, 2021 at 5:34 AM Bharath Rupireddy wrote: > The startup process main function looks to be to do some > initialization required for recovery and do the recovery, exit if it > is a crash recovery or stay there if it is a standby recovery. Apart > from these I'm not sure it does any

[RFC] ASOF Join

2021-11-18 Thread Alexander Kuzmenkov
Hi hackers, There was some interest in implementing ASOF joins in Postgres, see e.g. this prototype patch by Konstantin Knizhnik: https://www.postgresql.org/message-id/flat/bc494762-26bd-b100-e1f9-a97901ddad57%40postgrespro.ru I't like to discuss the possible ways of implementation, if there is

Re: Propose a new hook for mutating the query bounds

2021-11-18 Thread Tomas Vondra
On 11/18/21 10:59, Xiaozhe Yao wrote: Hi, Thanks for the previous feedbacks! > The way the hook is used seems pretty inconvenient, though. I see the problem, and I agree. I looked into how other hooks work, and I am wondering if it looks ok if we: pass a pointer to the hook, and let the

Re: Skipping logical replication transactions on subscriber side

2021-11-18 Thread Masahiko Sawada
On Wed, Nov 17, 2021 at 7:46 PM vignesh C wrote: > > On Tue, Nov 16, 2021 at 12:01 PM Masahiko Sawada > wrote: > > > > On Mon, Nov 15, 2021 at 11:43 PM vignesh C wrote: > > > > > > On Mon, Nov 15, 2021 at 2:48 PM Masahiko Sawada > > > wrote: > > > > > > > > On Mon, Nov 15, 2021 at 4:49 PM

Re: Skipping logical replication transactions on subscriber side

2021-11-18 Thread Masahiko Sawada
On Wed, Nov 17, 2021 at 12:43 PM houzj.f...@fujitsu.com wrote: > > On Tues, Nov 16, 2021 2:31 PM Masahiko Sawada wrote: > > Right. I've fixed this issue and attached an updated patch. > > Hi, > > Thanks for updating the patch. > Here are few comments. Thank you for the comments! > > 1) > > +

Re: Allow CURRENT_ROLE in GRANTED BY

2021-11-18 Thread Daniel Gustafsson
> On 18 Nov 2021, at 14:41, Peter Eisentraut > wrote: > > On 16.11.21 15:27, Daniel Gustafsson wrote: On 16 Nov 2021, at 15:04, Daniel Gustafsson wrote: >>> ..or should the attached small diff be applied to fix it? >> Actually it shouldn't, I realized when hitting Send that it was the

Re: Allow CURRENT_ROLE in GRANTED BY

2021-11-18 Thread Peter Eisentraut
On 16.11.21 15:27, Daniel Gustafsson wrote: On 16 Nov 2021, at 15:04, Daniel Gustafsson wrote: ..or should the attached small diff be applied to fix it? Actually it shouldn't, I realized when hitting Send that it was the wrong version. The attached is the proposed diff. This appears to

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2021-11-18 Thread Bharath Rupireddy
On Thu, Oct 7, 2021 at 10:43 AM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > > Looking at the proposed API from the initial email, I like that there's > > both stats functionality and WAL record inspection functionality > > (similar to pg_waldump). I like that there's the

Re: XLogReadRecord() error in XlogReadTwoPhaseData()

2021-11-18 Thread Andrey Borodin
> 18 нояб. 2021 г., в 12:05, Noah Misch написал(а): > > What else might help? Let's add more tests that check survival of 2PC through crash recovery? We do now only one restart. Maybe it worth to do 4 or 8? Best regards, Andrey Borodin.

Re: row filtering for logical replication

2021-11-18 Thread Amit Kapila
On Thu, Nov 18, 2021 at 11:02 AM Peter Smith wrote: > > On Mon, Nov 15, 2021 at 9:31 PM Amit Kapila wrote: > > > > 5. Why do you need a separate variable rowfilter_valid to indicate > > whether a valid row filter exists? Why exprstate is not sufficient? > > Can you update comments to indicate

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2021-11-18 Thread Etsuro Fujita
On Thu, Nov 18, 2021 at 1:09 PM Fujii Masao wrote: > On 2021/11/16 18:55, Etsuro Fujita wrote: > > Sorry, my explanation was not enough, but I don’t think this is always > > true. Let me explain using an example: > > > > create server loopback foreign data wrapper postgres_fdw options > >

RE: [Proposal] Add foreign-server health checks infrastructure

2021-11-18 Thread kuroda.hay...@fujitsu.com
Dear Kato-san, Thank you for your interest! > > I also want you to review the postgres_fdw part, > > but I think it should not be attached because cfbot cannot understand > > such a dependency > > and will throw build error. Do you know how to deal with them in this > > case? > > I don't know

Re: Should rename "startup process" to something else?

2021-11-18 Thread Amul Sul
On Thu, Nov 18, 2021 at 6:06 PM Andrey Borodin wrote: > > > > > 15 нояб. 2021 г., в 19:32, Rushabh Lathia > > написал(а): > > > > Open for suggestions and thoughts. > > > How about walapplier ? > Similar to walsender, walreciver.. > Or maybe walreplayer ? Regards, Amul

Re: Should rename "startup process" to something else?

2021-11-18 Thread Andrey Borodin
> 15 нояб. 2021 г., в 19:32, Rushabh Lathia > написал(а): > > Open for suggestions and thoughts. How about walapplier ? Similar to walsender, walreciver.. Best regards, Andrey Borodin.

Re: xlog.c: removing ReadRecPtr and EndRecPtr

2021-11-18 Thread Amul Sul
On Thu, Nov 18, 2021 at 3:31 AM Robert Haas wrote: > > I spent a lot of time trying to figure out why xlog.c has global > variables ReadRecPtr and EndRecPtr instead of just relying on the > eponymous structure members inside the XLogReaderState. I concluded > that the values are the same at most

Re: Skipping logical replication transactions on subscriber side

2021-11-18 Thread Masahiko Sawada
On Thu, Nov 18, 2021 at 5:45 PM tanghy.f...@fujitsu.com wrote: > > On Tuesday, November 16, 2021 2:31 PM Masahiko Sawada > wrote: > > > > Right. I've fixed this issue and attached an updated patch. > > > > > > Thanks for your patch. > > I read the discussion about stats entries for table sync

Re: Non-superuser subscription owners

2021-11-18 Thread Amit Kapila
On Thu, Nov 4, 2021 at 1:20 AM Mark Dilger wrote: > > > On Nov 1, 2021, at 10:58 AM, Mark Dilger > > wrote: > > > > ALTER SUBSCRIPTION..[ENABLE | DISABLE] do not synchronously start or stop > > subscription workers. The ALTER command updates the catalog's subenabled > > field, but workers

Re: Failed transaction statistics to measure the logical replication progress

2021-11-18 Thread vignesh C
On Tue, Nov 16, 2021 at 6:04 PM osumi.takami...@fujitsu.com wrote: > > On Monday, November 15, 2021 9:14 PM I wrote: > > I've conducted some update for this. > > (The rebased part is only C code and checked by pgindent) > I'll update my patches since a new skip xid patch > has been shared in [1].

Re: Should we improve "PID XXXX is not a PostgreSQL server process" warning for pg_terminate_backend(<>)?

2021-11-18 Thread Bharath Rupireddy
On Thu, Nov 18, 2021 at 12:30 AM Euler Taveira wrote: > > On Mon, Nov 15, 2021, at 4:27 AM, Bharath Rupireddy wrote: > > As there is some interest shown in this thread at [1], I'm attaching a > new v3 patch here. Please review it. > > I took a look at this patch. I have a few comments. Thanks a

Re: Non-superuser subscription owners

2021-11-18 Thread Amit Kapila
On Wed, Nov 17, 2021 at 11:56 PM Mark Dilger wrote: > > > On Nov 17, 2021, at 9:33 AM, Jeff Davis wrote: > > > > > This would not address the weirdness of the existing code where a > > superuser loses their superuser privileges but still owns a > > subscription. But perhaps we can solve that a

Re: row filtering for logical replication

2021-11-18 Thread Greg Nancarrow
On Thu, Nov 18, 2021 at 12:33 PM Peter Smith wrote: > > PSA new set of v40* patches. > Thanks for the patch updates. A couple of comments so far: (1) compilation warning WIth the patches applied, there's a single compilation warning when Postgres is built: pgoutput.c: In function

Re: Should rename "startup process" to something else?

2021-11-18 Thread Bharath Rupireddy
On Mon, Nov 15, 2021 at 8:03 PM Rushabh Lathia wrote: > Robert and I wondered whether we would like to rename the startup > > process. The reason for doing this is that the current name doesn't > > make any sense, as in the stand-by mode replay loop as the main > > loop, the startup process

Re: Propose a new hook for mutating the query bounds

2021-11-18 Thread Xiaozhe Yao
Hi, Thanks for the previous feedbacks! > The way the hook is used seems pretty inconvenient, though. I see the problem, and I agree. I looked into how other hooks work, and I am wondering if it looks ok if we: pass a pointer to the hook, and let the hook check if there is any information

RE: Skipping logical replication transactions on subscriber side

2021-11-18 Thread tanghy.f...@fujitsu.com
On Tuesday, November 16, 2021 2:31 PM Masahiko Sawada wrote: > > Right. I've fixed this issue and attached an updated patch. > > Thanks for your patch. I read the discussion about stats entries for table sync worker[1], the statistics are retained after table sync worker finished its jobs

Re: pg_waldump stucks with options --follow or -f and --stats or -z

2021-11-18 Thread Bharath Rupireddy
On Thu, Nov 18, 2021 at 1:51 PM Michael Paquier wrote: > > On Thu, Nov 18, 2021 at 01:32:08PM +0530, Bharath Rupireddy wrote: > > Hm. So, the pg_waldump can have handlers for SIGINT, SIGTERM, SIGQUIT > > and then it should emit the computed stats in those handlers the > > comobinations -

Re: pg_waldump stucks with options --follow or -f and --stats or -z

2021-11-18 Thread Michael Paquier
On Thu, Nov 18, 2021 at 01:32:08PM +0530, Bharath Rupireddy wrote: > Hm. So, the pg_waldump can have handlers for SIGINT, SIGTERM, SIGQUIT > and then it should emit the computed stats in those handlers the > comobinations - "-s/-f/-z" and "-s/-e/-f/-z". I'm okay with this > behaviour. Michael

Re: CREATE tab completion

2021-11-18 Thread Kyotaro Horiguchi
At Thu, 18 Nov 2021 17:17:25 +0900, Michael Paquier wrote in > On Thu, Nov 18, 2021 at 04:25:30PM +0900, Ken Kato wrote: > > For this part, I did the following: > > + else if (TailMatches("CREATE", "SEQUENCE", MatchAny, "AS") || > > +TailMatches("CREATE", "TEMP|TEMPORARY",

Re: CREATE tab completion

2021-11-18 Thread Michael Paquier
On Thu, Nov 18, 2021 at 04:25:30PM +0900, Ken Kato wrote: > For this part, I did the following: > + else if (TailMatches("CREATE", "SEQUENCE", MatchAny, "AS") || > + TailMatches("CREATE", "TEMP|TEMPORARY", "SEQUENCE", > MatchAny, "AS")) > +

Re: pg_waldump stucks with options --follow or -f and --stats or -z

2021-11-18 Thread Bharath Rupireddy
On Thu, Nov 18, 2021 at 12:05 AM Alvaro Herrera wrote: > > On 2021-Nov-17, Bharath Rupireddy wrote: > > > On Wed, Nov 17, 2021 at 7:49 AM Michael Paquier wrote: > > > > > > At the same time, we could also just let things as they are. --follow > > > and --stats being specified together is what