Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-24 Thread Kyotaro Horiguchi
At Tue, 24 Jan 2023 11:28:58 +0530, Amit Kapila wrote in > On Tue, Jan 24, 2023 at 6:17 AM Kyotaro Horiguchi > wrote: > > > > IMHO "foo > bar" is not an "option". I think we say "foo and bar are > > mutually exclusive options" but I think don't say "foo = x and bar = y > > are.. options". I

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-24 Thread Amit Kapila
On Tue, Jan 24, 2023 at 12:44 PM Peter Smith wrote: > > On Tue, Jan 24, 2023 at 5:58 PM Amit Kapila wrote: > > > > On Tue, Jan 24, 2023 at 8:15 AM Kyotaro Horiguchi > > wrote: > > > > > > > Attached the updated patch v19. > > > > > > + maybe_delay_apply(TransactionId xid, TimestampTz finish_ts)

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-01-24 Thread Kyotaro Horiguchi
Hello. At Thu, 19 Jan 2023 21:15:34 -0500, Melanie Plageman wrote in > Oh dear-- an extra FlushBuffer() snuck in there somehow. > Removed it in attached v51. > Also, I fixed an issue in my tablespace.sql updates I only looked 0002 and 0004. (Sorry for the random order of the comment..) 0002:

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-01-24 Thread Kyotaro Horiguchi
At Tue, 24 Jan 2023 17:22:03 +0900 (JST), Kyotaro Horiguchi wrote in > +pgstat_count_io_op(IOObject io_object, IOContext io_context, IOOp io_op) > +{ > + Assert(io_object < IOOBJECT_NUM_TYPES); > + Assert(io_context < IOCONTEXT_NUM_TYPES); > + Assert(io_op < IOOP_NUM_TYPES); > +

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-24 Thread Takamichi Osumi (Fujitsu)
On Monday, January 23, 2023 9:06 PM Amit Kapila wrote: > On Sun, Jan 22, 2023 at 6:12 PM Takamichi Osumi (Fujitsu) > wrote: > > > > > > Attached the updated patch v19. > > > > Few comments: > = > 1. > } > + > + > +/* > > Only one empty line is sufficient between different

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-24 Thread Takamichi Osumi (Fujitsu)
On Tuesday, January 24, 2023 3:58 PM Amit Kapila wrote: > > send_feedback(): > > +* If the subscriber side apply is delayed (because of time-delayed > > +* replication) then do not tell the publisher that the received > > latest > > +* LSN is already applied and flushed,

Re: WAL Insertion Lock Improvements (was: Re: Avoid LWLockWaitForVar() for currently held WAL insertion lock in WaitXLogInsertionsToFinish())

2023-01-24 Thread Bharath Rupireddy
On Tue, Dec 6, 2022 at 12:00 AM Andres Freund wrote: > > Hi Thanks for reviewing. > FWIW, I don't see an advantage in 0003. If it allows us to make something else > simpler / faster, cool, but on its own it doesn't seem worthwhile. I've discarded this change. > On 2022-12-02 16:31:58 -0800,

Re: Reduce timing overhead of EXPLAIN ANALYZE using rdtsc?

2023-01-24 Thread David Geier
Hi I think at least some should be converted to just accumulate in an instr_time... I think that's for a later patch though? Yep, at least quite similar. OK. I coded it up in the latest version of the patch. Depending on how low we want to keep the error, I don't think we can: If I set the

Re: Record queryid when auto_explain.log_verbose is on

2023-01-24 Thread torikoshia
On 2023-01-23 09:35, Michael Paquier wrote: On Fri, Jan 20, 2023 at 12:32:58PM +0900, Michael Paquier wrote: FWIW, no objections from here. This maps with EXPLAIN where the query ID is only printed under VERBOSE. While looking at this change, I have been wondering about something.. Isn't the

Re: Improve GetConfigOptionValues function

2023-01-24 Thread Bharath Rupireddy
On Mon, Jan 23, 2023 at 9:51 PM Tom Lane wrote: > > Bharath Rupireddy writes: > > LGTM. I've marked it RfC. > > After looking at this, it seemed to me that the factorization > wasn't quite right after all: specifically, the new function > could be used in several more places if it confines

Re: Schema variables - new implementation for Postgres 15 (typo)

2023-01-24 Thread Dmitry Dolgov
> On Mon, Jan 23, 2023 at 07:09:27PM +0100, Pavel Stehule wrote: > po 23. 1. 2023 v 15:25 odesílatel Dmitry Dolgov <9erthali...@gmail.com> > napsal: > > > > On Sun, Jan 22, 2023 at 07:47:07PM +0100, Pavel Stehule wrote: > > > pá 20. 1. 2023 v 21:35 odesílatel Dmitry Dolgov <9erthali...@gmail.com>

Re: old_snapshot_threshold bottleneck on replica

2023-01-24 Thread Pavel Borisov
Hi, Maxim! On Mon, 23 Jan 2023 at 18:40, Maxim Orlov wrote: > > Hi! > > One of our customers stumble onto a significant performance degradation while > running multiple OLAP-like queries on a replica. > After some investigation, it became clear that the problem is in accessing >

Re: run pgindent on a regular basis / scripted manner

2023-01-24 Thread Jelte Fennema
> One issue on requiring patches to have run pgindent previously is > actually the typedef list. If someone adds a typedef in a commit, they > will see different pgident output in the committed files, and perhaps > others, and the new typedefs might only appear after the commit, causing > later

RE: Perform streaming logical transactions by background workers and parallel apply

2023-01-24 Thread houzj.f...@fujitsu.com
On Tuesday, January 24, 2023 3:19 PM Peter Smith wrote: > > Here are some review comments for v86-0002 > > == > Commit message > > 1. > Use the use the existing developer option logical_replication_mode to test the > parallel apply of large transaction on subscriber. > > ~ > > Typo “Use

RE: Perform streaming logical transactions by background workers and parallel apply

2023-01-24 Thread houzj.f...@fujitsu.com
On Tuesday, January 24, 2023 11:43 AM Peter Smith wrote: > > Here are my review comments for patch v86-0001. Thanks for your comments. > > > == > Commit message > > 2. > Since we may extend the developer option logical_decoding_mode to to test the > parallel apply of large transaction

RE: Perform streaming logical transactions by background workers and parallel apply

2023-01-24 Thread houzj.f...@fujitsu.com
On Monday, January 23, 2023 8:34 PM Kuroda, Hayato wrote: > > Followings are my comments. Thanks for your comments. > > 1. guc_tables.c > > ``` > static const struct config_enum_entry logical_decoding_mode_options[] = { > - {"buffered", LOGICAL_DECODING_MODE_BUFFERED, false}, > -

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-24 Thread Takamichi Osumi (Fujitsu)
On Monday, January 23, 2023 7:45 PM Amit Kapila wrote: > On Mon, Jan 23, 2023 at 1:36 PM Peter Smith > wrote: > > > > Here are my review comments for v19-0001. > > > ... > > > > 5. parse_subscription_options > > > > + /* > > + * The combination of parallel streaming mode and min_apply_delay is >

RE: Perform streaming logical transactions by background workers and parallel apply

2023-01-24 Thread Hayato Kuroda (Fujitsu)
Dear Hou, > Sorry, the patch set was somehow attached twice. Here is the correct new > version > patch set which addressed all comments so far. Thank you for updating the patch! I confirmed that All of my comments are addressed. One comment: In this test the rollback-prepared seems not to be

RE: Perform streaming logical transactions by background workers and parallel apply

2023-01-24 Thread houzj.f...@fujitsu.com
On Tuesday, January 24, 2023 8:47 PM Hou, Zhijie wrote: > > On Tuesday, January 24, 2023 3:19 PM Peter Smith > wrote: > > > > Here are some review comments for v86-0002 > > Sorry, the patch set was somehow attached twice. Here is the correct new version patch set which addressed all comments so

Re: Monotonic WindowFunc support for ntile(), percent_rank() and cume_dist()

2023-01-24 Thread Melanie Plageman
On Tue, Jan 24, 2023 at 02:00:33PM +1300, David Rowley wrote: > Thanks for having a look at this. > > On Tue, 24 Jan 2023 at 13:26, Melanie Plageman > wrote: > > > Since all three cases are exactly the same code, maybe you could > > macro-ize it and add a single comment? > > Hmm, I kinda like

Re: CREATEROLE users vs. role properties

2023-01-24 Thread tushar
On Mon, Jan 23, 2023 at 10:28 PM Robert Haas wrote: > > In previous releases, you needed to have CREATEROLE in order to be > able to perform user management functions. In master, you still need > CREATEROLE, and you also need ADMIN OPTION on the role. In this > scenario, only t1 meets those

Re: Minimal logical decoding on standbys

2023-01-24 Thread Drouvot, Bertrand
Hi, On 1/24/23 6:20 AM, Drouvot, Bertrand wrote: Hi, On 1/24/23 1:46 AM, Andres Freund wrote: Hi, On 2023-01-19 10:43:27 +0100, Drouvot, Bertrand wrote: Sure, so with: 1) hot_standby_feedback set to off on the standby 2) create 2 logical replication slots on the standby and activate one 3)

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-24 Thread Takamichi Osumi (Fujitsu)
On Tuesday, January 24, 2023 8:32 AM Euler Taveira wrote: > Good to know that you keep improving this patch. I have a few suggestions that > were easier to provide a patch on top of your latest patch than to provide an > inline suggestions. Thanks for your review ! We basically adopted your

Re: Making Vars outer-join aware

2023-01-24 Thread Hans Buschmann
Hello Tom I just noticed your new efforts in this area. I wanted to recurr to my old thread [1] considering constant propagation of quals. You gave an elaborated explanation at that time, but my knowledge was/is not yet sufficient to reveil the technical details. In our application the

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-24 Thread Takamichi Osumi (Fujitsu)
Hi, On Tuesday, January 24, 2023 5:52 PM Amit Kapila wrote: > On Tue, Jan 24, 2023 at 12:44 PM Peter Smith > wrote: > > > > On Tue, Jan 24, 2023 at 5:58 PM Amit Kapila > wrote: > > > > > > On Tue, Jan 24, 2023 at 8:15 AM Kyotaro Horiguchi > > > wrote: > > > > > > > > > Attached the updated

to_hex() for negative inputs

2023-01-24 Thread Dean Rasheed
I only recently realised that to_hex() converts its input to unsigned before converting it to hex (something that's not mentioned in the docs): to_hex(-1) -> I think that's something that some users might find surprising, especially if they were expecting to be able to use it to

Re: to_hex() for negative inputs

2023-01-24 Thread Aleksander Alekseev
Hi Dean, > I only recently realised that to_hex() converts its input to unsigned > before converting it to hex (something that's not mentioned in the > docs): Technically the documentation is accurate [1]: """ Converts the number to its equivalent hexadecimal representation. """ But I agree

Re: Non-superuser subscription owners

2023-01-24 Thread Robert Haas
On Mon, Jan 23, 2023 at 7:24 PM Jacob Champion wrote: > > The password requirement just *barely* prevents that attack from > > working, almost, maybe, while at the same time managing to block > > things that people want to do for totally legitimate reasons. But > > IMHO, the real problem is that

Re: run pgindent on a regular basis / scripted manner

2023-01-24 Thread Tom Lane
Jelte Fennema writes: >> One issue on requiring patches to have run pgindent previously is >> actually the typedef list. If someone adds a typedef in a commit, they >> will see different pgident output in the committed files, and perhaps >> others, and the new typedefs might only appear after

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-24 Thread Hayato Kuroda (Fujitsu)
Dear Amit, Horiguchi-san, > > > > send_feedback(): > > +* If the subscriber side apply is delayed (because of time-delayed > > +* replication) then do not tell the publisher that the received > > latest > > +* LSN is already applied and flushed, otherwise, it leads to the

Re: Non-decimal integer literals

2023-01-24 Thread Dean Rasheed
On Tue, 24 Jan 2023 at 00:47, Ranier Vilela wrote: > > On 13.01.23 11:01, Dean Rasheed wrote: > > So I'm feeling quite good about the end result -- I set out hoping not > > to make performance noticeably worse, but ended up making it > > significantly better. > Hi Dean, thanks for your work. > >

Re: Non-decimal integer literals

2023-01-24 Thread Ranier Vilela
Em ter., 24 de jan. de 2023 às 07:24, Dean Rasheed escreveu: > On Tue, 24 Jan 2023 at 00:47, Ranier Vilela wrote: > > > > On 13.01.23 11:01, Dean Rasheed wrote: > > > So I'm feeling quite good about the end result -- I set out hoping not > > > to make performance noticeably worse, but ended up

Re: Exclusion constraints on partitioned tables

2023-01-24 Thread Ronan Dunklau
Le vendredi 16 décembre 2022, 06:11:49 CET Paul Jungwirth a écrit : > On 12/15/22 16:12, Tom Lane wrote: > >> This patch also requires the matching constraint columns to use equality > >> comparisons (`(foo WITH =)`), so it is really equivalent to the existing > >> b-tree rule. > > > > That's not

Re: Minimal logical decoding on standbys

2023-01-24 Thread Drouvot, Bertrand
Hi, On 1/24/23 12:21 AM, Melanie Plageman wrote: I'm new to this thread and subject, but I had a few basic thoughts about the first patch in the set. Thanks for looking at it! On Mon, Jan 23, 2023 at 12:03:35PM +0100, Drouvot, Bertrand wrote: Please find V42 attached. From

Re: Schema variables - new implementation for Postgres 15 (typo)

2023-01-24 Thread Pavel Stehule
> > > I can be wrong, but from these numbers I don't think so these sync cycles > > should to contain CHECK_FOR_INTERRUPTS > > > > What do you think? > > Well, there is always possibility someone will create more variables > than any arbitrary limit we have tested for. But I see your point and >

Re: Test failures of 100_bugs.pl

2023-01-24 Thread Amit Kapila
On Tue, Jan 24, 2023 at 8:53 AM Andres Freund wrote: > > cfbot, the buildfarm and locally I have seen 100_bugs.pl fail > occasionally. Just rarely enough that I never got around to looking into it > for real. > ... > > We see t2 added to the publication: > 2023-01-24 00:57:30.099 UTC

Re: Reduce timing overhead of EXPLAIN ANALYZE using rdtsc?

2023-01-24 Thread David Geier
Hi, On 1/23/23 18:41, Andres Freund wrote: If we add it, it probably shouldn't depend on TIMING, but on SUMMARY. Regression test queries showing EXPLAIN ANALYZE output all do something like EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF) the SUMMARY OFF gets rid of the "top-level"

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-24 Thread Takamichi Osumi (Fujitsu)
On Monday, January 23, 2023 5:07 PM Peter Smith wrote: > Here are my review comments for v19-0001. Thanks for your review ! > > == > Commit message > > 1. > The combination of parallel streaming mode and min_apply_delay is not > allowed. The subscriber in the parallel streaming mode

Re: CREATEROLE users vs. role properties

2023-01-24 Thread Robert Haas
On Tue, Jan 24, 2023 at 9:07 AM tushar wrote: > right, Neha/I have tested with different scenarios using > createdb/replication/bypassrls and other > privileges properties on the role. also checked pg_dumpall/pg_basebackup and > everything looks fine. Thanks. I have committed the patch. --

Re: Unicode grapheme clusters

2023-01-24 Thread Isaac Morland
On Tue, 24 Jan 2023 at 11:40, Greg Stark wrote: > > At the end of the day Unicode kind of assumes a variable-width display > where the rendering is handled by something that has access to the > actual font metrics. So anything trying to line things up in columns > in a way that works with any

Re: cutting down the TODO list thread

2023-01-24 Thread Bruce Momjian
On Tue, Jan 24, 2023 at 10:46:34AM +0700, John Naylor wrote: > > On Wed, Jan 18, 2023 at 3:13 AM Bruce Momjian wrote: > > > I think we risk overloading people with too many words above, and they > > will not read it fully.  Can it be simplified?  I wonder if some of this > > belows in the

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-24 Thread Tom Lane
Nathan Bossart writes: > IMO ideally there should be a DSA_HANDLE_INVALID and DSHASH_HANDLE_INVALID > for use with dsa_handle and dshash_table_handle, respectively. But your > patch does seem like an improvement. Yeah, particularly given that dsa.h says /* * The handle for a dsa_area is

Re: run pgindent on a regular basis / scripted manner

2023-01-24 Thread Vladimir Sitnikov
Andres> Of course, but I somehow feel a change of formatting should be reviewable to Andres> at least some degree One way of reviewing the formatting changes is to compare the compiled binaries. If the binaries before and after formatting are the same, then there's a high chance the behaviour is

Re: to_hex() for negative inputs

2023-01-24 Thread Dean Rasheed
On Tue, 24 Jan 2023 at 13:43, Aleksander Alekseev wrote: > > Adding extra arguments for something the user can implement > (him/her)self doesn't seem to be a great idea. With this approach we > may end up with hundreds of arguments one day. > I don't see how a couple of extra arguments will

Re: Making Vars outer-join aware

2023-01-24 Thread Tom Lane
Hans Buschmann writes: > I just noticed your new efforts in this area. > I wanted to recurr to my old thread [1] considering constant propagation of > quals. > [1] https://www.postgresql.org/message-id/1571413123735.26...@nidsa.net Yeah, this patch series is not yet quite up to the point of

Re: old_snapshot_threshold bottleneck on replica

2023-01-24 Thread Robert Haas
On Mon, Jan 23, 2023 at 9:40 AM Maxim Orlov wrote: > One of our customers stumble onto a significant performance degradation while > running multiple OLAP-like queries on a replica. > After some investigation, it became clear that the problem is in accessing > old_snapshot_threshold parameter.

Re: run pgindent on a regular basis / scripted manner

2023-01-24 Thread Tom Lane
Jelte Fennema writes: > Sounds like this conflict could be handled fairly easily by > having a local git hook rerunning pgindent whenever > you rebase a commit: > 1. if you changed typedefs.list the hook would format all files > 2. if you didn't it only formats the files that you changed I think

plpython vs _POSIX_C_SOURCE

2023-01-24 Thread Andres Freund
Hi, A recent commit of mine [1] broke compilation of plpython on AIX [2]. But my commit turns out to only be very tangentially related - it only causes a failure because it references clock_gettime() in an inline function instead of a macro and, as it turns out, plpython currently breaks

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-24 Thread Ankit Kumar Pandey
I think more benchmarking is required so we can figure out if this is a corner case or a common case I did some more benchmarks: #1. AIM: Pushdown column whose size is very high create table test(a int, b int, c text); insert into test select a,b,c from generate_series(1,1000)a,

Re: pgindent vs variable declaration across multiple lines

2023-01-24 Thread Tom Lane
Andrew Dunstan writes: > On 2023-01-22 Su 17:34, Tom Lane wrote: >> I've also attached a diff >> representing the delta between what current pg_bsd_indent wants to do >> to HEAD and what this would do. All the changes it wants to make look >> good, although I can't say whether there are other

Re: run pgindent on a regular basis / scripted manner

2023-01-24 Thread Jelte Fennema
> As a concrete example, suppose Alice commits some code that uses "foo" > as a variable name, and more or less concurrently, Bob commits something > that defines "foo" as a typedef. Bob's change is likely to have > side-effects on the formatting of Alice's code. If they're working in >

Re: run pgindent on a regular basis / scripted manner

2023-01-24 Thread Andrew Dunstan
On 2023-01-24 Tu 11:43, Tom Lane wrote: > Jelte Fennema writes: >> Sounds like this conflict could be handled fairly easily by >> having a local git hook rerunning pgindent whenever >> you rebase a commit: >> 1. if you changed typedefs.list the hook would format all files >> 2. if you didn't it

Re: psql: Add role's membership options to the \du+ command

2023-01-24 Thread David G. Johnston
On Mon, Jan 9, 2023 at 9:09 AM Pavel Luzanov wrote: > When you include one role in another, you can specify three options: > ADMIN, INHERIT (added in e3ce2de0) and SET (3d14e171). > > For example. > > CREATE ROLE alice LOGIN; > > GRANT pg_read_all_settings TO alice WITH ADMIN TRUE, INHERIT TRUE,

Re: plpython vs _POSIX_C_SOURCE

2023-01-24 Thread Tom Lane
Andres Freund writes: > The background for the undefines is that _POSIX_C_SOURCE needs to be defined > the same for the whole compilation, not change in the middle, and Python.h > defines it. To protect "our" parts a11cf433413 instituted the rule that all > postgres headers have to be included

Re: Improve GetConfigOptionValues function

2023-01-24 Thread Tom Lane
Bharath Rupireddy writes: > On Mon, Jan 23, 2023 at 9:51 PM Tom Lane wrote: >> Also, I intentionally dropped the GUC_NO_SHOW_ALL check in >> get_explain_guc_options, because it seems redundant given >> the preceding GUC_EXPLAIN check. It's unlikely we'd ever have >> a variable that's marked

Re: pgindent vs variable declaration across multiple lines

2023-01-24 Thread Andrew Dunstan
On 2023-01-22 Su 17:34, Tom Lane wrote: > I've also attached a diff > representing the delta between what current pg_bsd_indent wants to do > to HEAD and what this would do. All the changes it wants to make look > good, although I can't say whether there are other places it's failing > to

Re: Unicode grapheme clusters

2023-01-24 Thread Greg Stark
On Sat, 21 Jan 2023 at 13:17, Tom Lane wrote: > > Probably our long-term answer is to avoid depending on wcwidth > and use wcswidth instead. But it's hard to get excited about > doing the legwork for that until popular libc implementations > get it right. Here's an interesting blog post about

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-24 Thread Nathan Bossart
On Tue, Jan 24, 2023 at 02:55:07AM +, houzj.f...@fujitsu.com wrote: > I noticed one minor thing in this commit. > > - > LogicalRepCtx->last_start_dsh = DSM_HANDLE_INVALID; > - > > The code takes the last_start_dsh as dsm_handle, but it seems it is a > dsa_pointer. > " typedef dsa_pointer

Re: Non-superuser subscription owners

2023-01-24 Thread Andrew Dunstan
On 2023-01-24 Tu 08:50, Robert Haas wrote: > > What do you think about something in the spirit of a > reverse-pg_hba.conf? The idea being that PostgreSQL facilities that > make outbound connections are supposed to ask it whether those > connections are OK to initiate. Then you could have a

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-24 Thread Nathan Bossart
On Tue, Jan 24, 2023 at 01:13:55PM -0500, Tom Lane wrote: > Either that comment needs to be rewritten or we need to invent some > more macros. Here is a first attempt at a patch. I scanned through all the existing uses of InvalidDsaPointer and DSM_HANDLE_INVALID and didn't notice anything else

Re: run pgindent on a regular basis / scripted manner

2023-01-24 Thread Jelte Fennema
> I think that would be undesirable, because then reindentation noise > in completely-unrelated files would get baked into feature commits, > complicating review and messing up "git blame" history. With a rebase hook similar to the the pre-commit hook that I shared upthread, your files will be

Re: Reducing power consumption on idle servers

2023-01-24 Thread Thomas Munro
On Wed, Nov 30, 2022 at 7:40 PM Simon Riggs wrote: > On Wed, 30 Nov 2022 at 03:50, Thomas Munro wrote: > > I'm just curious, and not suggesting that 60s wakeups are a problem > > for the polar ice caps, but why even time out at all? Are the latch > > protocols involved not reliable enough? At

postgres_fdw, dblink, and CREATE SUBSCRIPTION security

2023-01-24 Thread Robert Haas
[ Changing subject line to something more appropriate: This is branched from the "Non-superuser subscription owners" thread, but the topic has become connection security more generally for outbound connections from a PostgreSQL instance, the inadequacies of just trying to require that such

Re: plpython vs _POSIX_C_SOURCE

2023-01-24 Thread Tom Lane
Andres Freund writes: > Python's _POSIX_C_SOURCE value is set to a specific value in their configure > script: > if test $define_xopen_source = yes > then > ... > AC_DEFINE(_POSIX_C_SOURCE, 200809L, Define to activate features from IEEE > Stds 1003.1-2008) > fi Hm. I looked into Python

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-24 Thread Peter Smith
On Tue, Jan 24, 2023 at 11:49 PM houzj.f...@fujitsu.com wrote: > ... > > Sorry, the patch set was somehow attached twice. Here is the correct new > version > patch set which addressed all comments so far. > Here are my review comments for patch v87-0001. ==

Re: Helper functions for wait_for_catchup() in Cluster.pm

2023-01-24 Thread Alvaro Herrera
Looking again, I have two thoughts for making things easier: 1. I don't think wait_for_write_catchup is necessary, because calling wait_for_catchup() and omitting the 'mode' and 'lsn' arguments would already do the same thing. So what we should do is patch places that currently give those two

Re: run pgindent on a regular basis / scripted manner

2023-01-24 Thread Bruce Momjian
On Tue, Jan 24, 2023 at 09:54:57AM -0500, Tom Lane wrote: > As another example, the mechanisms we use to create the typedefs list > in the first place are pretty squishy/leaky: they depend on which > buildfarm animals are running the typedef-generation step, and on > whether anything's broken

Re: Making Vars outer-join aware

2023-01-24 Thread Tom Lane
"David G. Johnston" writes: > On Tue, Jan 24, 2023 at 12:31 PM Tom Lane wrote: >> select ... from t1 left join t2 on (t1.x = t2.y and t1.x = 1); >> >> If we turn the generic equivclass.c logic loose on these clauses, >> it will deduce t2.y = 1, which is good, and then apply t2.y = 1 at >> the

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-24 Thread Peter Geoghegan
On Tue, Jan 24, 2023 at 11:21 AM Robert Haas wrote: > > The whole article was about how this DROP TRIGGER pattern worked just > > fine most of the time, because most of the time autovacuum was just > > autocancelled. They say this at one point: > > > > "The normal autovacuum mechanism is skipped

Re: Making Vars outer-join aware

2023-01-24 Thread David G. Johnston
On Tue, Jan 24, 2023 at 1:25 PM Tom Lane wrote: > "David G. Johnston" writes: > > On Tue, Jan 24, 2023 at 12:31 PM Tom Lane wrote: > >> select ... from t1 left join t2 on (t1.x = t2.y and t1.x = 1); > >> > >> If we turn the generic equivclass.c logic loose on these clauses, > >> it will deduce

Re: Reducing power consumption on idle servers

2023-01-24 Thread Tom Lane
Thomas Munro writes: > Yeah, I definitely want to fix it. I just worry that 60s is so long > that it also needs that analysis work to be done to explain that it's > OK that we're a bit sloppy on noticing when to wake up, at which point > you might as well go to infinity. Yeah. The

Re: plpython vs _POSIX_C_SOURCE

2023-01-24 Thread Andres Freund
Hi, On 2023-01-24 12:55:15 -0500, Tom Lane wrote: > Andres Freund writes: > > The background for the undefines is that _POSIX_C_SOURCE needs to be defined > > the same for the whole compilation, not change in the middle, and Python.h > > defines it. To protect "our" parts a11cf433413 instituted

Re: Non-superuser subscription owners

2023-01-24 Thread Jacob Champion
On Tue, Jan 24, 2023 at 5:50 AM Robert Haas wrote: > I think this has some potential, but it's pretty complex, seeming to > require protocol extensions and having backward-compatibility problems > and so on. Yeah. > What do you think about something in the spirit of a > reverse-pg_hba.conf? The

Re: Making Vars outer-join aware

2023-01-24 Thread David G. Johnston
On Tue, Jan 24, 2023 at 12:31 PM Tom Lane wrote: > I wrote: > > Hans Buschmann writes: > >> I just noticed your new efforts in this area. > >> I wanted to recurr to my old thread [1] considering constant > propagation of quals. > >> [1] >

Re: heapgettup refactoring

2023-01-24 Thread Melanie Plageman
Thanks for taking a look! On Mon, Jan 23, 2023 at 6:08 AM David Rowley wrote: > > On Thu, 19 Jan 2023 at 00:04, Peter Eisentraut > wrote: > > In your v2 patch, you remove these assertions: > > > > - /* check that rs_cindex is in sync */ > > - Assert(scan->rs_cindex <

Re: HOT chain validation in verify_heapam()

2023-01-24 Thread Robert Haas
On Sun, Jan 22, 2023 at 10:19 AM Himanshu Upadhyaya wrote: > I was trying to use lp_valid as I need to identify the root of the HOT chain > and we are doing validation on the root of the HOT chain when we loop over > the predecessor array. > Was resetting lp_valid in the last patch because we

Re: Unicode grapheme clusters

2023-01-24 Thread Bruce Momjian
On Tue, Jan 24, 2023 at 11:40:01AM -0500, Greg Stark wrote: > On Sat, 21 Jan 2023 at 13:17, Tom Lane wrote: > > > > Probably our long-term answer is to avoid depending on wcwidth > > and use wcswidth instead. But it's hard to get excited about > > doing the legwork for that until popular libc

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-24 Thread Robert Haas
On Fri, Jan 20, 2023 at 4:24 PM Peter Geoghegan wrote: > > It sounds like they used DROP TRIGGER pretty regularly. So I think this > > sounds like exactly the kind of case I was talking about, where > > autovacuums keep getting cancelled until we decide to stop cancelling > > them. > > I don't

Re: Making Vars outer-join aware

2023-01-24 Thread Tom Lane
I wrote: > Hans Buschmann writes: >> I just noticed your new efforts in this area. >> I wanted to recurr to my old thread [1] considering constant propagation of >> quals. >> [1] https://www.postgresql.org/message-id/1571413123735.26...@nidsa.net > Yeah, this patch series is not yet quite up

Re: run pgindent on a regular basis / scripted manner

2023-01-24 Thread Andrew Dunstan
On 2023-01-24 Tu 13:42, Jelte Fennema wrote: >> Here's another improvement I think will be useful when the new gadgets >> are used in a git hook: first, look for the excludes file under the >> current directory if we aren't setting $code_base (e.g if we have files >> given on the command line),

Re: Non-superuser subscription owners

2023-01-24 Thread Robert Haas
On Mon, Jan 23, 2023 at 3:50 PM Jeff Davis wrote: > I believe your patch conflates two use cases: > > (A) Tightly-coupled servers that are managed by the administrator. In > this case, there are a finite number of connection strings to make, and > the admin knows about all of them. Validation is

Re: Update comments in multixact.c

2023-01-24 Thread Peter Geoghegan
On Wed, Jan 18, 2023 at 2:02 AM shiy.f...@fujitsu.com wrote: > Thanks for your reply. > > Attach a patch which fixed them. Pushed something close to that just now. I decided that it was better to not specify when truncation happened in these two places at all, though. The important detail is

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-24 Thread Kyotaro Horiguchi
At Tue, 24 Jan 2023 11:45:36 +0530, Amit Kapila wrote in > Personally, I would prefer the above LOGs to be in reverse order as it > doesn't make much sense to me to first say that we are skipping > changes and then say the transaction is delayed. What do you think? In the first place, I

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-24 Thread Kyotaro Horiguchi
At Tue, 24 Jan 2023 14:22:19 +0530, Amit Kapila wrote in > On Tue, Jan 24, 2023 at 12:44 PM Peter Smith wrote: > > > > On Tue, Jan 24, 2023 at 5:58 PM Amit Kapila wrote: > > > > > > On Tue, Jan 24, 2023 at 8:15 AM Kyotaro Horiguchi > > > wrote: > > > > > > > > > Attached the updated patch

Re: Improve logging when using Huge Pages

2023-01-24 Thread Justin Pryzby
On Mon, Jan 23, 2023 at 05:33:35PM -0800, Andres Freund wrote: > Hi, > > On 2023-01-23 19:21:00 -0600, Justin Pryzby wrote: > > Michael seemed to support this idea and nobody verbalized hatred of it, > > so I implemented it. In v15, we have shared_memory_size_in_huge_pages, > > so this adds

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-01-24 Thread Andres Freund
Hi, On 2023-01-24 17:22:03 +0900, Kyotaro Horiguchi wrote: > Hello. > > At Thu, 19 Jan 2023 21:15:34 -0500, Melanie Plageman > wrote in > > Oh dear-- an extra FlushBuffer() snuck in there somehow. > > Removed it in attached v51. > > Also, I fixed an issue in my tablespace.sql updates > > I

Re: 011_crash_recovery.pl intermittently fails

2023-01-24 Thread Thomas Munro
On Mon, Mar 8, 2021 at 9:32 PM Kyotaro Horiguchi wrote: > At Sun, 07 Mar 2021 20:09:33 -0500, Tom Lane wrote in > > Thomas Munro writes: > > > Thanks! I'm afraid I wouldn't get around to it for a few weeks, so if > > > you have time, please do. (I'm not sure if it's strictly necessary to > >

Re: Generating code for query jumbling through gen_node_support.pl

2023-01-24 Thread Michael Paquier
On Tue, Jan 24, 2023 at 03:57:56PM +0900, Michael Paquier wrote: > Makes sense. That would be my intention if 0004 is the most > acceptable and splitting things makes things a bit easier to review. There was a silly mistake in 0004 where the jumbling code relied on compute_query_id rather than

Re: heapgettup refactoring

2023-01-24 Thread Melanie Plageman
On Tue, Jan 24, 2023 at 04:17:23PM -0500, Melanie Plageman wrote: > Thanks for taking a look! > > On Mon, Jan 23, 2023 at 6:08 AM David Rowley wrote: > > > > On Thu, 19 Jan 2023 at 00:04, Peter Eisentraut > > wrote: > > > In your v2 patch, you remove these assertions: > > > > > > - /*

Re: Add LZ4 compression in pg_dump

2023-01-24 Thread Justin Pryzby
On Tue, Jan 24, 2023 at 03:56:20PM +, gkokola...@pm.me wrote: > On Monday, January 23rd, 2023 at 7:00 PM, Justin Pryzby > wrote: > > On Mon, Jan 23, 2023 at 05:31:55PM +, gkokola...@pm.me wrote: > > > > > Please find attached v23 which reintroduces the split. > > > > > > 0001 is

Re: suppressing useless wakeups in logical/worker.c

2023-01-24 Thread Tom Lane
Nathan Bossart writes: > [ v2-0001-suppress-unnecessary-wakeups-in-logical-worker.c.patch ] I took a look through this, and have a number of mostly-cosmetic issues: * It seems wrong that next_sync_start isn't handled as one of the wakeup[NUM_LRW_WAKEUPS] entries. I see that it needs to be

Re: 011_crash_recovery.pl intermittently fails

2023-01-24 Thread Thomas Munro
On Wed, Jan 25, 2023 at 1:02 PM Michael Paquier wrote: > Well, this host has a problem, for what looks like a kernel issue, I > guess.. This is repeatable across all the branches, randomly, with > various errors with the POSIX DSM implementation: > # [63cf68b7.5e5a:1] ERROR: could not open

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-24 Thread Kyotaro Horiguchi
In short, I'd like to propose renaming the parameter in_delayed_apply of send_feedback to "has_unprocessed_change". At Tue, 24 Jan 2023 12:27:58 +0530, Amit Kapila wrote in > > send_feedback(): > > +* If the subscriber side apply is delayed (because of time-delayed > > +*

Re: Mutable CHECK constraints?

2023-01-24 Thread Laurenz Albe
On Tue, 2023-01-24 at 01:38 -0500, Tom Lane wrote: > Laurenz Albe writes: > > We throw an error if the expression in a CREATE INDEX statement is not > > IMMUTABLE. > > But while the documentation notes that expressions in CHECK constraints are > > not > > to be immutable, we don't enforce that. 

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-24 Thread Peter Smith
Here are my review comments for patch v87-0002. == doc/src/sgml/config.sgml 1. -Allows streaming or serializing changes immediately in logical decoding. The allowed values of logical_replication_mode are -buffered and immediate. When set -to

Getting relations accessed by a query using the raw query string

2023-01-24 Thread Amin
Hi, Having a query string, I am trying to use the postgres parser to find which relations the query accesses. This is what I currently have: const char *query_string="select * from dummytable;"; List *parsetree_list=pg_parse_query(query_string); ListCell *parsetree_item;

Re: 011_crash_recovery.pl intermittently fails

2023-01-24 Thread Michael Paquier
On Wed, Jan 25, 2023 at 12:40:02PM +1300, Thomas Munro wrote: > I remembered this thread after seeing the failure of Michael's new > build farm animal "tanager". I think we need to solve this somehow... Well, this host has a problem, for what looks like a kernel issue, I guess.. This is

Re: 011_crash_recovery.pl intermittently fails

2023-01-24 Thread Michael Paquier
On Wed, Jan 25, 2023 at 01:20:39PM +1300, Thomas Munro wrote: > Something to do with > https://www.postgresql.org/docs/current/kernel-resources.html#SYSTEMD-REMOVEIPC > ? Still this is unrelated? This is a buildfarm instance, so the backend does not run with systemd. > The failure I saw looked

Re: 011_crash_recovery.pl intermittently fails

2023-01-24 Thread Tom Lane
Michael Paquier writes: > On Wed, Jan 25, 2023 at 01:20:39PM +1300, Thomas Munro wrote: >> Something to do with >> https://www.postgresql.org/docs/current/kernel-resources.html#SYSTEMD-REMOVEIPC >> ? > Still this is unrelated? This is a buildfarm instance, so the backend > does not run with

heapgettup() with NoMovementScanDirection unused in core?

2023-01-24 Thread Melanie Plageman
Hi, David Rowley and I were discussing how to test the NoMovementScanDirection case for heapgettup() and heapgettup_pagemode() in [1] (since there is not currently coverage). We are actually wondering if it is dead code (in core). This is a link to the code in question on github in [2] (side

  1   2   >