Assert fcinfo has enough args before allowing parameter access (was: Re: generate_series for timestamptz and time zone problem)

2023-01-30 Thread Gurjeet Singh
On Mon, Jan 30, 2023 at 11:50 PM Gurjeet Singh wrote: > It was the classical case of out-of-bounds access. > This mistake would've been caught early if there were assertions > preventing access beyond the number of arguments passed to the > function. I'll send the assert_enough_args.patch, that

Re: generate_series for timestamptz and time zone problem

2023-01-30 Thread Gurjeet Singh
On Mon, Jan 30, 2023 at 4:07 PM Tom Lane wrote: > Gurjeet Singh writes: > > [ generate_series_with_timezone.v6.patch ] > > The cfbot isn't terribly happy with this. It looks like UBSan > is detecting some undefined behavior. Possibly an uninitialized > variable? It was the classical case of

Re: MacOS: xsltproc fails with "warning: failed to load external entity"

2023-01-30 Thread Peter Eisentraut
On 30.01.23 20:04, Aleksander Alekseev wrote: I would appreciate it if you could help figuring out how to do this for MacPorts, since I'm not a MacPorts user. I'll figure out how to do this for Homebrew. I'm on macOS Monterey and Homebrew. I'm sure I have gone through many variations of this

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-30 Thread Hayato Kuroda (Fujitsu)
Dear Horiguchi-san, > I'm not sure. I think that int is generally thought that it is tied > with an integer type of any size. min_apply_delay is tightly bond > with a catalog column of int32 thus I thought that (PG_)INT32_MAX is > the right limit. So, as I expressed before, if we assume

Re: Assertion failure in SnapBuildInitialSnapshot()

2023-01-30 Thread Masahiko Sawada
On Tue, Jan 31, 2023 at 3:56 PM Amit Kapila wrote: > > On Tue, Jan 31, 2023 at 11:12 AM Masahiko Sawada > wrote: > > > > On Mon, Jan 30, 2023 at 9:41 PM Masahiko Sawada > > wrote: > > > > I've attached patches for HEAD and backbranches. Please review them. > > > > Shall we add a comment like

Re: Assertion failure in SnapBuildInitialSnapshot()

2023-01-30 Thread Amit Kapila
On Tue, Jan 31, 2023 at 11:12 AM Masahiko Sawada wrote: > > On Mon, Jan 30, 2023 at 9:41 PM Masahiko Sawada wrote: > > I've attached patches for HEAD and backbranches. Please review them. > Shall we add a comment like the one below in ReplicationSlotsComputeRequiredXmin()? diff --git

Re: Refactoring postgres_fdw/connection.c

2023-01-30 Thread Fujii Masao
On 2023/01/29 19:31, Etsuro Fujita wrote: I agree that if the name of an existing function was bad, we should rename it, but I do not think the name pgfdw_get_cleanup_result is bad; I think it is good in the sense that it well represents what the function waits for. The patch you proposed

Re: Generating code for query jumbling through gen_node_support.pl

2023-01-30 Thread Michael Paquier
On Mon, Jan 30, 2023 at 11:48:45AM +0100, Peter Eisentraut wrote: > I'm going to set this thread as "Ready for Committer". Either wait a bit > for more feedback on this topic, or just go ahead with either solution. We > can leave it as a semi-open item for reconsideration later. All the

Contractions in full text search result in very surprising stemming

2023-01-30 Thread Sam Saffron
Per: ``` select ts_debug('english', 'you''re a star'); ts_debug --- (asciiword,"Word, all ASCII",you,{english_stem},english_stem,{}) (blank,"Space symbols",',{},,) (asciiword,"Word, all

Re: pub/sub - specifying optional parameters without values.

2023-01-30 Thread Amit Kapila
On Tue, Jan 31, 2023 at 4:25 AM Tom Lane wrote: > > Peter Smith writes: > > I'd forgotten about the 'streaming' option -- AFAIK this was > > previously a boolean parameter and so its [= value] part can also be > > omitted. However, in PG16 streaming became an enum type > > (on/off/parallel), and

Re: Assertion failure in SnapBuildInitialSnapshot()

2023-01-30 Thread Masahiko Sawada
On Mon, Jan 30, 2023 at 9:41 PM Masahiko Sawada wrote: > > On Mon, Jan 30, 2023 at 8:30 PM Amit Kapila wrote: > > > > On Fri, Jan 27, 2023 at 4:31 PM Hayato Kuroda (Fujitsu) > > wrote: > > > > > > Thank you for making the patch! I'm still considering whether this > > > approach is > > >

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-30 Thread Peter Smith
Thanks for the updates to address all of my previous review comments. Patch v90-0001 LGTM. -- Kind Reagrds, Peter Smith. Fujitsu Australia

Re: odd buildfarm failure - "pg_ctl: control file appears to be corrupt"

2023-01-30 Thread Thomas Munro
On Tue, Jan 31, 2023 at 2:10 PM Anton A. Melnikov wrote: > Also checked for a few hours that the patch 0002 fixes this error, > but there are some questions to its logical structure. Hi Anton, Thanks for looking! > The equality between the previous and newly calculated crc is checked only > if

Re: Generating code for query jumbling through gen_node_support.pl

2023-01-30 Thread Michael Paquier
On Mon, Jan 30, 2023 at 11:46:06AM +0100, Peter Eisentraut wrote: > Either that or src/backend/nodes/README. The README holds nothing about the node attributes currently, so nodes.h feels most adapted here at the end.. Thanks, -- Michael signature.asc Description: PGP signature

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-30 Thread Kyotaro Horiguchi
At Mon, 30 Jan 2023 14:24:31 +0530, Amit Kapila wrote in > On Mon, Jan 30, 2023 at 12:38 PM Kyotaro Horiguchi > wrote: > > > > At Mon, 30 Jan 2023 11:56:33 +0530, Amit Kapila > > wrote in > > > #define TimestampTzPlusMilliseconds(tz,ms) ((tz) + ((ms) * (int64) 1000)) > > > > Which can lead

Re: Scan buffercache for a table

2023-01-30 Thread Amin
Thank you Justin. I started a new thread because the context is a little bit different. I am no longer interested in statistics anymore. I want to find exact individual pages of a table which are cached and are/aren't dirty. pg_buffercache implements the loop, but it goes over all the buffers.

Re: Improve logging when using Huge Pages

2023-01-30 Thread Justin Pryzby
On Tue, Jan 24, 2023 at 07:37:29PM -0600, Justin Pryzby wrote: > BTW, I didn't realize it when I made the suggestion, nor when I wrote > the patch, but a GUC was implemented in the v2 patch. >

Re: Something is wrong with wal_compression

2023-01-30 Thread Michael Paquier
On Mon, Jan 30, 2023 at 02:57:13PM +0900, Michael Paquier wrote: > 1) means more test cycles, and perhaps we could enforce compression of > WAL while on it? At the end, my vote would just go for 3) and drop > the whole scenario, though there may be an argument in 1). And actually I was under the

RE: Perform streaming logical transactions by background workers and parallel apply

2023-01-30 Thread houzj.f...@fujitsu.com
On Tuesday, January 31, 2023 8:23 AM Peter Smith wrote: > > On Mon, Jan 30, 2023 at 5:23 PM houzj.f...@fujitsu.com > wrote: > > > > On Monday, January 30, 2023 12:13 PM Peter Smith > wrote: > > > > > > Here are my review comments for v88-0002. > > > > Thanks for your comments. > > > > > > > >

RE: Perform streaming logical transactions by background workers and parallel apply

2023-01-30 Thread houzj.f...@fujitsu.com
On Monday, January 30, 2023 10:20 PM Masahiko Sawada wrote: > > > I have one comment on v89 patch: > > + /* > +* Using 'immediate' mode returns false to cause a switch to > +* PARTIAL_SERIALIZE mode so that the remaining changes will > be serialized. > +*/ > +

Re: Scan buffercache for a table

2023-01-30 Thread Justin Pryzby
On Mon, Jan 30, 2023 at 06:01:08PM -0800, Amin wrote: > Hi, > > I am looking for function calls to scan the buffer cache for a table and > find the cached pages. I want to find out which pages are cached and which > of them are dirty. Having the relation id, how can I do that? I have gone >

Re: Allow an extention to be updated without a script

2023-01-30 Thread Yugo NAGATA
Hi, Thank you for your comment. On Mon, 30 Jan 2023 16:05:52 -0500 Tom Lane wrote: > Yugo NAGATA writes: > > Currently, even when we don't need to execute any command to update an > > extension from one version to the next, we need to provide an update > > script that doesn't contain any

Scan buffercache for a table

2023-01-30 Thread Amin
Hi, I am looking for function calls to scan the buffer cache for a table and find the cached pages. I want to find out which pages are cached and which of them are dirty. Having the relation id, how can I do that? I have gone through bufmgr.c and relcache.c, but could not find a way to get

Re: Assertion failure in SnapBuildInitialSnapshot()

2023-01-30 Thread Masahiko Sawada
On Mon, Jan 30, 2023 at 8:24 PM Amit Kapila wrote: > > On Thu, Dec 8, 2022 at 8:17 AM Masahiko Sawada wrote: > > > > On Mon, Nov 21, 2022 at 4:31 PM Amit Kapila wrote: > > > > One idea to fix this issue is that in > > ReplicationSlotsComputeRequiredXmin(), we compute the minimum xmin > > while

Re: odd buildfarm failure - "pg_ctl: control file appears to be corrupt"

2023-01-30 Thread Anton A. Melnikov
Hello! On 24.11.2022 04:02, Thomas Munro wrote: On Thu, Nov 24, 2022 at 11:05 AM Tom Lane wrote: Thomas Munro writes: ERROR: calculated CRC checksum does not match value stored in file The attached draft patch fixes it. Tried to catch this error on my PC, but failed to do it within a

Re: pg_upgrade test failure

2023-01-30 Thread Thomas Munro
On Thu, Jan 5, 2023 at 4:11 PM Thomas Munro wrote: > On Wed, Dec 7, 2022 at 7:15 AM Andres Freund wrote: > > On 2022-11-08 01:16:09 +1300, Thomas Munro wrote: > > > So [1] on its own didn't fix this. My next guess is that the attached > > > might help. > > > What is our plan here? This afaict

Re: postgres_fdw uninterruptible during connection establishment / ProcSignalBarrier

2023-01-30 Thread Robins Tharakan
Hi Andres, On Tue, 31 Jan 2023 at 06:31, Nathan Bossart wrote: > > On Mon, Jan 30, 2023 at 11:49:37AM -0800, Andres Freund wrote: > > Why don't the dblink tests catch this? Any chance you or Robins could > > prepare > > a patch with fix and test, given that you know how to trigger this? > >

Re: Introduce "log_connection_stages" setting.

2023-01-30 Thread Justin Pryzby
Thanks for updating the patch. It's currently failing check-world, due to a test that was added on January 23 (a9dc7f941): http://cfbot.cputube.org/sergey-dudoladov.html [19:15:57.101] Summary of Failures: [19:15:57.101] [19:15:57.101] 250/251 postgresql:ldap / ldap/002_bindpasswd ERROR

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-30 Thread Peter Smith
On Mon, Jan 30, 2023 at 5:23 PM houzj.f...@fujitsu.com wrote: > > On Monday, January 30, 2023 12:13 PM Peter Smith > wrote: > > > > Here are my review comments for v88-0002. > > Thanks for your comments. > > > > > == > > General > > > > 1. > > The test cases are checking the log content but

Re: generate_series for timestamptz and time zone problem

2023-01-30 Thread Tom Lane
Gurjeet Singh writes: > [ generate_series_with_timezone.v6.patch ] The cfbot isn't terribly happy with this. It looks like UBSan is detecting some undefined behavior. Possibly an uninitialized variable? regards, tom lane

Re: Making background psql nicer to use in tap tests

2023-01-30 Thread Andres Freund
Hi, On 2023-01-30 15:06:46 -0500, Tom Lane wrote: > Andres Freund writes: > > It's annoyingly hard to wait for the result of a query in a generic way with > > background_psql(), and more generally for psql. background_psql() uses > > -XAtq, > > which means that we'll not get "status" output

monitoring usage count distribution

2023-01-30 Thread Nathan Bossart
My colleague Jeremy Schneider (CC'd) was recently looking into usage count distributions for various workloads, and he mentioned that it would be nice to have an easy way to do $SUBJECT. I've attached a patch that adds a pg_buffercache_usage_counts() function. This function returns a row per

Re: heapgettup refactoring

2023-01-30 Thread Melanie Plageman
v7 attached On Fri, Jan 27, 2023 at 10:34 PM David Rowley wrote: > > "On Wed, 25 Jan 2023 at 10:17, Melanie Plageman > wrote: > > I've added a comment but I didn't include the function name in it -- I > > find it repetitive when the comments above functions do that -- however, > > I'm not

Re: recovery modules

2023-01-30 Thread Michael Paquier
On Mon, Jan 30, 2023 at 12:04:22PM -0800, Nathan Bossart wrote: > On Mon, Jan 30, 2023 at 11:48:10AM -0800, Andres Freund wrote: >> I don't think _PG_archive_module_init() should actually allocate a memory >> context and do other similar initializations. Instead it should just return >> 'const

Re: pub/sub - specifying optional parameters without values.

2023-01-30 Thread Tom Lane
Peter Smith writes: > I'd forgotten about the 'streaming' option -- AFAIK this was > previously a boolean parameter and so its [= value] part can also be > omitted. However, in PG16 streaming became an enum type > (on/off/parallel), and the value can still be omitted but that is not > really

Re: pub/sub - specifying optional parameters without values.

2023-01-30 Thread Peter Smith
On Tue, Jan 31, 2023 at 4:00 AM Tom Lane wrote: > > Peter Smith writes: > > The v3 patch LGTM (just for the logical replication commands). > > Pushed then. > Thanks for pushing the v3 patch. I'd forgotten about the 'streaming' option -- AFAIK this was previously a boolean parameter and so its

Re: Transparent column encryption

2023-01-30 Thread Jacob Champion
On Wed, Jan 25, 2023 at 11:00 AM Peter Eisentraut wrote: > > When writing the paragraph on RSA-OAEP I was reminded that we didn't > > really dig into the asymmetric/symmetric discussion. Assuming that most > > first-time users will pick the builtin CMK encryption method, do we > > still want to

Re: postgres_fdw, dblink, and CREATE SUBSCRIPTION security

2023-01-30 Thread Robert Haas
On Mon, Jan 30, 2023 at 4:12 PM Jacob Champion wrote: > For our case, assuming that connections have side effects, one > solution could be for the client to signal to the server that the > connection should use in-band authentication only; i.e. fail the > connection if the credentials provided

Re: Non-superuser subscription owners

2023-01-30 Thread Robert Haas
On Mon, Jan 30, 2023 at 3:27 PM Mark Dilger wrote: > That was an aspirational example in which there's infinite daylight between > the publisher and subscriber. I, too, doubt that's ever going to be > possible. But I still think we should aspire to some extra daylight between > the two.

Re: postgres_fdw, dblink, and CREATE SUBSCRIPTION security

2023-01-30 Thread Jacob Champion
On Fri, Jan 27, 2023 at 1:08 PM Robert Haas wrote: > > 1) Forwarding the original ambient context along with the request, so > > the server can check it too. > > Right, so a protocol extension. Reasonable idea, but a big lift. Not > only do you need everyone to be running a new enough version of

Re: Allow an extention to be updated without a script

2023-01-30 Thread Tom Lane
Yugo NAGATA writes: > Currently, even when we don't need to execute any command to update an > extension from one version to the next, we need to provide an update > script that doesn't contain any command. Preparing such meaningless > files are sometimes annoying. If you have no update script,

Re: Authentication fails for md5 connections if ~/.postgresql/postgresql.{crt and key} exist

2023-01-30 Thread Jacob Champion
On Sun, Jan 29, 2023 at 5:02 AM Jim Jones wrote: > On 27.01.23 21:13, Cary Huang wrote: > > But, if the server does request clientcert but client uses > "sslcertmode=disable" to connect and not give a certificate, it would > also result in authentication failure. In this case, we actually would

Re: MacOS: xsltproc fails with "warning: failed to load external entity"

2023-01-30 Thread Tom Lane
Aleksander Alekseev writes: >> What we do actually have already is a recommendation to install >> appropriate MacPorts or Homebrew packages: >> https://www.postgresql.org/docs/devel/docguide-toolsets.html#DOCGUIDE-TOOLSETS-INST-MACOS >> and it works okay for me as long as I use MacPorts' version

Re: Authentication fails for md5 connections if ~/.postgresql/postgresql.{crt and key} exist

2023-01-30 Thread Jacob Champion
On Fri, Jan 27, 2023 at 12:13 PM Cary Huang wrote: > > (Eventually I'd like to teach the server not to ask for a client > > certificate if it's not going to use it.) > > If clientcert is not requested by the server, but yet the client still > sends the certificate, the server will still verify

Re: heapgettup() with NoMovementScanDirection unused in core?

2023-01-30 Thread Melanie Plageman
v2 attached On Fri, Jan 27, 2023 at 6:28 PM David Rowley wrote: > > On Sat, 28 Jan 2023 at 12:15, Tom Lane wrote: > > /* > > * Determine the net effect of two direction specifications. > > * This relies on having ForwardScanDirection = +1, BackwardScanDirection = > > -1, > > * and will

Re: run pgindent on a regular basis / scripted manner

2023-01-30 Thread Bruce Momjian
On Sat, Jan 28, 2023 at 05:06:03PM -0800, Noah Misch wrote: > On Tue, Jan 24, 2023 at 02:04:02PM -0500, Bruce Momjian wrote: > > On Tue, Jan 24, 2023 at 09:54:57AM -0500, Tom Lane wrote: > > > As another example, the mechanisms we use to create the typedefs list > > > in the first place are pretty

Re: Non-superuser subscription owners

2023-01-30 Thread Robert Haas
On Fri, Jan 27, 2023 at 5:00 PM Andres Freund wrote: > > Or, another thought, maybe this should be checking for CREATE on the > > current database + also pg_create_subscription. That seems like it > > might be the right idea, actually. > > Yes, that seems like a good idea. Done in this version.

Re: Non-superuser subscription owners

2023-01-30 Thread Mark Dilger
> On Jan 30, 2023, at 11:30 AM, Robert Haas wrote: > > CREATE SUBSCRIPTION > provides no tools at all for filtering the data that the subscriber > chooses to send. > > Now that can be changed, I suppose, and a run-as user would be one way > to make progress in that direction. But I'm not

Allow an extention to be updated without a script

2023-01-30 Thread Yugo NAGATA
Hi hackers, I propose to add a new option "updates_without_script" to extension's control file which a list of updates that do not need update script. This enables to update an extension by ALTER EXTENSION even if the extension module doesn't provide the update script. Currently, even when we

Re: Add SHELL_EXIT_CODE to psql

2023-01-30 Thread Corey Huinker
> > > Unfortunately, there is a fail in FreeBSD > https://cirrus-ci.com/task/6466749487382528 > > Maybe, this patch is need to be rebased? > > That failure is in postgres_fdw, which this code doesn't touch. I'm not able to get to

Re: BUG: Postgres 14 + vacuum_defer_cleanup_age + FOR UPDATE + UPDATE

2023-01-30 Thread Andres Freund
Hi, On 2023-01-10 21:32:54 +0100, Matthias van de Meent wrote: > On Tue, 10 Jan 2023 at 20:14, Andres Freund wrote: > > On 2023-01-10 15:03:42 +0100, Matthias van de Meent wrote: > > What precisely do you mean with "skew" here? Do you just mean that it'd > > take a > > long time until

Re: Making background psql nicer to use in tap tests

2023-01-30 Thread Tom Lane
Andres Freund writes: > It's annoyingly hard to wait for the result of a query in a generic way with > background_psql(), and more generally for psql. background_psql() uses -XAtq, > which means that we'll not get "status" output (like "BEGIN" or "(1 row)"), > and that queries not returning

Re: recovery modules

2023-01-30 Thread Nathan Bossart
On Mon, Jan 30, 2023 at 11:48:10AM -0800, Andres Freund wrote: > I don't think _PG_archive_module_init() should actually allocate a memory > context and do other similar initializations. Instead it should just return > 'const ArchiveModuleCallbacks*', typically a single line. > > Allocations etc

Re: postgres_fdw uninterruptible during connection establishment / ProcSignalBarrier

2023-01-30 Thread Nathan Bossart
On Mon, Jan 30, 2023 at 11:49:37AM -0800, Andres Freund wrote: > Why don't the dblink tests catch this? Any chance you or Robins could prepare > a patch with fix and test, given that you know how to trigger this? It's trivially reproducible by calling 1-argument dblink_connect() multiple times

Re: postgres_fdw uninterruptible during connection establishment / ProcSignalBarrier

2023-01-30 Thread Andres Freund
Hi, On 2023-01-30 11:30:08 -0800, Nathan Bossart wrote: > On Mon, Jan 23, 2023 at 07:28:06PM -0800, Andres Freund wrote: > > After a tiny bit further polishing, and after separately pushing a resource > > leak fix for walrcv_connect(), I pushed this. > > My colleague Robins Tharakan (CC'd)

Re: recovery modules

2023-01-30 Thread Andres Freund
Hi, On 2023-01-30 16:51:38 +0900, Michael Paquier wrote: > On Fri, Jan 27, 2023 at 10:27:29PM -0800, Nathan Bossart wrote: > > Here is a work-in-progress patch set for adjusting the archive modules > > interface. Is this roughly what you had in mind? > > I have been catching up with what is

Making background psql nicer to use in tap tests

2023-01-30 Thread Andres Freund
Hi, Plenty tap tests require a background psql. But they're pretty annoying to use. I think the biggest improvement would be an easy way to run a single query and get the result of that query. Manually having to pump_until() is awkward and often leads to hangs/timeouts, instead of test failures,

Re: recovery modules

2023-01-30 Thread Nathan Bossart
On Mon, Jan 30, 2023 at 04:51:38PM +0900, Michael Paquier wrote: > Now, I find this part, where we use a double pointer to allow the > module initialization to create and give back a private area, rather > confusing, and I think that this could be bug-prone, as well. Once > you incorporate some

Re: Non-superuser subscription owners

2023-01-30 Thread Robert Haas
On Mon, Jan 30, 2023 at 1:46 PM Mark Dilger wrote: > I have a grim view of the requirement that publishers and subscribers trust > each other. Even when they do trust each other, they can firewall attacks by > acting as if they do not. I think it's OK if the CREATE PUBLICATION user doesn't

Re: postgres_fdw uninterruptible during connection establishment / ProcSignalBarrier

2023-01-30 Thread Nathan Bossart
On Mon, Jan 23, 2023 at 07:28:06PM -0800, Andres Freund wrote: > After a tiny bit further polishing, and after separately pushing a resource > leak fix for walrcv_connect(), I pushed this. My colleague Robins Tharakan (CC'd) noticed crashes when testing recent commits, and he traced it back to

Re: Non-superuser subscription owners

2023-01-30 Thread Mark Dilger
> On Jan 30, 2023, at 9:26 AM, Robert Haas wrote: > > So basically this doesn't really feel like a valid scenario to me. > We're supposed to believe that Alice is hostile to Bob, but the > superuser doesn't seem to have thought very carefully about how Bob is > supposed to defend himself

Re: MacOS: xsltproc fails with "warning: failed to load external entity"

2023-01-30 Thread Aleksander Alekseev
Hi Tom, Thanks for the feedback. > Hmm, there is no such directory on my Mac, and indeed this recipe > does not work here. I tried to transpose it to MacPorts by > substituting /opt/local/etc/xml/catalog, which does exist --- but > the recipe still doesn't work. Well, that's a bummer. > What

Re: Introduce "log_connection_stages" setting.

2023-01-30 Thread Sergey Dudoladov
Hello, hackers Thank you for the reviews. I've modified the patch to incorporate your suggestions: + flag bits are now used to encode different connection stages + failing tests are now fixed. It was not a keyword issue but rather "check_log_connection_messages" not allocating memory properly

Re: Non-superuser subscription owners

2023-01-30 Thread Mark Dilger
> On Jan 30, 2023, at 9:26 AM, Robert Haas wrote: > > First, it doesn't seem to make a lot of sense to have one person > managing the publications and someone else managing the subscriptions, > and especially if those parties are mutually untrusting. I can't think > of any real reason to set

Re: Add n_tup_newpage_upd to pg_stat table views

2023-01-30 Thread Andres Freund
Hi, On 2023-01-30 13:40:15 -0500, Corey Huinker wrote: > I must be missing something, I only see the check for running out of space, > not the check for exhausting line pointers. I agree dividing them would be > interesting. See PageGetHeapFreeSpace(), particularly the header comment and the

Re: Add n_tup_newpage_upd to pg_stat table views

2023-01-30 Thread Corey Huinker
On Fri, Jan 27, 2023 at 6:55 PM Andres Freund wrote: > Hi, > > On 2023-01-27 18:23:39 -0500, Corey Huinker wrote: > > This patch adds the n_tup_newpage_upd to all the table stat views. > > > > Just as we currently track HOT updates, it should be beneficial to track > > updates where the new

Re: Optimizing PostgreSQL with LLVM's PGO+LTO

2023-01-30 Thread Andres Freund
Hi, On 2023-01-30 10:24:02 -0700, João Paulo Labegalini de Carvalho wrote: > > What compiler / version / flags / OS did you try? > > > > I am running experiment on a machine with: > >- Intel(R) Xeon(R) Platinum 8268 CPU @ 2.90GHz >- Ubuntu 18.04.6 LTS >- LLVM/Clang 15.0.6 (build

Re: meson: Optionally disable installation of test modules

2023-01-30 Thread Andres Freund
Hi, On 2023-01-30 08:37:42 +0100, Peter Eisentraut wrote: > One open issue (IMO) with the meson build system is that it installs the > test modules under src/test/modules/ as part of a normal installation. This > is because there is no way to set up up the build system to install extra > things

Re: Non-superuser subscription owners

2023-01-30 Thread Robert Haas
On Mon, Jan 30, 2023 at 11:11 AM Mark Dilger wrote: > > On Jan 30, 2023, at 7:44 AM, Robert Haas wrote: > > > > And if we suppose that > > that already works and is safe, well then what's the case where I do > > need a run-as user? > > A) Alice publishes tables, and occasionally adds new tables

Re: Optimizing PostgreSQL with LLVM's PGO+LTO

2023-01-30 Thread João Paulo Labegalini de Carvalho
> What compiler / version / flags / OS did you try? > I am running experiment on a machine with: - Intel(R) Xeon(R) Platinum 8268 CPU @ 2.90GHz - Ubuntu 18.04.6 LTS - LLVM/Clang 15.0.6 (build from source) These are the flags I am using: CFLAGS = -O3 -fuse-ld=lld -gline-tables-only

Re: Deadlock between logrep apply worker and tablesync worker

2023-01-30 Thread vignesh C
On Mon, 30 Jan 2023 at 17:30, vignesh C wrote: > > On Mon, 30 Jan 2023 at 13:00, houzj.f...@fujitsu.com > wrote: > > > > On Monday, January 30, 2023 2:32 PM Amit Kapila > > wrote: > > > > > > On Mon, Jan 30, 2023 at 9:20 AM vignesh C wrote: > > > > > > > > On Sat, 28 Jan 2023 at 11:26, Amit

Re: pub/sub - specifying optional parameters without values.

2023-01-30 Thread Tom Lane
Peter Smith writes: > The v3 patch LGTM (just for the logical replication commands). Pushed then. regards, tom lane

Re: Making Vars outer-join aware

2023-01-30 Thread Tom Lane
Richard Guo writes: > Sorry for the delayed reply. I don't have any more review comments at > the moment, except a nitpicking one. > In optimizer/README at line 729 there is a query as > SELECT * FROM a > LEFT JOIN (SELECT * FROM b WHERE b.z = 1) ss ON (a.x = b.y) > WHERE a.x =

Re: MacOS: xsltproc fails with "warning: failed to load external entity"

2023-01-30 Thread Tom Lane
Aleksander Alekseev writes: >> I've found a solution: >> >> ``` >> export SGML_CATALOG_FILES=/usr/local/etc/xml/catalog >> export XMLLINT="xmllint --catalogs" >> export XSLTPROC="xsltproc --catalogs" >> ``` Hmm, there is no such directory on my Mac, and indeed this recipe does not work here. I

Re: Non-superuser subscription owners

2023-01-30 Thread Mark Dilger
> On Jan 30, 2023, at 7:44 AM, Robert Haas wrote: > > And if we suppose that > that already works and is safe, well then what's the case where I do > need a run-as user? A) Alice publishes tables, and occasionally adds new tables to existing publications. B) Bob manages subscriptions, and

Re: Timeline ID hexadecimal format

2023-01-30 Thread Sébastien Lardière
On 27/01/2023 15:55, Peter Eisentraut wrote: On 27.01.23 14:52, Sébastien Lardière wrote: The attached patch proposes to change the format of timelineid from %u to %X. I think your complaint has merit.  But note that if we did a change like this, then log files or reports from different

Re: JSONPath Child Operator?

2023-01-30 Thread David E . Wheeler
On Jan 30, 2023, at 08:17, Filipp Krylov wrote: >> My question: Are there plans to support square bracket syntax for JSON >> object field name strings like this? Or to update to follow the standard as >> it’s finalized? > > This syntax is a part of "jsonpath syntax extensions" patchset: >

Re: Non-superuser subscription owners

2023-01-30 Thread Robert Haas
On Fri, Jan 27, 2023 at 5:56 PM Mark Dilger wrote: > If the owner cannot modify the subscription, then the owner degenerates into > a mere "run-as" user. Note that this isn't how things work now, and even if > we disallowed owners from modifying the connection string, there would still > be

Re: cutting down the TODO list thread

2023-01-30 Thread Bruce Momjian
On Mon, Jan 30, 2023 at 01:13:45PM +0700, John Naylor wrote: > On Tue, Jan 24, 2023 at 11:57 PM Bruce Momjian wrote: > > I think we just point them at the TODO list and they will read the top > > of the list first, no?  I think you are right that we updated the top of > > the TODO but didn't

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-01-30 Thread Masahiko Sawada
On Mon, Jan 30, 2023 at 1:31 PM John Naylor wrote: > > On Sun, Jan 29, 2023 at 9:50 PM Masahiko Sawada wrote: > > > > On Sat, Jan 28, 2023 at 8:33 PM John Naylor > > wrote: > > > > The first implementation should be simple, easy to test/verify, easy to > > > understand, and easy to replace. As

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-30 Thread Masahiko Sawada
On Mon, Jan 30, 2023 at 3:23 PM houzj.f...@fujitsu.com wrote: > > On Monday, January 30, 2023 12:13 PM Peter Smith > wrote: > > > > Here are my review comments for v88-0002. > > Thanks for your comments. > > > > > == > > General > > > > 1. > > The test cases are checking the log content but

Re: Amcheck verification of GiST and GIN

2023-01-30 Thread Aleksander Alekseev
Hi Andrey, > Thanks! I also found out that there was a CI complaint about amcheck.h > not including some necessary stuff. Here's a version with a fix for > that. Thanks for the updated patchset. One little nitpick I have is that the tests cover only cases when all the checks pass successfully.

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-30 Thread Takamichi Osumi (Fujitsu)
On Monday, January 30, 2023 7:05 PM I wrote: > On Saturday, January 28, 2023 1:28 PM I wrote: > > Attached the updated patch v24. > I've conducted the rebase affected by the commit(1e8b61735c) by renaming > the GUC to logical_replication_mode accordingly, because it's utilized in the > TAP test of

Re: HOT chain validation in verify_heapam()

2023-01-30 Thread Himanshu Upadhyaya
Hi Hackers, On Sun, Jan 22, 2023 at 8:48 PM Himanshu Upadhyaya < upadhyaya.himan...@gmail.com> wrote: > > The test if (pred_in_progress || TransactionIdDidCommit(curr_xmin)) >> seems wrong to me. Shouldn't it be &&? Has this code been tested at >> all? It doesn't seem to have a test case. Some

Re: dynamic result sets support in extended query protocol

2023-01-30 Thread Alvaro Herrera
On 2022-Nov-22, Peter Eisentraut wrote: > I added tests using the new psql \bind command to test this functionality in > the extended query protocol, which showed that this got broken since I first > wrote this patch. This "blame" is on the pipeline mode in libpq patch >

Re: JSONPath Child Operator?

2023-01-30 Thread Filipp Krylov
Hi David, On 2022-11-10 21:55, David E. Wheeler wrote: My question: Are there plans to support square bracket syntax for JSON object field name strings like this? Or to update to follow the standard as it’s finalized? This syntax is a part of "jsonpath syntax extensions" patchset:

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-01-30 Thread Amit Kapila
On Fri, Jan 27, 2023 at 6:32 PM Önder Kalacı wrote: >> >> I suppose the options are: >> 1. use regular planner uniformly >> 2. use regular planner only when there's no replica identity (or >> configurable?) >> 3. only use low-level functions >> 4. keep using sequential scans for every single

RE: Perform streaming logical transactions by background workers and parallel apply

2023-01-30 Thread Hayato Kuroda (Fujitsu)
Dear Hou, Thank you for updating the patch! I checked your replies and new patch, and it seems good. Currently I have no comments Best Regards, Hayato Kuroda FUJITSU LIMITED

Re: Assertion failure in SnapBuildInitialSnapshot()

2023-01-30 Thread Masahiko Sawada
On Mon, Jan 30, 2023 at 8:30 PM Amit Kapila wrote: > > On Fri, Jan 27, 2023 at 4:31 PM Hayato Kuroda (Fujitsu) > wrote: > > > > Thank you for making the patch! I'm still considering whether this approach > > is > > correct, but I can put a comment to your patch anyway. > > > > ``` > > -

Re: Allow logical replication to copy tables in binary format

2023-01-30 Thread Bharath Rupireddy
On Mon, Jan 30, 2023 at 4:19 PM Melih Mutlu wrote: > Thanks for providing an updated patch. >> On Thu, Jan 12, 2023 at 1:53 PM Melih Mutlu wrote: >> 1. The performance numbers posted upthread [1] look impressive for the >> use-case tried, that's a 2.25X improvement or 55.6% reduction in >>

Re: generate_series for timestamptz and time zone problem

2023-01-30 Thread Przemysław Sztoch
Gurjeet Singh wrote on 30.01.2023 08:18: On Sat, Nov 12, 2022 at 10:44 AM Tom Lane wrote: However, what it shouldn't be is part of this patch. It's worth pushing it separately to have a record of that decision. I've now done that, so you'll need to rebase to remove that delta. I looked

Re: Deadlock between logrep apply worker and tablesync worker

2023-01-30 Thread vignesh C
On Mon, 30 Jan 2023 at 13:00, houzj.f...@fujitsu.com wrote: > > On Monday, January 30, 2023 2:32 PM Amit Kapila > wrote: > > > > On Mon, Jan 30, 2023 at 9:20 AM vignesh C wrote: > > > > > > On Sat, 28 Jan 2023 at 11:26, Amit Kapila wrote: > > > > > > > > One thing that looks a bit odd is that

Re: MacOS: xsltproc fails with "warning: failed to load external entity"

2023-01-30 Thread Aleksander Alekseev
Hi hackers, > I've found a solution: > > ``` > export SGML_CATALOG_FILES=/usr/local/etc/xml/catalog > export XMLLINT="xmllint --catalogs" > export XSLTPROC="xsltproc --catalogs" > ``` > > I will submit a patch for the documentation in a bit, after I'll check > it properly. PFA the patch. I

Re: Fix GUC_NO_SHOW_ALL test scenario in 003_check_guc.pl

2023-01-30 Thread Nitin Jadhav
> I kind of think this is a lot of unnecessary work. The case that is > problematic is a GUC that's marked GUC_NO_SHOW_ALL but not marked > GUC_NOT_IN_SAMPLE. There aren't any of those, and I don't think there > are likely to be any in future, because it doesn't make a lot of sense. > Why don't

Re: Assertion failure in SnapBuildInitialSnapshot()

2023-01-30 Thread Amit Kapila
On Fri, Jan 27, 2023 at 4:31 PM Hayato Kuroda (Fujitsu) wrote: > > Thank you for making the patch! I'm still considering whether this approach is > correct, but I can put a comment to your patch anyway. > > ``` > - Assert(!already_locked || LWLockHeldByMe(ProcArrayLock)); > - > - if

Re: Assertion failure in SnapBuildInitialSnapshot()

2023-01-30 Thread Amit Kapila
On Mon, Jan 30, 2023 at 11:34 AM Amit Kapila wrote: > > I have reproduced it manually. For this, I had to manually make the > debugger call ReplicationSlotsComputeRequiredXmin(false) via path > SnapBuildProcessRunningXacts()->LogicalIncreaseXminForSlot()->LogicalConfirmReceivedLocation() >

Re: Assertion failure in SnapBuildInitialSnapshot()

2023-01-30 Thread Amit Kapila
On Thu, Dec 8, 2022 at 8:17 AM Masahiko Sawada wrote: > > On Mon, Nov 21, 2022 at 4:31 PM Amit Kapila wrote: > > One idea to fix this issue is that in > ReplicationSlotsComputeRequiredXmin(), we compute the minimum xmin > while holding both ProcArrayLock and ReplicationSlotControlLock, and >

Re: MacOS: xsltproc fails with "warning: failed to load external entity"

2023-01-30 Thread Aleksander Alekseev
Hi hackers, > At this point I could use a friendly piece of advice from the community. I've found a solution: ``` export SGML_CATALOG_FILES=/usr/local/etc/xml/catalog export XMLLINT="xmllint --catalogs" export XSLTPROC="xsltproc --catalogs" ``` I will submit a patch for the documentation in a

Re: Allow logical replication to copy tables in binary format

2023-01-30 Thread Melih Mutlu
Hi Bharath, Thanks for reviewing. Bharath Rupireddy , 18 Oca 2023 Çar, 10:17 tarihinde şunu yazdı: > On Thu, Jan 12, 2023 at 1:53 PM Melih Mutlu > wrote: > 1. The performance numbers posted upthread [1] look impressive for the > use-case tried, that's a 2.25X improvement or 55.6% reduction in

Re: Generating code for query jumbling through gen_node_support.pl

2023-01-30 Thread Peter Eisentraut
On 27.01.23 03:59, Michael Paquier wrote: At the end, that would be unnoticeable for the average user, I guess, but here are the numbers I get on my laptop :) Personally, I think we do not want the two jumble methods in parallel. Maybe there are other opinions. I'm going to set this thread

  1   2   >