Re: Introduce a new view for checkpointer related stats

2022-11-21 Thread Drouvot, Bertrand
Hi, On 11/17/22 1:51 PM, Bharath Rupireddy wrote: Hi, pg_stat_bgwriter view currently reports checkpointer stats as well. It is that way because historically checkpointer was part of bgwriter until the commits 806a2ae and bf405ba, that went into PG 9.2, separated them out. I think it is time

Re: Bug in row_number() optimization

2022-11-21 Thread Richard Guo
On Wed, Nov 16, 2022 at 7:38 AM Sergey Shinderuk wrote: > The failing query is: > SELECT * FROM >(SELECT *, >count(salary) OVER (PARTITION BY depname || '') c1, -- w1 >row_number() OVER (PARTITION BY depname) rn, -- w2 >count(*) OVER (PARTITION BY depname)

Re: proposal: possibility to read dumped table's name from file

2022-11-21 Thread Pavel Stehule
út 22. 11. 2022 v 8:39 odesílatel Andres Freund napsal: > Hi, > > On 2022-11-13 20:32:47 +0100, Pavel Stehule wrote: > > updated patch attached > > It fails with address sanitizer that's now part of CI: > > https://cirrus-ci.com/task/6031397744279552?logs=test_world#L2659 > > [06:33:11.271] #

Re: proposal: possibility to read dumped table's name from file

2022-11-21 Thread Andres Freund
Hi, On 2022-11-13 20:32:47 +0100, Pavel Stehule wrote: > updated patch attached It fails with address sanitizer that's now part of CI: https://cirrus-ci.com/task/6031397744279552?logs=test_world#L2659 [06:33:11.271] # ==31965==ERROR: AddressSanitizer: heap-buffer-overflow on address

RE: wake up logical workers after ALTER SUBSCRIPTION

2022-11-21 Thread houzj.f...@fujitsu.com
On Tuesday, November 22, 2022 2:49 PM Hayato Kuroda (Fujitsu) > > Dear Nathan, > > > I think you are correct. I did it this way in v2. I've also moved > > the bulk of the logic to logical/worker.c. > > Thanks for updating! It becomes better. Further comments: > > 01. AlterSubscription() >

FOR UPDATE may leave spurious locks

2022-11-21 Thread Kyotaro Horiguchi
Hello. I noticed that $SUBJECT. "spurious" here means the locks on the rows that are not seemingly qualified by the query condition (that is, EPQ failure). It doesn't seem to be a bug to me (or it seems just inevitable.). But that doesn't seems to be described either in the doc. If I'm right

RE: wake up logical workers after ALTER SUBSCRIPTION

2022-11-21 Thread Takamichi Osumi (Fujitsu)
On Tuesday, November 22, 2022 1:39 PM Nathan Bossart wrote: > On Tue, Nov 22, 2022 at 03:03:52AM +, Hayato Kuroda (Fujitsu) wrote: > > Just One comment: IIUC the statement "ALTER SUBSCRIPTION" can be > > executed inside the transaction. So if two subscriptions are altered > > in the same

Re: Split index and table statistics into different types of stats

2022-11-21 Thread Drouvot, Bertrand
Hi, On 11/22/22 7:19 AM, Bharath Rupireddy wrote: On Mon, Nov 21, 2022 at 7:03 PM Drouvot, Bertrand wrote: On 11/21/22 12:19 AM, Andres Freund wrote: That's better, but still seems like quite a bit of repetition, given the number of accessors. I think I like my idea of a macro defining the

Generate pg_stat_get_* functions with Macros

2022-11-21 Thread Drouvot, Bertrand
Hi hackers, Please find attached a patch proposal to $SUBJECT. The idea has been proposed by Andres in [1] and can be seen as preparatory work for [1]. The patch introduces 2 new Macros, PGSTAT_DEFINE_REL_INT64_FIELD_ACCESSOR and PGSTAT_DEFINE_REL_TSTZ_FIELD_ACCESSOR. For some functions

RE: wake up logical workers after ALTER SUBSCRIPTION

2022-11-21 Thread Hayato Kuroda (Fujitsu)
Dear Nathan, > I think you are correct. I did it this way in v2. I've also moved the > bulk of the logic to logical/worker.c. Thanks for updating! It becomes better. Further comments: 01. AlterSubscription() ``` + LogicalRepWorkersWakeupAtCommit(subid); + ``` Currently subids will be

Re: Collation version tracking for macOS

2022-11-21 Thread Jeff Davis
On Sat, 2022-10-22 at 14:22 +1300, Thomas Munro wrote: > Problem 2: If ICU 67 ever decides to report a different version for > a > given collation (would it ever do that? I don't expect so, but ...), > we'd be unable to open the collation with the search-by-collversion > design, and

Re: Improve tab completion for ALTER FUNCTION/PROCEDURE/ROUTINE

2022-11-21 Thread vignesh C
On Tue, 22 Nov 2022 at 05:59, Michael Paquier wrote: > > On Fri, Oct 28, 2022 at 05:34:37PM +0530, vignesh C wrote: > > Those also should be handled, attached v2 version includes the changes > > for the same. > > The basic options supported by PROCEDURE are a subset of ROUTINE with a > difference

Re: Split index and table statistics into different types of stats

2022-11-21 Thread Bharath Rupireddy
On Mon, Nov 21, 2022 at 7:03 PM Drouvot, Bertrand wrote: > > On 11/21/22 12:19 AM, Andres Freund wrote: > > > > That's better, but still seems like quite a bit of repetition, given the > > number of accessors. I think I like my idea of a macro defining the whole > > function a bit better. > > > >

Re: Proposal: Allow user with pg_monitor role to call pg_stat_reset* functions

2022-11-21 Thread Bharath Rupireddy
On Tue, Nov 22, 2022 at 2:15 AM Andres Freund wrote: > > Hi, > > On 2022-11-21 00:16:20 -0800, sirisha chamarthi wrote: > > At present, calling pg_stat_reset* functions requires super user access > > unless explicitly grant execute permission on those. In this thread, I am > > proposing to grant

Re: proposal: possibility to read dumped table's name from file

2022-11-21 Thread Pavel Stehule
út 22. 11. 2022 v 6:26 odesílatel Julien Rouhaud napsal: > Hi, > > On Sun, Nov 13, 2022 at 08:32:47PM +0100, Pavel Stehule wrote: > > > > updated patch attached > > Thanks! > > Some enhancement could probably be done by a native english speaker, but > apart > from that it looks good to me, so

Re: proposal: possibility to read dumped table's name from file

2022-11-21 Thread Julien Rouhaud
Hi, On Sun, Nov 13, 2022 at 08:32:47PM +0100, Pavel Stehule wrote: > > updated patch attached Thanks! Some enhancement could probably be done by a native english speaker, but apart from that it looks good to me, so hearing no other complaints I'm marking the CF entry as Ready for Committer!

Re: Perform streaming logical transactions by background workers and parallel apply

2022-11-21 Thread Peter Smith
Thanks for addressing my review comments on v47-0001. Here are my review comments for v49-0001. == src/backend/replication/logical/applyparallelworker.c 1. GENERAL - NULL checks There is inconsistent NULL checking in the patch. Sometimes it is like (!winfo) Sometimes explicit NULL checks

Re: Partial aggregates pushdown

2022-11-21 Thread Ted Yu
On Mon, Nov 21, 2022 at 5:02 PM fujii.y...@df.mitsubishielectric.co.jp < fujii.y...@df.mitsubishielectric.co.jp> wrote: > Hi Mr.Vondra, Mr.Pyhalov, Everyone. > > I discussed with Mr.Pyhalov about the above draft by directly sending mail > to > him(outside of pgsql-hackers). Mr.Pyhalov allowed me

Re: wake up logical workers after ALTER SUBSCRIPTION

2022-11-21 Thread Nathan Bossart
On Tue, Nov 22, 2022 at 03:03:52AM +, Hayato Kuroda (Fujitsu) wrote: > Just One comment: IIUC the statement "ALTER SUBSCRIPTION" can be executed > inside the transaction. So if two subscriptions are altered in the same > transaction, only one of them will awake. Is it expected behavior? > > I

Re: perform_spin_delay() vs wait events

2022-11-21 Thread Andres Freund
Hi, On 2022-11-22 12:51:25 +0900, Michael Paquier wrote: > On Mon, Nov 21, 2022 at 07:01:18PM -0800, Andres Freund wrote: > > I plan to push this soon unless somebody has further comments. > > > @@ -146,7 +146,8 @@ typedef enum > > WAIT_EVENT_RECOVERY_RETRIEVE_RETRY_INTERVAL, > >

Re: perform_spin_delay() vs wait events

2022-11-21 Thread Michael Paquier
On Mon, Nov 21, 2022 at 07:01:18PM -0800, Andres Freund wrote: > I plan to push this soon unless somebody has further comments. > @@ -146,7 +146,8 @@ typedef enum > WAIT_EVENT_RECOVERY_RETRIEVE_RETRY_INTERVAL, > WAIT_EVENT_REGISTER_SYNC_REQUEST, > WAIT_EVENT_VACUUM_DELAY, > -

Re: ps command does not show walsender's connected db

2022-11-21 Thread Bharath Rupireddy
On Tue, Nov 22, 2022 at 6:14 AM Michael Paquier wrote: > > On Thu, Nov 17, 2022 at 01:32:11PM +0900, Ian Lawrence Barwick wrote: > > Fujii-san is marked as committer on the commifest entry for this patch [1]; > > are you able to go ahead and get it committed? > > That's the state of the patch

Re: Proposal: Allow user with pg_monitor role to call pg_stat_reset* functions

2022-11-21 Thread Julien Rouhaud
On Mon, Nov 21, 2022 at 03:47:38PM -0500, Robert Haas wrote: > On Mon, Nov 21, 2022 at 3:45 PM Andres Freund wrote: > > On 2022-11-21 00:16:20 -0800, sirisha chamarthi wrote: > > > At present, calling pg_stat_reset* functions requires super user access > > > unless explicitly grant execute

Re: [BUG] FailedAssertion in SnapBuildPurgeOlderTxn

2022-11-21 Thread Amit Kapila
On Tue, Nov 22, 2022 at 2:22 AM Andres Freund wrote: > > On 2022-11-21 15:47:12 +0300, Maxim Orlov wrote: > > After some investigation, I think, the problem is in the snapbuild.c > > (commit 272248a0c1b1, see [0]). We do allocate InitialRunningXacts > > array in the context of builder->context,

RE: wake up logical workers after ALTER SUBSCRIPTION

2022-11-21 Thread Hayato Kuroda (Fujitsu)
Hi Nathan, I have done almost same thing locally for [1], but I thought your code seemed better. Just One comment: IIUC the statement "ALTER SUBSCRIPTION" can be executed inside the transaction. So if two subscriptions are altered in the same transaction, only one of them will awake. Is it

Re: perform_spin_delay() vs wait events

2022-11-21 Thread Andres Freund
Hi, On 2022-11-22 00:03:23 +0300, Alexander Korotkov wrote: > On Tue, Nov 22, 2022 at 12:01 AM Andres Freund wrote: > > On November 21, 2022 12:58:16 PM PST, Alexander Korotkov > > wrote: > > >On Mon, Nov 21, 2022 at 2:10 AM Andres Freund wrote: > > >+1 for making a group of individual names

Re: wake up logical workers after ALTER SUBSCRIPTION

2022-11-21 Thread Nathan Bossart
On Tue, Nov 22, 2022 at 03:16:05PM +1300, Thomas Munro wrote: > Maybe a comment to explain why a single variable is enough? This crossed my mind shortly after sending my previous message. Looking closer, I see that several types of ALTER SUBSCRIPTION do not call PreventInTransactionBlock(), so a

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-21 Thread Chris Travers
On Mon, Nov 21, 2022 at 10:40 AM Pavel Borisov wrote: > > I have a very serious concern about the current patch set. as someone > who has faced transaction id wraparound in the past. > > > > I can start by saying I think it would be helpful (if the other issues > are approached reasonably) to

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-21 Thread Chris Travers
On Mon, Nov 21, 2022 at 12:25 PM Aleksander Alekseev < aleksan...@timescale.com> wrote: > Hi hackers, > > > > I have a very serious concern about the current patch set. as someone > who has faced transaction id wraparound in the past. > > > > [...] > > > > I had a similar stance when I started

Re: wake up logical workers after ALTER SUBSCRIPTION

2022-11-21 Thread Thomas Munro
On Tue, Nov 22, 2022 at 1:41 PM Nathan Bossart wrote: > On my machine, the attached patch > improved 'check-world -j8' run time by ~12 seconds (from 3min 8sec to 2min > 56 sec) and src/test/subscription test time by ~17 seconds (from 139 > seconds to 122 seconds). Nice! Maybe a comment to

Re: missing indexes in indexlist with partitioned tables

2022-11-21 Thread Andres Freund
Hi, On 2022-11-02 01:50:38 +, Arne Roland wrote: > I mainly changed the comments, the Assert and some casing. The tests have been failing for a while https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest/40/3452 https://api.cirrus-ci.com/v1/task/6190372803051520/logs/cores.log

Re: ps command does not show walsender's connected db

2022-11-21 Thread Michael Paquier
On Thu, Nov 17, 2022 at 01:32:11PM +0900, Ian Lawrence Barwick wrote: > Fujii-san is marked as committer on the commifest entry for this patch [1]; > are you able to go ahead and get it committed? That's the state of the patch since the 11th of October. Seeing the lack of activity, I propose to

wake up logical workers after ALTER SUBSCRIPTION

2022-11-21 Thread Nathan Bossart
Hi hackers, While working on avoiding unnecessary wakeups in logical/worker.c (as was done for walreceiver.c in 05a7be9), I noticed that the tests began taking much longer. This seems to be caused by the reduced frequency of calls to maybe_reread_subscription() in LogicalRepApplyLoop().

Re: Improve tab completion for ALTER FUNCTION/PROCEDURE/ROUTINE

2022-11-21 Thread Michael Paquier
On Fri, Oct 28, 2022 at 05:34:37PM +0530, vignesh C wrote: > Those also should be handled, attached v2 version includes the changes > for the same. The basic options supported by PROCEDURE are a subset of ROUTINE with a difference of COST, IMMUTABLE, [NOT] LEAKPROOF, ROWS, STABLE and VOLATILE.

Re: Error-safe user functions

2022-11-21 Thread Michael Paquier
On Mon, Nov 21, 2022 at 12:26:45AM -0500, Tom Lane wrote: > Corey Huinker writes: >> I'm making an attempt at this or something very similar to it. I don't yet >> have a patch ready. Nice to hear that. If a WIP or a proof of concept takes more than a few hours, how about beginning a new thread

Re: ubsan

2022-11-21 Thread Andres Freund
Hi, On November 21, 2022 3:42:38 PM PST, Justin Pryzby wrote: >On Mon, Nov 21, 2022 at 03:15:03PM -0800, Andres Freund wrote: >> Hi, >> >> On 2022-09-29 18:17:55 -0700, Andres Freund wrote: >> > Attached is a rebased version of this patch. Hopefully with a reasonable >> > amount of comments?

Re: ubsan

2022-11-21 Thread Justin Pryzby
On Mon, Nov 21, 2022 at 03:15:03PM -0800, Andres Freund wrote: > Hi, > > On 2022-09-29 18:17:55 -0700, Andres Freund wrote: > > Attached is a rebased version of this patch. Hopefully with a reasonable > > amount of comments? I kind of wanted to add a comment to reached_main, but > > it > > just

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-21 Thread Robert Haas
On Mon, Nov 21, 2022 at 6:17 PM Tom Lane wrote: > evidence that it's a live problem. API warts are really hard to > get rid of once instituted. Yeah, agreed. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-21 Thread Tom Lane
Robert Haas writes: > On Mon, Nov 21, 2022 at 5:15 PM Tom Lane wrote: >> I think we should content ourselves with improving the demonstrated >> case, which is where we're forced to do a lot of heap fetches due >> to lots of not-all-visible tuples. > All right. I've been bitten by this problem

Re: ubsan

2022-11-21 Thread Andres Freund
Hi, On 2022-09-29 18:17:55 -0700, Andres Freund wrote: > Attached is a rebased version of this patch. Hopefully with a reasonable > amount of comments? I kind of wanted to add a comment to reached_main, but it > just seems to end up restating the variable name... I've now pushed a version of

Re: More efficient build farm animal wakeup?

2022-11-21 Thread Magnus Hagander
On Mon, Nov 21, 2022 at 11:42 PM Andrew Dunstan wrote: > > On 2022-11-21 Mo 16:20, Magnus Hagander wrote: > > n Mon, Nov 21, 2022 at 9:58 PM Tom Lane wrote: > > > > Andrew Dunstan writes: > > > The buildfarm server now creates a companion to > > branches_of_interest.txt > > >

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-21 Thread Robert Haas
On Mon, Nov 21, 2022 at 5:15 PM Tom Lane wrote: > I think we should content ourselves with improving the demonstrated > case, which is where we're forced to do a lot of heap fetches due > to lots of not-all-visible tuples. Whether we can spend a lot of > time scanning the index without ever

Re: CI and test improvements

2022-11-21 Thread Andres Freund
Hi, On 2022-11-13 17:53:04 -0600, Justin Pryzby wrote: > On Fri, Nov 04, 2022 at 06:59:46PM -0700, Andres Freund wrote: > > > diff --git a/.cirrus.yml b/.cirrus.yml > > > index 9f2282471a9..99ac09dc679 100644 > > > --- a/.cirrus.yml > > > +++ b/.cirrus.yml > > > @@ -451,12 +451,20 @@ task: > > >

Re: More efficient build farm animal wakeup?

2022-11-21 Thread Magnus Hagander
On Mon, Nov 21, 2022 at 11:41 PM Tom Lane wrote: > Andrew Dunstan writes: > > On 2022-11-21 Mo 15:58, Tom Lane wrote: > >> But if we're trying to improve matters in this area, this doesn't seem > >> like quite the way to go. > > > Well, 5 minutes was originally chosen because it was sufficient

Re: More efficient build farm animal wakeup?

2022-11-21 Thread Andrew Dunstan
On 2022-11-21 Mo 16:20, Magnus Hagander wrote: > n Mon, Nov 21, 2022 at 9:58 PM Tom Lane wrote: > > Andrew Dunstan writes: > > The buildfarm server now creates a companion to > branches_of_interest.txt > > called branches_of_interest.json which looks like this: > > ... okay

Re: More efficient build farm animal wakeup?

2022-11-21 Thread Tom Lane
Andrew Dunstan writes: > On 2022-11-21 Mo 15:58, Tom Lane wrote: >> But if we're trying to improve matters in this area, this doesn't seem >> like quite the way to go. > Well, 5 minutes was originally chosen because it was sufficient for the > purpose for which up to now the server used its

Re: More efficient build farm animal wakeup?

2022-11-21 Thread Andrew Dunstan
On 2022-11-21 Mo 15:58, Tom Lane wrote: > Andrew Dunstan writes: >> The buildfarm server now creates a companion to branches_of_interest.txt >> called branches_of_interest.json which looks like this: > ... okay ... > >> It updates this every time it does a git fetch, currently every 5 minutes.

Re: [PATCH] Add <> support to sepgsql_restorecon

2022-11-21 Thread Joe Conway
On 11/21/22 15:57, Ted Toth wrote: In SELinux file context files you can specify <> for a file meaning you don't want restorecon to relabel it. <> is especially useful in an SELinux MLS environment when objects are created at a specific security level and you don't want restorecon to relabel

Re: More efficient build farm animal wakeup?

2022-11-21 Thread Magnus Hagander
On Mon, Nov 21, 2022 at 11:27 PM Andrew Dunstan wrote: > > On 2022-11-21 Mo 16:26, Magnus Hagander wrote: > > > > > Is there a reason this file is a list of hashes each hash with a > > single value in it? Would it make more sense if it was: > > { > > "REL_11_STABLE": "140c803723", > >

Re: More efficient build farm animal wakeup?

2022-11-21 Thread Andrew Dunstan
On 2022-11-21 Mo 16:26, Magnus Hagander wrote: > > Is there a reason this file is a list of hashes each hash with a > single value in it? Would it make more sense if it was: > { >   "REL_11_STABLE": "140c803723", >   "REL_12_STABLE": "4cbcb7ed85", >   "REL_13_STABLE": "c13667b518", >  

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-21 Thread Tom Lane
Andres Freund writes: > On 2022-11-21 16:17:56 -0500, Robert Haas wrote: >> But ... what if they're not? Could the index contain a large number of >> pages containing just 1 tuple each, or no tuples at all? If so, maybe >> we can read ten bazillion index pages trying to find each heap tuple >>

Re: CI and test improvements

2022-11-21 Thread Andres Freund
Hi, On 2022-11-19 13:18:54 -0800, Andres Freund wrote: > I'll try to repost a version of the ubsan/asan patch together with the > sanitycheck patch and see how that looks. I just pushed the prerequisite patch making UBSAN_OPTIONS work. Attached is 1) addition of SanityCheck 2) use of asan and

Re: HOT chain validation in verify_heapam()

2022-11-21 Thread Peter Geoghegan
On Mon, Nov 21, 2022 at 1:34 PM Andres Freund wrote: > Hm. But to get to that point we already need to have decided that xmax > is not a normal xid. Unhelpfully we reuse the 'xid' variable for xmax as > well: > xid = HeapTupleHeaderGetRawXmax(tuple); > > I don't really know the

Re: psql: Add command to use extended query protocol

2022-11-21 Thread Corey Huinker
On Tue, Nov 15, 2022 at 8:29 AM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > On 09.11.22 00:12, Corey Huinker wrote: > > As for the docs, they're very clear and probably sufficient as-is, but I > > wonder if we should we explicitly state that the bind-state and bind > >

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-21 Thread Andres Freund
Hi, On 2022-11-21 16:17:56 -0500, Robert Haas wrote: > But ... what if they're not? Could the index contain a large number of > pages containing just 1 tuple each, or no tuples at all? If so, maybe > we can read ten bazillion index pages trying to find each heap tuple > and still end up in

Re: HOT chain validation in verify_heapam()

2022-11-21 Thread Andres Freund
Hi, On 2022-11-20 11:58:12 -0800, Peter Geoghegan wrote: > There is code in heap_prepare_freeze_tuple() that treats a raw xmax as > "xmax_already_frozen = true", even when the raw xmax value isn't > already set to InvalidTransactionId. I'm referring to this code: > > if ( ... ) // process raw

Re: More efficient build farm animal wakeup?

2022-11-21 Thread Magnus Hagander
On Mon, Nov 21, 2022 at 9:51 PM Andrew Dunstan wrote: > > On 2022-11-20 Su 17:32, Thomas Munro wrote: > > On Sun, Nov 20, 2022 at 2:44 AM Andrew Dunstan > wrote: > >> It might not suit your use case, but one of the things I do to reduce > >> fetch load is to run a local mirror which runs > >> >

Re: More efficient build farm animal wakeup?

2022-11-21 Thread Magnus Hagander
n Mon, Nov 21, 2022 at 9:58 PM Tom Lane wrote: > Andrew Dunstan writes: > > The buildfarm server now creates a companion to branches_of_interest.txt > > called branches_of_interest.json which looks like this: > > ... okay ... > Yeah, it's not as efficient as something like long polling or web

Re: Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-21 Thread Robert Haas
On Mon, Nov 21, 2022 at 2:15 PM Tom Lane wrote: > Andres Freund writes: > > On November 21, 2022 10:44:17 AM PST, Simon Riggs > > wrote: > >> Robert, something like this perhaps? limit on both the index and the heap. > > > I don't think we should add additional code / struct members into very

Re: perform_spin_delay() vs wait events

2022-11-21 Thread Alexander Korotkov
On Tue, Nov 22, 2022 at 12:01 AM Andres Freund wrote: > On November 21, 2022 12:58:16 PM PST, Alexander Korotkov > wrote: > >On Mon, Nov 21, 2022 at 2:10 AM Andres Freund wrote: > >> On 2022-11-20 17:26:11 -0500, Robert Haas wrote: > >> > On Sun, Nov 20, 2022 at 3:43 PM Andres Freund wrote: >

Re: perform_spin_delay() vs wait events

2022-11-21 Thread Andres Freund
Hi, On November 21, 2022 12:58:16 PM PST, Alexander Korotkov wrote: >On Mon, Nov 21, 2022 at 2:10 AM Andres Freund wrote: >> On 2022-11-20 17:26:11 -0500, Robert Haas wrote: >> > On Sun, Nov 20, 2022 at 3:43 PM Andres Freund wrote: >> > > I couldn't quite decide what wait_event_type to best

Re: perform_spin_delay() vs wait events

2022-11-21 Thread Alexander Korotkov
On Mon, Nov 21, 2022 at 2:10 AM Andres Freund wrote: > On 2022-11-20 17:26:11 -0500, Robert Haas wrote: > > On Sun, Nov 20, 2022 at 3:43 PM Andres Freund wrote: > > > I couldn't quite decide what wait_event_type to best group this under? In > > > the > > > attached patch I put it under

Re: More efficient build farm animal wakeup?

2022-11-21 Thread Tom Lane
Andrew Dunstan writes: > The buildfarm server now creates a companion to branches_of_interest.txt > called branches_of_interest.json which looks like this: ... okay ... > It updates this every time it does a git fetch, currently every 5 minutes. That up-to-five-minute delay, on top of whatever

[PATCH] Add <> support to sepgsql_restorecon

2022-11-21 Thread Ted Toth
In SELinux file context files you can specify <> for a file meaning you don't want restorecon to relabel it. <> is especially useful in an SELinux MLS environment when objects are created at a specific security level and you don't want restorecon to relabel them to the wrong security level. Ted

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-21 Thread Robert Haas
On Mon, Nov 21, 2022 at 1:17 PM Andres Freund wrote: > On November 21, 2022 9:37:34 AM PST, Robert Haas > wrote: > >On Mon, Nov 21, 2022 at 12:30 PM Andres Freund wrote: > >> This can't quite be right - isn't this only applying the limit if we found > >> a > >> visible tuple? > > > >It

Re: [BUG] FailedAssertion in SnapBuildPurgeOlderTxn

2022-11-21 Thread Andres Freund
Hi, On 2022-11-21 15:47:12 +0300, Maxim Orlov wrote: > After some investigation, I think, the problem is in the snapbuild.c > (commit 272248a0c1b1, see [0]). We do allocate InitialRunningXacts > array in the context of builder->context, but for the time when we call > SnapBuildPurgeOlderTxn this

Re: More efficient build farm animal wakeup?

2022-11-21 Thread Andrew Dunstan
On 2022-11-20 Su 17:32, Thomas Munro wrote: > On Sun, Nov 20, 2022 at 2:44 AM Andrew Dunstan wrote: >> It might not suit your use case, but one of the things I do to reduce >> fetch load is to run a local mirror which runs >> >>git fetch -q --prune >> >> every 5 minutes. It also runs a git

Re: [PATCH] Allow specification of custom slot for custom nodes

2022-11-21 Thread Alexander Korotkov
On Mon, Nov 21, 2022 at 4:34 PM Pavel Borisov wrote: > The following review has been posted through the commitfest application: > make installcheck-world: tested, passed > Implements feature: tested, passed > Spec compliant: not tested > Documentation:not tested > >

Re: Reducing power consumption on idle servers

2022-11-21 Thread Laurenz Albe
On Mon, 2022-11-21 at 12:11 -0500, Tom Lane wrote: > Robert Haas writes: > > The reason that I pushed back -- not as successfully as I would have > > liked -- on the changes to pg_stop_backup / pg_start_backup is that I > > know there are people using the old method successfully, and it's not > >

Re: Proposal: Allow user with pg_monitor role to call pg_stat_reset* functions

2022-11-21 Thread Robert Haas
On Mon, Nov 21, 2022 at 3:45 PM Andres Freund wrote: > On 2022-11-21 00:16:20 -0800, sirisha chamarthi wrote: > > At present, calling pg_stat_reset* functions requires super user access > > unless explicitly grant execute permission on those. In this thread, I am > > proposing to grant execute on

Re: Proposal: Allow user with pg_monitor role to call pg_stat_reset* functions

2022-11-21 Thread Andres Freund
Hi, On 2022-11-21 00:16:20 -0800, sirisha chamarthi wrote: > At present, calling pg_stat_reset* functions requires super user access > unless explicitly grant execute permission on those. In this thread, I am > proposing to grant execute on them to users with pg_monitor role > permissions. This

Re: Understanding WAL - large amount of activity from removing data

2022-11-21 Thread Andres Freund
Hi, On 2022-11-20 19:02:12 -0700, David G. Johnston wrote: > Both of these are written to the WAL, and a record is always written > to the WAL as a self-contained unit, so the old record is full sized > in the newly written WAL. That's not really true. Normally the update record just logs the

fixing CREATEROLE

2022-11-21 Thread Robert Haas
The CREATEROLE permission is in a very bad spot right now. The biggest problem that I know about is that it allows you to trivially access the OS user account under which PostgreSQL is running, which is expected behavior for a superuser but simply wrong behavior for any other user. This is because

Re: pgsql: Prevent instability in contrib/pageinspect's regression test.

2022-11-21 Thread Tom Lane
Andres Freund writes: > On 2022-11-21 15:12:15 -0500, Tom Lane wrote: >> If I were trying to find a better fix I'd be looking for ways for >> parallel workers to be able to read the parent's temp tables. >> (Perhaps that could tie in with the blue-sky discussion we had >> the other day about

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-21 Thread Andres Freund
Hi, On 2022-11-21 15:16:46 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2022-11-21 14:21:35 -0500, Tom Lane wrote: > >> pg_resetwal does seem like a better, more useful home for this; it'd > >> allow you to adjust these numbers after initial creation which might be > >> useful. I'm not

Re: pgsql: Prevent instability in contrib/pageinspect's regression test.

2022-11-21 Thread Andres Freund
Hi, On 2022-11-21 15:12:15 -0500, Tom Lane wrote: > If I were trying to find a better fix I'd be looking for ways for > parallel workers to be able to read the parent's temp tables. > (Perhaps that could tie in with the blue-sky discussion we had > the other day about allowing autovacuum on temp

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-21 Thread Tom Lane
Andres Freund writes: > On 2022-11-21 14:21:35 -0500, Tom Lane wrote: >> pg_resetwal does seem like a better, more useful home for this; it'd >> allow you to adjust these numbers after initial creation which might be >> useful. I'm not sure how flexible it is right now in terms of where >> you

Re: pgsql: Prevent instability in contrib/pageinspect's regression test.

2022-11-21 Thread David G. Johnston
On Mon, Nov 21, 2022 at 1:12 PM Tom Lane wrote: > Andres Freund writes: > > On 2022-11-21 12:52:01 -0500, Robert Haas wrote: > >> On Mon, Nov 21, 2022 at 12:35 PM Tom Lane wrote: > >>> Why in the world is get_raw_page() marked as parallel safe? > >>> It clearly isn't, given this restriction. >

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-21 Thread Andres Freund
Hi, On 2022-11-21 14:21:35 -0500, Tom Lane wrote: > Peter Eisentraut writes: > >> To date testing database cluster wraparund was not easy as initdb has > >> always > >> inited it with default xid/mxid/mxoff. The option to specify any valid > >> xid/mxid/mxoff at cluster startup will make these

Re: pgsql: Prevent instability in contrib/pageinspect's regression test.

2022-11-21 Thread Tom Lane
Andres Freund writes: > On 2022-11-21 12:52:01 -0500, Robert Haas wrote: >> On Mon, Nov 21, 2022 at 12:35 PM Tom Lane wrote: >>> Why in the world is get_raw_page() marked as parallel safe? >>> It clearly isn't, given this restriction. > It's somewhat sad to add this restriction - I've used

Re: Cleanup: Duplicated, misplaced comment in HeapScanDescData

2022-11-21 Thread Andres Freund
Hi, On 2022-11-21 12:34:12 +0100, Matthias van de Meent wrote: > On Mon, 21 Nov 2022 at 12:12, Matthias van de Meent > wrote: > > > > Hi, > > > > I noticed that the comment on/beneath rs_numblocks in HeapScanDescData > > is duplicated above rs_strategy. I don't know if there should have > > been

Re: $1 IS NULL with unknown type

2022-11-21 Thread Tom Lane
Daniele Varrazzo writes: > The operator `IS NULL` doesn't work if the argument has unknown type. > conn.execute("select %s is null", ['foo']).fetchone() > IndeterminateDatatype: could not determine data type of parameter $1 Yeah. > It doesn't seem necessary to specify a type for an argument

Re: pgsql: Prevent instability in contrib/pageinspect's regression test.

2022-11-21 Thread Andres Freund
Hi, On 2022-11-21 12:52:01 -0500, Robert Haas wrote: > On Mon, Nov 21, 2022 at 12:35 PM Tom Lane wrote: > > I wrote: > > > Andres Freund writes: > > >> Looks like a chunk of the buildfarm doesn't like this - presumably > > >> because > > >> they use force_parallel_mode = regress. Seems ok to

$1 IS NULL with unknown type

2022-11-21 Thread Daniele Varrazzo
Hello, The operator `IS NULL` doesn't work if the argument has unknown type. In psycopg 3: >>> conn.execute("select %s is null", ['foo']).fetchone() IndeterminateDatatype: could not determine data type of parameter $1 This can get in the way of using the unknown type for strings (but

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-21 Thread Tom Lane
Peter Eisentraut writes: >> To date testing database cluster wraparund was not easy as initdb has always >> inited it with default xid/mxid/mxoff. The option to specify any valid >> xid/mxid/mxoff at cluster startup will make these things easier. > Doesn't pg_resetwal already provide that

Re: Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-21 Thread Tom Lane
Andres Freund writes: > On November 21, 2022 10:44:17 AM PST, Simon Riggs > wrote: >> Robert, something like this perhaps? limit on both the index and the heap. > I don't think we should add additional code / struct members into very common > good paths for these limits. Yeah, I don't like

Re: pgsql: Prevent instability in contrib/pageinspect's regression test.

2022-11-21 Thread Tom Lane
I wrote: > I'll check to see if any sibling functions have the same issue, > and push a patch to adjust them. > Presumably the parallel labeling has to be fixed as far back as > it's marked that way (didn't look). Maybe we should push the > test change further back too, just to exercise this.

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-21 Thread Andres Freund
Hi, On November 21, 2022 10:44:17 AM PST, Simon Riggs wrote: >Robert, something like this perhaps? limit on both the index and the heap. I don't think we should add additional code / struct members into very common good paths for these limits. I don't really understand the point of

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-21 Thread Peter Eisentraut
Question about """ Subject: [PATCH v50 5/8] Add initdb option to initialize cluster with non-standard xid/mxid/mxoff. To date testing database cluster wraparund was not easy as initdb has always inited it with default xid/mxid/mxoff. The option to specify any valid xid/mxid/mxoff at cluster

Re: Catalog_xmin is not advanced when a logical slot is lost

2022-11-21 Thread sirisha chamarthi
On Mon, Nov 21, 2022 at 10:56 AM Alvaro Herrera wrote: > On 2022-Nov-21, sirisha chamarthi wrote: > > > I have a old .partial file in the data directory to reproduce this. > > I don't think the .partial file is in itself important. But I think > this whole thing is a distraction. Yes, sorry

Re: Catalog_xmin is not advanced when a logical slot is lost

2022-11-21 Thread Alvaro Herrera
On 2022-Nov-21, sirisha chamarthi wrote: > I have a old .partial file in the data directory to reproduce this. I don't think the .partial file is in itself important. But I think this whole thing is a distraction. I managed to reproduce it eventually, by messing with the slot and WAL at

Re: Catalog_xmin is not advanced when a logical slot is lost

2022-11-21 Thread sirisha chamarthi
On Mon, Nov 21, 2022 at 10:40 AM sirisha chamarthi < sirichamarth...@gmail.com> wrote: > > > On Mon, Nov 21, 2022 at 10:11 AM Alvaro Herrera > wrote: > >> On 2022-Nov-21, sirisha chamarthi wrote: >> >> > It appears to be. wal_sender is setting restart_lsn to a valid LSN even >> > when the slot

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-21 Thread Simon Riggs
On Mon, 21 Nov 2022 at 18:17, Andres Freund wrote: > > Hi, > > On November 21, 2022 9:37:34 AM PST, Robert Haas > wrote: > >On Mon, Nov 21, 2022 at 12:30 PM Andres Freund wrote: > >> This can't quite be right - isn't this only applying the limit if we found > >> a > >> visible tuple? > > >

Re: Catalog_xmin is not advanced when a logical slot is lost

2022-11-21 Thread sirisha chamarthi
On Mon, Nov 21, 2022 at 10:11 AM Alvaro Herrera wrote: > On 2022-Nov-21, sirisha chamarthi wrote: > > > It appears to be. wal_sender is setting restart_lsn to a valid LSN even > > when the slot is invalidated. > > > postgres@pgvm:~$ /usr/local/pgsql/bin/pg_receivewal -S s1 -D . > >

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-21 Thread Andres Freund
Hi, On November 21, 2022 9:37:34 AM PST, Robert Haas wrote: >On Mon, Nov 21, 2022 at 12:30 PM Andres Freund wrote: >> This can't quite be right - isn't this only applying the limit if we found a >> visible tuple? > >It doesn't look that way to me, but perhaps I'm just too dense to see >the

Re: Catalog_xmin is not advanced when a logical slot is lost

2022-11-21 Thread Alvaro Herrera
On 2022-Nov-21, sirisha chamarthi wrote: > It appears to be. wal_sender is setting restart_lsn to a valid LSN even > when the slot is invalidated. > postgres@pgvm:~$ /usr/local/pgsql/bin/pg_receivewal -S s1 -D . > pg_receivewal: error: unexpected termination of replication stream: ERROR: >

Re: pgsql: Prevent instability in contrib/pageinspect's regression test.

2022-11-21 Thread Tom Lane
Robert Haas writes: > On Mon, Nov 21, 2022 at 12:35 PM Tom Lane wrote: >> Hmm, so the problem is: >> >> SELECT octet_length(get_raw_page('test1', 'main', 0)) AS main_0; >> ERROR: cannot access temporary tables during a parallel operation >> >> Why in the world is get_raw_page() marked as

Re: pgsql: Prevent instability in contrib/pageinspect's regression test.

2022-11-21 Thread Robert Haas
On Mon, Nov 21, 2022 at 12:35 PM Tom Lane wrote: > I wrote: > > Andres Freund writes: > >> Looks like a chunk of the buildfarm doesn't like this - presumably because > >> they use force_parallel_mode = regress. Seems ok to just force that to off > >> in > >> this test? > > > Ugh ... didn't

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-21 Thread Tom Lane
Robert Haas writes: > On Mon, Nov 21, 2022 at 12:38 PM Tom Lane wrote: >> What it's restricting is the number of heap page fetches, which >> might be good enough. We don't have a lot of visibility here >> into how many index pages were scanned before returning the next >> not-dead index entry,

Re: Damage control for planner's get_actual_variable_endpoint() runaway

2022-11-21 Thread Robert Haas
On Mon, Nov 21, 2022 at 12:38 PM Tom Lane wrote: > Andres Freund writes: > > This can't quite be right - isn't this only applying the limit if we found a > > visible tuple? > > What it's restricting is the number of heap page fetches, which > might be good enough. We don't have a lot of

  1   2   >