Re: shared-memory based stats collector - v67

2022-03-23 Thread Kyotaro Horiguchi
At Tue, 22 Mar 2022 11:56:40 +0900 (JST), Kyotaro Horiguchi wrote in > Docs.. Yeah I'll try it. This is the first cut, based on the earlier patchset. monitoring.sgml: >When using the statistics to monitor collected data, it is important I failed to read this clearly. I modified the part

Re: Assert in pageinspect with NULL pages

2022-03-23 Thread Michael Paquier
On Fri, Mar 18, 2022 at 06:43:39AM +0300, Alexander Lakhin wrote: > Hello Michael, > No, just x86_64, Ubuntu 20.04, gcc 11, valgrind 3.15. I just put that query > in page.sql and see the server abort. Bah, of course, I have missed the valgrind part of the problem. The problem is that we attempt t

Re: shared-memory based stats collector - v67

2022-03-23 Thread Kyotaro Horiguchi
At Mon, 21 Mar 2022 14:30:17 -0700, Andres Freund wrote in > > Right now we reset stats for replicas, even if we start from a shutdown > > checkpoint. That seems pretty unnecessary with this patch: > > - 0021-pgstat-wip-only-reset-pgstat-data-after-crash-re.patch > > Might raise this in another

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-23 Thread Dilip Kumar
On Tue, Mar 22, 2022 at 8:53 PM Robert Haas wrote: > > On Tue, Mar 22, 2022 at 5:00 AM Dilip Kumar wrote: > > In my previous patch mistakenly I used src_dboid instead of > > dest_dboid. Fixed in this version. For destination db I have used > > lock mode as AccessSharedLock. Logically if we se

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-23 Thread Dilip Kumar
On Wed, Mar 23, 2022 at 2:14 AM Robert Haas wrote: > > So I talked to Andres and Thomas about this and they told me that I > was right to worry about this problem. Over on the thread about "wrong > fds used for refilenodes after pg_upgrade relfilenode changes > Reply-To:" there is a plan to make u

Re: Map WAL segment files on PMEM as WAL buffers

2022-03-23 Thread Takashi Menjo
Hi Andres, Thank you for your report. I rebased and made patchset v9 attached to this email. Note that v9-0009 and v9-0010 are for those who want to pass their own Cirrus CI. Regards, Takashi On Tue, Mar 22, 2022 at 9:44 AM Andres Freund wrote: > > Hi, > > On 2022-01-20 14:55:13 +0900, Takashi

Re: ExecRTCheckPerms() and many prunable partitions

2022-03-23 Thread Alvaro Herrera
On 2022-Mar-23, Amit Langote wrote: > As the changes being made with the patch are non-trivial and the patch > hasn't been reviewed very significantly since Alvaro's comments back > in Sept 2021 which I've since addressed, I'm thinking of pushing this > one into the version 16 dev cycle. Let's no

Re: Optimize external TOAST storage

2022-03-23 Thread Dilip Kumar
On Wed, Mar 23, 2022 at 9:58 AM Michael Paquier wrote: > > On Tue, Mar 22, 2022 at 02:42:53PM -0700, Nathan Bossart wrote: > > On Tue, Mar 22, 2022 at 04:34:05PM -0400, Robert Haas wrote: > >> Then, too, I'm not very confident about the usefulness of EXTENDED, > >> EXTERNAL, and MAIN. I think it's

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-03-23 Thread Tatsuo Ishii
>> I attached the updated patch. I also fixed the following paragraph which I >> had >> forgotten to fix in the previous patch. >> >> The first seven lines report some of the most important parameter settings. >> The sixth line reports the maximum number of tries for transactions with >> seria

Re: Column Filtering in Logical Replication

2022-03-23 Thread Alvaro Herrera
On 2022-Mar-23, Amit Kapila wrote: > On Wed, Mar 23, 2022 at 12:54 AM Alvaro Herrera > wrote: > > > > On 2022-Mar-19, Tomas Vondra wrote: > > > > > @@ -174,7 +182,13 @@ ALTER PUBLICATION noinsert SET (publish = 'update, > > > delete'); > > > > > > Add some tables to the publication: > >

Re: Optimize external TOAST storage

2022-03-23 Thread Dilip Kumar
On Fri, Mar 18, 2022 at 1:35 AM Nathan Bossart wrote: > > > I guess I think we should be slightly more ambitious. One idea could be to > create a default_toast_compression_ratio GUC with a default of 0.95. This > means that, by default, a compressed attribute must be 0.95x or less of the > size

[Proposal] pg_rewind integration into core

2022-03-23 Thread RKN Sai Krishna
Hi, It's possible to have a good number of standbys (in the context of async streaming replication) as part of the client architecture. Rather than asking the client to look into the intricacies of comparing the LSN of each standby with that of primary and performing the pg_rewind, isn't it a good

Re: logical decoding and replication of sequences

2022-03-23 Thread Amit Kapila
On Tue, Mar 22, 2022 at 5:41 PM Petr Jelinek wrote: > > > On 22. 3. 2022, at 13:09, Amit Kapila wrote: > > > > On Mon, Mar 21, 2022 at 4:25 AM Tomas Vondra > > wrote: > >> > >> Hi, > >> > >> Attached is a rebased patch, addressing most of the remaining issues. > >> > > > > It appears that on the

Re: cpluspluscheck vs ICU

2022-03-23 Thread Andrew Dunstan
On 3/22/22 22:23, Andres Freund wrote: > Hi, > > On 2022-03-22 17:20:24 -0700, Andres Freund wrote: >> I was about to propose adding headerscheck / cpluspluscheck to the CI file so >> that cfbot can catch future issues. > The attached patch does so, with ICU disabled to avoid the problems discuss

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-23 Thread Robert Haas
On Wed, Mar 23, 2022 at 4:42 AM Dilip Kumar wrote: > Okay this is an interesting point. So one option is that in case of > failure while using the wal log strategy we do not remove the database > directory, because an abort transaction will take care of removing the > relation file. But then in

Re: SQL/JSON: functions

2022-03-23 Thread Justin Pryzby
At least 0002-SQL-JSON-constructors-v64.patch has an issue with nodes, per COPY_PARSE_PLAN_TREES. +ERROR: unrecognized node type: 157

Re: make MaxBackends available in _PG_init

2022-03-23 Thread Robert Haas
On Wed, Mar 23, 2022 at 12:53 AM Julien Rouhaud wrote: > Unless I'm missing something, the new situation is that the system is supposed > to prevent access to MaxBackends during s_p_l_pg_init, for reasons I totally > agree with, but without doing anything for extensions that actually need to > acc

Support isEmptyStringInfo

2022-03-23 Thread Ashutosh Bapat
Hi All, We have (StringInfo::len == 0) checks at many places. I thought it would be better to wrap that into a function isEmptyStringInfo() to make those checks more readable and also abstract the logic to check emptiness of a StringInfo. I think this will be useful to extensions outside core which

Re: Support isEmptyStringInfo

2022-03-23 Thread Robert Haas
On Wed, Mar 23, 2022 at 8:33 AM Ashutosh Bapat wrote: > We have (StringInfo::len == 0) checks at many places. I thought it > would be better to wrap that into a function isEmptyStringInfo() to > make those checks more readable and also abstract the logic to check > emptiness of a StringInfo. I thi

Re: logical decoding and replication of sequences

2022-03-23 Thread Petr Jelinek
> On 23. 3. 2022, at 12:50, Amit Kapila wrote: > > On Tue, Mar 22, 2022 at 5:41 PM Petr Jelinek > mailto:petr.jeli...@enterprisedb.com>> wrote: >> >>> On 22. 3. 2022, at 13:09, Amit Kapila wrote: >>> >>> On Mon, Mar 21, 2022 at 4:25 AM Tomas Vondra >>> wrote: Hi, Attach

Re: Skip partition tuple routing with constant partition key

2022-03-23 Thread Amit Langote
Hi Greg, On Wed, Mar 16, 2022 at 6:54 AM Greg Stark wrote: > There are a whole lot of different patches in this thread. > > However this last one https://commitfest.postgresql.org/37/3270/ > created by Amit seems like a fairly straightforward optimization that > can be evaluated on its own separa

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-23 Thread Bharath Rupireddy
On Tue, Mar 22, 2022 at 11:30 PM Andres Freund wrote: > > > > Why "CALLED ON NULL INPUT"? It doesn't make sense to call the function > > > with a > > > NULL lsn, does it? Also, that's the default, why is it restated here? > > > > pg_get_wal_records_info needed that option (if end_lsn being the >

Re: psql - add SHOW_ALL_RESULTS option

2022-03-23 Thread Fabien COELHO
Hello Peter, Attached v17 is another try. The point is to record the current status, whatever it is, buggy or not, and to update the test when libpq fixes things, whenever this is done. [...] The expected output (which passes) contains this line twice: psql::2: FATAL: terminating connect

Re: make MaxBackends available in _PG_init

2022-03-23 Thread Julien Rouhaud
On Wed, Mar 23, 2022 at 08:32:39AM -0400, Robert Haas wrote: > On Wed, Mar 23, 2022 at 12:53 AM Julien Rouhaud wrote: > > Unless I'm missing something, the new situation is that the system is > > supposed > > to prevent access to MaxBackends during s_p_l_pg_init, for reasons I totally > > agree w

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-23 Thread Dilip Kumar
On Wed, Mar 23, 2022 at 5:54 PM Robert Haas wrote: > > On Wed, Mar 23, 2022 at 4:42 AM Dilip Kumar wrote: > > Okay this is an interesting point. So one option is that in case of > > failure while using the wal log strategy we do not remove the database > > directory, because an abort transaction

Re: refactoring basebackup.c (zstd workers)

2022-03-23 Thread Robert Haas
On Tue, Mar 22, 2022 at 11:37 AM Robert Haas wrote: > On Mon, Mar 21, 2022 at 2:41 PM Dagfinn Ilmari Mannsåker > wrote: > > This is no longer the accurate. How about something like like "Specifies > > details of the chosen compression method"? > > Good catch. v5 attached. And committed. -- Rob

clean up test_rls_hooks module

2022-03-23 Thread Andrew Dunstan
Over at [1] Mark Dilger got led astray because he used src/test/modules/test_rls_hooks as a template for his new test module. It looks like that module was created with some over eager copying of another test module, but it has a couple of things wrong with it. . it isn't an extension, so the Mak

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-23 Thread Robert Haas
On Wed, Mar 23, 2022 at 9:19 AM Dilip Kumar wrote: > I think directly using smgrcreate() is a better idea instead of first > registering and then unregistering it. I have made that change in > the attached patch. After this change now we can merge creating the > MAIN_FORKNUM also in the loop be

Re: Add pg_freespacemap extension sql test

2022-03-23 Thread Fabrízio de Royes Mello
On Wed, Mar 23, 2022 at 3:05 AM Michael Paquier wrote: > > After review, I don't like much the idea of allowing concurrent > autovacuums to run in parallel of the table(s) of this test, so we'd > better disable it explicitely. Make sense. > "t1" is also a very generic name to use in a regressio

Re: clean up test_rls_hooks module

2022-03-23 Thread Tom Lane
Andrew Dunstan writes: > Here's a patch to fix those things. Seems reasonable to me, except that I think I'd leave the formatting of the OBJS macro alone. IIRC, Andres or someone went around and made all of those follow this one-file-per-line style some time ago. regards

Re: Support isEmptyStringInfo

2022-03-23 Thread Tom Lane
Robert Haas writes: > I think that the code is perfectly readable as it is and that this > change makes it less so. Yeah, after a quick look through this patch I'm unimpressed too. The new code is strictly longer, and it requires the introduction of distracting "!" and "&" operators in many place

Re: Parameter for planner estimate of recursive queries

2022-03-23 Thread Simon Riggs
On Tue, 22 Mar 2022 at 00:04, Andres Freund wrote: > On 2022-03-10 17:42:14 +, Simon Riggs wrote: > > Shall I set this as Ready For Committer? > > Currently this CF entry fails on cfbot: > https://cirrus-ci.com/task/4531771134967808?logs=test_world#L1158 > > [16:27:35.772] # Failed test 'n

Re: Use JOIN USING aliases in ruleutils.c

2022-03-23 Thread Tom Lane
Peter Eisentraut writes: > When reverse-compiling a query, ruleutils.c has some complicated code to > handle the join output columns of a JOIN USING join. There used to be > no way to qualify those columns, and so if there was a naming conflict > anywhere in the query, those output columns had

Re: New Object Access Type hooks

2022-03-23 Thread Andrew Dunstan
On 3/22/22 20:11, Andrew Dunstan wrote: > On 3/22/22 20:07, Tom Lane wrote: >> Andrew Dunstan writes: >>> On 3/22/22 18:18, Tom Lane wrote: Now, if our attitude to the OAT hooks is that we are going to sprinkle some at random and whether they are useful is someone else's problem,

Re: [PATCH] Implement INSERT SET syntax

2022-03-23 Thread Tom Lane
Justin Pryzby writes: > You have to either include the pre-requisite patches as 0001, and your patch > as > 0002 (as I'm doing now), or name your patch something other than *.diff or > *.patch, so cfbot doesn't think it's a new version of the patch to be tested. This patch has been basically ign

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-23 Thread Dilip Kumar
On Wed, Mar 23, 2022 at 7:03 PM Robert Haas wrote: > > On Wed, Mar 23, 2022 at 9:19 AM Dilip Kumar wrote: > > I think directly using smgrcreate() is a better idea instead of first > > registering and then unregistering it. I have made that change in > > the attached patch. After this change no

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-23 Thread Andres Freund
Hi, On 2022-03-23 18:49:11 +0530, Dilip Kumar wrote: > I think directly using smgrcreate() is a better idea instead of first > registering and then unregistering it. I have made that change in > the attached patch. After this change now we can merge creating the > MAIN_FORKNUM also in the loop

Re: clean up test_rls_hooks module

2022-03-23 Thread Andres Freund
Hi, On 2022-03-23 10:07:06 -0400, Tom Lane wrote: > Andrew Dunstan writes: > > Here's a patch to fix those things. > > Seems reasonable to me, except that I think I'd leave the formatting > of the OBJS macro alone. IIRC, Andres or someone went around and > made all of those follow this one-file

Re: Adding CI to our tree

2022-03-23 Thread Andres Freund
Hi, On 2022-03-22 23:14:23 -0500, Justin Pryzby wrote: > On Fri, Mar 18, 2022 at 03:45:03PM -0700, Andres Freund wrote: > > Pushed 0001, 0002. Only change I made was to add > > Thanks - is there any reason not to do the MSVC compiler warnings one, too ? Purely a lack of round tuits. IIRC I thoug

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-23 Thread Dilip Kumar
On Wed, Mar 23, 2022 at 9:13 PM Andres Freund wrote: > > Hi, > > On 2022-03-23 18:49:11 +0530, Dilip Kumar wrote: > > I think directly using smgrcreate() is a better idea instead of first > > registering and then unregistering it. I have made that change in > > the attached patch. After this ch

Re: cpluspluscheck vs ICU

2022-03-23 Thread Andres Freund
Hi, On 2022-03-23 08:19:38 -0400, Andrew Dunstan wrote: > On 3/22/22 22:23, Andres Freund wrote: > > On 2022-03-22 17:20:24 -0700, Andres Freund wrote: > >> I was about to propose adding headerscheck / cpluspluscheck to the CI file > >> so > >> that cfbot can catch future issues. > > The attached

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-23 Thread Bharath Rupireddy
On Tue, Mar 22, 2022 at 11:30 PM Andres Freund wrote: > > > > To me duplicating this much code from waldump seems like a bad idea from a > > > maintainability POV. > > > > Even if we were to put the above code from pg_walinspect and > > pg_waldump into, say, walutils.c or some other existing file,

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-23 Thread Dilip Kumar
On Wed, Mar 23, 2022 at 9:05 PM Dilip Kumar wrote: > > On Wed, Mar 23, 2022 at 7:03 PM Robert Haas wrote: > > > > On Wed, Mar 23, 2022 at 9:19 AM Dilip Kumar wrote: > > > I think directly using smgrcreate() is a better idea instead of first > > > registering and then unregistering it. I have m

Remove an unnecessary errmsg_plural in dependency.c

2022-03-23 Thread Bharath Rupireddy
Hi, It looks like the following errmsg_plural in dependency.c is unnecessary as numReportedClient > 1 always and numNotReportedClient can never be < 0. Therefore plural version of the error message is sufficient. Attached a patch to fix it. @@ -1200,10 +1200,8 @@ reportDependentObjects(const Obje

Re: freeing bms explicitly

2022-03-23 Thread Zhihong Yu
On Tue, Mar 22, 2022 at 9:04 PM Zhihong Yu wrote: > > > On Tue, Mar 22, 2022 at 8:45 PM Amit Kapila > wrote: > >> On Tue, Mar 22, 2022 at 3:39 AM Zhihong Yu wrote: >> > >> > On Mon, Mar 21, 2022 at 3:05 PM Tom Lane wrote: >> >> >> >> Zhihong Yu writes: >> >> >> I was looking at calls to bms_f

Re: Remove an unnecessary errmsg_plural in dependency.c

2022-03-23 Thread Peter Eisentraut
On 23.03.22 17:33, Bharath Rupireddy wrote: It looks like the following errmsg_plural in dependency.c is unnecessary as numReportedClient > 1 always and numNotReportedClient can never be < 0. Therefore plural version of the error message is sufficient. Attached a patch to fix it. Some languages

Re: Patch to avoid orphaned dependencies

2022-03-23 Thread Tom Lane
Nathan Bossart writes: > Bertand, do you think this has any chance of making it into v15? If not, > are you alright with adjusting the commitfest entry to v16 and moving it to > the next commitfest? I looked this over briefly, and IMO it should have no chance of being committed in anything like

Re: Skip partition tuple routing with constant partition key

2022-03-23 Thread Zhihong Yu
On Wed, Mar 23, 2022 at 5:52 AM Amit Langote wrote: > Hi Greg, > > On Wed, Mar 16, 2022 at 6:54 AM Greg Stark wrote: > > There are a whole lot of different patches in this thread. > > > > However this last one https://commitfest.postgresql.org/37/3270/ > > created by Amit seems like a fairly str

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-23 Thread Dilip Kumar
On Wed, Mar 23, 2022 at 9:25 PM Dilip Kumar wrote: > > On Wed, Mar 23, 2022 at 9:13 PM Andres Freund wrote: > > > > Hi, > > > > On 2022-03-23 18:49:11 +0530, Dilip Kumar wrote: > > > I think directly using smgrcreate() is a better idea instead of first > > > registering and then unregistering it.

Re: pgsql: Unbreak the build.

2022-03-23 Thread Robert Haas
On Wed, Mar 23, 2022 at 12:03 PM Robert Haas wrote: > I'm looking into this now, but that's not the same Assert(false). The > one Andres is talking about is at the end of get_bc_algorithm_name(). > This one is in tar_open_for_write(). AFAIK, the first of these is > actually unreachable or at least

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-23 Thread Andres Freund
Hi, On 2022-03-23 22:29:40 +0530, Dilip Kumar wrote: > I could not see any reason for it to fail, and I could not reproduce > it either. Is it possible to access the server log for this cfbot > failure? Yes, near the top, below the cpu / memory graphs, there's a file navigator. Should have all f

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-23 Thread Dilip Kumar
On Wed, Mar 23, 2022 at 10:37 PM Andres Freund wrote: > > Hi, > > On 2022-03-23 22:29:40 +0530, Dilip Kumar wrote: > > I could not see any reason for it to fail, and I could not reproduce > > it either. Is it possible to access the server log for this cfbot > > failure? > > Yes, near the top, bel

Re: [BUG] Panic due to incorrect missingContrecPtr after promotion

2022-03-23 Thread alvhe...@alvh.no-ip.org
On 2022-Mar-07, Imseih (AWS), Sami wrote: > I have gone ahead and backpatched this all the way to 10 as well. Thanks! I pushed this now. I edited the test though: I don't understand why you went to the trouble of setting stuff in order to call 'pg_ctl promote' (in different ways for older branc

ubsan

2022-03-23 Thread Andres Freund
Hi, I tried to run postgres with ubsan to debug something. I ran into two main issues: 1) Despite Tom's recent efforts in [1], I see two ubsan failures in HEAD. These are easy enough to fix as per the attached, although the fix for the GetConfigOptionByNum() isn't great - we should pro

Re: Parameter for planner estimate of recursive queries

2022-03-23 Thread Tom Lane
Robert Haas writes: > On Tue, Jan 25, 2022 at 4:44 AM Peter Eisentraut > wrote: >> On the one hand, this smells like a planner hint. But on the other >> hand, it doesn't look like we will come up with proper graph-aware >> selectivity estimation system any time soon, so just having all graph >>

Re: shared-memory based stats collector - v67

2022-03-23 Thread Andres Freund
Hi, On 2022-03-23 17:27:50 +0900, Kyotaro Horiguchi wrote: > At Mon, 21 Mar 2022 14:30:17 -0700, Andres Freund wrote > in > > > Right now we reset stats for replicas, even if we start from a shutdown > > > checkpoint. That seems pretty unnecessary with this patch: > > > - 0021-pgstat-wip-only-r

Re: ubsan

2022-03-23 Thread Tom Lane
Andres Freund writes: > I tried to run postgres with ubsan to debug something. For 0001, could we just replace configure's dlopen check with the dlsym check? Or are you afraid of reverse-case failures? 0002: ugh, but my only real complaint is that __ubsan_default_options needs more than zero co

Re: Parameter for planner estimate of recursive queries

2022-03-23 Thread Simon Riggs
On Wed, 23 Mar 2022 at 17:36, Tom Lane wrote: > > Robert Haas writes: > > On Tue, Jan 25, 2022 at 4:44 AM Peter Eisentraut > > wrote: > >> On the one hand, this smells like a planner hint. But on the other > >> hand, it doesn't look like we will come up with proper graph-aware > >> selectivity

Re: Reducing power consumption on idle servers

2022-03-23 Thread Simon Riggs
On Tue, 22 Mar 2022 at 00:54, Andres Freund wrote: > > On 2022-02-21 21:04:19 +, Simon Riggs wrote: > > On Mon, 21 Feb 2022 at 16:49, Chapman Flack wrote: > > > > > Shouldn't the comment be "with work_done=false" ? > > > > Good catch, thanks. > > > > I've also added docs to say that "promote_

Re: ubsan

2022-03-23 Thread Andres Freund
Hi, On 2022-03-23 13:54:50 -0400, Tom Lane wrote: > Andres Freund writes: > > I tried to run postgres with ubsan to debug something. > > For 0001, could we just replace configure's dlopen check with the > dlsym check? Or are you afraid of reverse-case failures? Yea, I was worried about that. B

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-03-23 Thread Tom Lane
Tatsuo Ishii writes: > The patch Pushed. Thank you! My hoary animal prairiedog doesn't like this [1]: # Failed test 'concurrent update with retrying stderr /(?s-xim:client (0|1) got an error in command 3 \\(SQL\\) of script 0; ERROR: could not serialize access due to concurrent update\\b.*\

Re: prevent immature WAL streaming

2022-03-23 Thread Alvaro Herrera
On 2021-Sep-03, Alvaro Herrera wrote: > The last commit is something I noticed in pg_rewind ... I had missed this one; it's pushed now. -- Álvaro HerreraBreisgau, Deutschland — https://www.EnterpriseDB.com/ "I can see support will not be a problem. 10 out of 10."(Simon Wittber)

Re: speed up a logical replica setup

2022-03-23 Thread Peter Eisentraut
On 18.03.22 23:34, Andrew Dunstan wrote: On 3/15/22 09:51, Peter Eisentraut wrote: On 21.02.22 13:09, Euler Taveira wrote: A new tool called pg_subscriber does this conversion and is tightly integrated with Postgres. Are we comfortable with the name pg_subscriber?  It seems too general. Are w

Re: warn if GUC set to an invalid shared library

2022-03-23 Thread Tom Lane
Maciek Sakrejda writes: > In v4, the message looks fine to me for shared_preload_libraries > (except there is a doubled "is"). However, I also get the message for > a simple SET with local_preload_libraries: > postgres=# set local_preload_libraries=xyz; > WARNING: could not access file "xyz" > H

Re: ubsan

2022-03-23 Thread Andres Freund
Hi, On 2022-03-23 11:21:37 -0700, Andres Freund wrote: > On 2022-03-23 13:54:50 -0400, Tom Lane wrote: > > Andres Freund writes: > > > I tried to run postgres with ubsan to debug something. > > > > For 0001, could we just replace configure's dlopen check with the > > dlsym check? Or are you afr

Re: Parameter for planner estimate of recursive queries

2022-03-23 Thread Simon Riggs
On Wed, 23 Mar 2022 at 18:20, Simon Riggs wrote: > [New patch version pending] -- Simon Riggshttp://www.EnterpriseDB.com/ recursive_worktable_estimate.v3.patch Description: Binary data

Re: SQL/JSON: functions

2022-03-23 Thread Andrew Dunstan
On 3/23/22 08:24, Justin Pryzby wrote: > At least 0002-SQL-JSON-constructors-v64.patch has an issue with nodes, > per COPY_PARSE_PLAN_TREES. > > +ERROR: unrecognized node type: 157 I just tried to reproduce this and was unable to.  Ubuntu 20.04, gcc 9.4.0. the build was done with CPPFLAGS=-

Re: Documenting when to retry on serialization failure

2022-03-23 Thread Tom Lane
Simon Riggs writes: > I've tried to sum up the various points from everybody into this doc > patch. Thanks all for replies. This seemed rather badly in need of copy-editing. How do you like the attached text? regards, tom lane diff --git a/doc/src/sgml/mvcc.sgml b/doc/s

Re: ubsan

2022-03-23 Thread Tom Lane
Andres Freund writes: > I think we should backpatch both, based on the reasoning in > 46ab07ffda9d6c8e63360ded2d4568aa160a7700 ? Yeah, I suppose. Is anyone going to step up and run a buildfarm member with ubsan enabled? (I'm already checking -fsanitize=alignment on longfin, but it seems advisab

Re: Proposal: Support custom authentication methods using hooks

2022-03-23 Thread Peter Eisentraut
On 15.03.22 20:27, samay sharma wrote: This patch-set adds the following: * Allow multiple custom auth providers to be registered (Addressing feedback from Aleksander and Andrew) * Modify the test extension to use SCRAM to exchange secrets (Based on Andres's suggestion) * Add support for custo

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-23 Thread Peter Geoghegan
On Sun, Mar 13, 2022 at 9:05 PM Peter Geoghegan wrote: > Attached is v10. While this does still include the freezing patch, > it's not in scope for Postgres 15. As I've said, I still think that it > makes sense to maintain the patch series with the freezing stuff, > since it's structurally related

Re: ubsan

2022-03-23 Thread Andres Freund
Hi, On 2022-03-23 15:58:09 -0400, Tom Lane wrote: > Andres Freund writes: > > I think we should backpatch both, based on the reasoning in > > 46ab07ffda9d6c8e63360ded2d4568aa160a7700 ? > > Yeah, I suppose. Is anyone going to step up and run a buildfarm > member with ubsan enabled? (I'm already

Re: SQL/JSON: functions

2022-03-23 Thread Andrew Dunstan
On 3/23/22 15:49, Andrew Dunstan wrote: > On 3/23/22 08:24, Justin Pryzby wrote: >> At least 0002-SQL-JSON-constructors-v64.patch has an issue with nodes, >> per COPY_PARSE_PLAN_TREES. >> >> +ERROR: unrecognized node type: 157 > > > I just tried to reproduce this and was unable to.  Ubuntu 20.04

Re: Parameter for planner estimate of recursive queries

2022-03-23 Thread Tom Lane
Simon Riggs writes: >> [New patch version pending] Do you have any objection if I rename the GUC to recursive_worktable_factor? That seems a bit clearer as to what it does, and it leaves more room for other knobs in the same area if we decide we need any. regards, tom la

multithreaded zstd backup compression for client and server

2022-03-23 Thread Robert Haas
[ Changing subject line in the hopes of attracting more eyeballs. ] On Mon, Mar 14, 2022 at 12:11 PM Dipesh Pandit wrote: > I tried to implement support for parallel ZSTD compression. Here's a new patch for this. It's more of a rewrite than an update, honestly; commit ffd53659c46a54a6978bcb8c442

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-23 Thread Robert Haas
On Wed, Mar 23, 2022 at 3:59 PM Peter Geoghegan wrote: > In other words, since DISABLE_PAGE_SKIPPING doesn't *consistently* > force lazy_scan_noprune to refuse to process a page on HEAD (it all > depends on FreezeLimit/vacuum_freeze_min_age), it is logical for > DISABLE_PAGE_SKIPPING to totally ge

Re: MDAM techniques and Index Skip Scan patch

2022-03-23 Thread Dmitry Dolgov
> On Tue, Mar 22, 2022 at 04:55:49PM -0400, Tom Lane wrote: > Peter Geoghegan writes: > > Like many difficult patches, the skip scan patch is not so much > > troubled by problems with the implementation as it is troubled by > > *ambiguity* about the design. Particularly concerning how skip scan >

Re: [PATCH] Remove workarounds to format [u]int64's

2022-03-23 Thread Peter Eisentraut
On 21.03.22 15:37, Aleksander Alekseev wrote: It would not simplify things for them at all, just mess it up. The master copies of the .po files are kept in a different repo. Also, I believe that extraction of new message strings is automated already. Got it, thanks. Here is the corrected patch.

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-23 Thread Peter Geoghegan
On Wed, Mar 23, 2022 at 1:41 PM Robert Haas wrote: > It seems to me that if DISABLE_PAGE_SKIPPING doesn't completely > disable skipping pages, we have a problem. It depends on how you define skipping. DISABLE_PAGE_SKIPPING was created at a time when a broader definition of skipping made a lot mor

Re: Adding CI to our tree

2022-03-23 Thread Thomas Munro
On Thu, Mar 10, 2022 at 9:37 AM Justin Pryzby wrote: > -Og Adding this to CXXFLAGS caused a torrent of warnings from g++ about LLVM headers, which I also see on my local system for LLVM 11 and LLVM 14: [19:47:11.047] /usr/lib/llvm-11/include/llvm/ADT/Twine.h: In member function ‘llvm::CallInst*

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-23 Thread Robert Haas
On Wed, Mar 23, 2022 at 4:49 PM Peter Geoghegan wrote: > On Wed, Mar 23, 2022 at 1:41 PM Robert Haas wrote: > > It seems to me that if DISABLE_PAGE_SKIPPING doesn't completely > > disable skipping pages, we have a problem. > > It depends on how you define skipping. DISABLE_PAGE_SKIPPING was > cre

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-03-23 Thread Peter Eisentraut
On 21.03.22 05:55, Thomas Munro wrote: [04:30:50.630] pg_waldump.c:963:26: error: format ‘%u’ expects argument of type ‘unsigned int *’, but argument 3 has type ‘ForkNumber *’ [-Werror=format=] [04:30:50.630] 963 | if (sscanf(optarg, "%u", &config.filter_by_relation_forknum) != 1 || [04:30:50.630

Re: SQL/JSON: functions

2022-03-23 Thread Justin Pryzby
On Wed, Mar 23, 2022 at 03:49:17PM -0400, Andrew Dunstan wrote: > > On 3/23/22 08:24, Justin Pryzby wrote: > > At least 0002-SQL-JSON-constructors-v64.patch has an issue with nodes, > > per COPY_PARSE_PLAN_TREES. > > > > +ERROR: unrecognized node type: 157 > > I just tried to reproduce this and

Re: ExecRTCheckPerms() and many prunable partitions

2022-03-23 Thread Zhihong Yu
On Wed, Mar 23, 2022 at 12:03 AM Amit Langote wrote: > On Mon, Mar 14, 2022 at 4:36 PM Amit Langote > wrote: > > Also needed fixes when rebasing. > > Needed another rebase. > > As the changes being made with the patch are non-trivial and the patch > hasn't been reviewed very significantly since

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-23 Thread Peter Geoghegan
On Wed, Mar 23, 2022 at 1:53 PM Robert Haas wrote: > I see what you mean about it depending on how you define "skipping". > But I think that DISABLE_PAGE_SKIPPING is intended as a sort of > emergency safeguard when you really, really don't want to leave > anything out. I agree. > And therefore I

Re: Schema variables - new implementation for Postgres 15

2022-03-23 Thread Pavel Stehule
Hi A bit more work seems to be needed for deparsing session variables: > > # create variable myvar text; > CREATE VARIABLE > > # create view myview as select myvar; > CREATE VIEW > > # \d+ myview > View "public.myview" > Column | Type | Collation | Nullable | Default |

Re: Adding CI to our tree

2022-03-23 Thread Justin Pryzby
On Thu, Mar 24, 2022 at 09:52:39AM +1300, Thomas Munro wrote: > On Thu, Mar 10, 2022 at 9:37 AM Justin Pryzby wrote: > > -Og > > Adding this to CXXFLAGS caused a torrent of warnings from g++ about > LLVM headers, which I also see on my local system for LLVM 11 and LLVM > 14: Yes, I mentioned see

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-23 Thread Thomas Munro
On Thu, Mar 24, 2022 at 9:59 AM Peter Geoghegan wrote: > On Wed, Mar 23, 2022 at 1:53 PM Robert Haas wrote: > > And therefore I favor defining it to mean that we don't > > skip any work at all. > > But even today DISABLE_PAGE_SKIPPING won't do pruning when we cannot > acquire a cleanup lock on a

Re: [PATCH] Remove workarounds to format [u]int64's

2022-03-23 Thread Pavel Borisov
> > On 21.03.22 15:37, Aleksander Alekseev wrote: > >> It would not simplify things for them at all, just mess it up. > >> The master copies of the .po files are kept in a different repo. > >> Also, I believe that extraction of new message strings is automated > >> already. > > > > Got it, thanks.

Re: multithreaded zstd backup compression for client and server

2022-03-23 Thread Andres Freund
Hi, On 2022-03-23 16:34:04 -0400, Robert Haas wrote: > Therefore, I think it might be safe to just ... turn this on. One reason I > think that is that this whole approach was recommended to me by Andres ... I didn't do a super careful analysis of the issues... But I do think it's pretty much the

Re: logical replication restrictions

2022-03-23 Thread Euler Taveira
On Mon, Mar 21, 2022, at 10:09 PM, Euler Taveira wrote: > On Mon, Mar 21, 2022, at 10:04 PM, Andres Freund wrote: >> On 2022-03-20 21:40:40 -0300, Euler Taveira wrote: >> > On Mon, Feb 28, 2022, at 9:18 PM, Euler Taveira wrote: >> > > Long time, no patch. Here it is. I will provide documentation in

Re: Proposal: allow database-specific role memberships

2022-03-23 Thread Kenaniah Cerny
Thanks Andres, An updated patch is attached. - Kenaniah On Mon, Mar 21, 2022 at 5:40 PM Andres Freund wrote: > Hi, > > On 2022-01-22 22:56:44 +0800, Julien Rouhaud wrote: > > On Sat, Jan 22, 2022 at 05:28:05AM -0800, Kenaniah Cerny wrote: > > > Thank you so much for the backtrace! > > > > > >

Re: [PoC] Let libpq reject unexpected authentication requests

2022-03-23 Thread Jacob Champion
On Mon, 2022-03-07 at 11:44 +0100, Laurenz Albe wrote: > I am all for the idea, but you implemented the reverse of proposal 2. (This email was caught in my spam filter; sorry for the delay.) > Wouldn't it be better to list the *rejected* authentication methods? > Then we could have "password" on

Re: MDAM techniques and Index Skip Scan patch

2022-03-23 Thread Tom Lane
Dmitry Dolgov <9erthali...@gmail.com> writes: > On Tue, Mar 22, 2022 at 04:55:49PM -0400, Tom Lane wrote: >> In short: I would throw out just about all the planner infrastructure >> that's been proposed so far. It looks bulky, expensive, and >> drastically undercommented, and I don't think it's bu

Re: multithreaded zstd backup compression for client and server

2022-03-23 Thread Justin Pryzby
+* We check for failure here because (1) older versions of the library +* do not support ZSTD_c_nbWorkers and (2) the library might want to +* reject an unreasonable values (though in practice it does not seem to do +* so). +*/ + ret = ZSTD_CCtx_setPar

Re: MDAM techniques and Index Skip Scan patch

2022-03-23 Thread Dmitry Dolgov
> On Wed, Mar 23, 2022 at 05:32:46PM -0400, Tom Lane wrote: > Dmitry Dolgov <9erthali...@gmail.com> writes: > > On Tue, Mar 22, 2022 at 04:55:49PM -0400, Tom Lane wrote: > >> In short: I would throw out just about all the planner infrastructure > >> that's been proposed so far. It looks bulky, exp

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-23 Thread Peter Geoghegan
On Wed, Mar 23, 2022 at 2:03 PM Thomas Munro wrote: > Yeah, I found it confusing that DISABLE_PAGE_SKIPPING doesn't disable > all page skipping, so 3414099c turned out to be not enough. The proposed change to DISABLE_PAGE_SKIPPING is partly driven by that, and partly driven by a similar concern a

Re: multithreaded zstd backup compression for client and server

2022-03-23 Thread Robert Haas
On Wed, Mar 23, 2022 at 5:14 PM Andres Freund wrote: > The most likely source of problem would errors thrown while zstd threads are > alive. Should make sure that that can't happen. > > What is the lifetime of the threads zstd spawns? Are they tied to a single > compression call? A single ZSTD_cre

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2022-03-23 Thread Peter Eisentraut
On 23.03.22 10:51, Maxim Orlov wrote: Thanks for updating the patchs. I have a little comment on 0002. +                                errmsg_internal("found xmax %llu" " (infomask 0x%04x) not frozen, not multi, not normal", +

Re: Proposal: allow database-specific role memberships

2022-03-23 Thread Kenaniah Cerny
Hi all, cfbot is once again green as of the v7 patch: https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest/37/3374 - Kenaniah

  1   2   >