Re: Tablesync early exit

2022-04-04 Thread Amit Kapila
On Tue, Apr 5, 2022 at 9:37 AM Peter Smith wrote: > > On Sat, Apr 2, 2022 at 5:17 PM Amit Kapila wrote: > > > > On Fri, Apr 1, 2022 at 1:52 PM Peter Smith wrote: > > > > > > On Wed, Mar 16, 2022 at 4:07 PM Amit Kapila > > > wrote: > > > > > > I think the STATE_CATCHUP guarantees the apply

Re: logical decoding and replication of sequences

2022-04-04 Thread Amit Kapila
On Sat, Apr 2, 2022 at 5:47 AM Tomas Vondra wrote: > > On 4/1/22 17:02, Tomas Vondra wrote: > > So, I investigated this a bit more, and I wrote a couple test_decoding > isolation tests (patch attached) demonstrating the issue. Actually, I > should say "issues" because it's a bit worse than you

Re: Tablesync early exit

2022-04-04 Thread Peter Smith
On Sat, Apr 2, 2022 at 5:17 PM Amit Kapila wrote: > > On Fri, Apr 1, 2022 at 1:52 PM Peter Smith wrote: > > > > On Wed, Mar 16, 2022 at 4:07 PM Amit Kapila wrote: > > > > I think the STATE_CATCHUP guarantees the apply worker must have > > received (or tried to receive) a message. See the

Re: Showing I/O timings spent reading/writing temp buffers in EXPLAIN

2022-04-04 Thread Julien Rouhaud
On Tue, Apr 05, 2022 at 10:40:04AM +0900, Masahiko Sawada wrote: > On Tue, Apr 5, 2022 at 1:31 AM Julien Rouhaud wrote: > > > > Yes. In normal circumstances it shouldn't need a lot of time to do that, > > but > > I'm not so sure with e.g. network filesystems. I'm not strongly in favor of > >

Re: Skipping logical replication transactions on subscriber side

2022-04-04 Thread Masahiko Sawada
On Tue, Apr 5, 2022 at 10:46 AM Noah Misch wrote: > > On Tue, Apr 05, 2022 at 10:13:06AM +0900, Masahiko Sawada wrote: > > On Tue, Apr 5, 2022 at 9:21 AM Noah Misch wrote: > > > On Mon, Apr 04, 2022 at 06:55:45PM +0900, Masahiko Sawada wrote: > > > > On Mon, Apr 4, 2022 at 3:26 PM Noah Misch

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-04-04 Thread Peter Geoghegan
On Mon, Apr 4, 2022 at 8:18 PM Andres Freund wrote: > The remaining patch are the warnings in vac_update_relstats(), correct? I > guess one could argue they should be LOG rather than WARNING, but I find the > project stance on that pretty impractical. So warning's ok with me. Right. The reason

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-04-04 Thread Andres Freund
Hi, On 2022-04-04 19:32:13 -0700, Peter Geoghegan wrote: > On Fri, Apr 1, 2022 at 10:54 AM Peter Geoghegan wrote: > > I also refined the WARNING patch in v15. It now actually issues > > WARNINGs (rather than PANICs, which were just a temporary debugging > > measure in v14). > > Going to commit

Re: Window Function "Run Conditions"

2022-04-04 Thread Andres Freund
On 2022-04-05 12:04:18 +1200, David Rowley wrote: > > This is afaics slightly cheaper than referencing a variable in a slot. > > I guess you must mean cheaper because it means there will be no > EEOP_*_FETCHSOME step? Otherwise it seems a fairly similar amount of > work. That, and slightly

Re: Handle infinite recursion in logical replication setup

2022-04-04 Thread Peter Smith
On Tue, Apr 5, 2022 at 12:36 PM Amit Kapila wrote: > > On Tue, Apr 5, 2022 at 6:21 AM Peter Smith wrote: > > > > Here are my comments for the latest patch v6-0001. > > > > (I will post my v6-0002 review comments separately) > > > > PATCH v6-0001 comments > > == > > > > 1.1

Re: Temporary tables versus wraparound... again

2022-04-04 Thread Greg Stark
So here's an updated patch. I had to add a public method to multixact.c to expose the locally calculated OldestMultiXactId. It's possible we could use something even tighter (like the current next mxid since we're about to commit) but I didn't see a point in going further and it would have become

Re: Handle infinite recursion in logical replication setup

2022-04-04 Thread Peter Smith
Here are my comments for the latest patch v6-0002. PATCH v6-0002 comments == 2.1 General - should this be an independent patch? In many ways, I think most of this patch is unrelated to the other "local_only" patch (v6-0001). For example, IIUC even in the current HEAD, we

Re: shared-memory based stats collector - v68

2022-04-04 Thread Andres Freund
Hi, On 2022-04-04 19:03:13 -0700, David G. Johnston wrote: > > > (if this is true...but given this is an optimization category I'm > > thinking > > > maybe it doesn't actually matter...) > > > > It is true. Not sure what you mean with "optimization category"? > > > > > I mean that distinguishing

Re: Handle infinite recursion in logical replication setup

2022-04-04 Thread Amit Kapila
On Tue, Apr 5, 2022 at 6:21 AM Peter Smith wrote: > > Here are my comments for the latest patch v6-0001. > > (I will post my v6-0002 review comments separately) > > PATCH v6-0001 comments > == > > 1.1 General - Option name > > I still feel like the option name is not ideal.

Re: Window Function "Run Conditions"

2022-04-04 Thread Andy Fan
> > > I'm pretty happy with this now. If anyone wants to have a look at > this, can they do so or let me know they're going to within the next > 24 hours. Otherwise I plan to move into commit mode with it. > > I just came to the office today to double check this patch. I probably can finish it

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-04-04 Thread Peter Geoghegan
On Fri, Apr 1, 2022 at 10:54 AM Peter Geoghegan wrote: > I also refined the WARNING patch in v15. It now actually issues > WARNINGs (rather than PANICs, which were just a temporary debugging > measure in v14). Going to commit this remaining patch tomorrow, barring objections. -- Peter

Re: generic plans and "initial" pruning

2022-04-04 Thread Amit Langote
On Mon, Apr 4, 2022 at 9:55 PM Amit Langote wrote: > On Sun, Apr 3, 2022 at 8:33 PM Alvaro Herrera wrote: > > I think the names ExecCreatePartitionPruneState and > > ExecInitPartitionPruning are too confusingly similar. Maybe the former > > should be renamed to somehow make it clear that it is

Re: Lowering the ever-growing heap->pd_lower

2022-04-04 Thread Peter Geoghegan
On Tue, Feb 15, 2022 at 10:48 AM Matthias van de Meent wrote: > # Truncating lp_array during pruning > === > > The following adversarial load grows the heap relation, but with the > patch the relation keeps its size. The point being that HOT updates > can temporarily

Re: wrong fds used for refilenodes after pg_upgrade relfilenode changes Reply-To:

2022-04-04 Thread Thomas Munro
On Tue, Apr 5, 2022 at 10:24 AM Thomas Munro wrote: > On Tue, Apr 5, 2022 at 2:18 AM Robert Haas wrote: > > I'm not sure that it really matters, but with the idea that I proposed > > it's possible to "save" a pending writeback, if we notice that we're > > accessing the relation with a proper

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2022-04-04 Thread Kyotaro Horiguchi
At Mon, 4 Apr 2022 21:14:27 +0530, Dilip Kumar wrote in > On Mon, Apr 4, 2022 at 2:25 PM Kyotaro Horiguchi > wrote: > > > > At Mon, 04 Apr 2022 17:29:48 +0900 (JST), Kyotaro Horiguchi > > wrote in > > > I haven't found how the patch caused creation of a relation file that > > > is to be

Re: Run pg_amcheck in 002_pg_upgrade.pl and 027_stream_regress.pl?

2022-04-04 Thread Michael Paquier
On Mon, Apr 04, 2022 at 05:39:58PM -0700, Andres Freund wrote: > On my workstation it takes about 2.39s to run pg_amcheck on a regression > database with all thoroughness options enabled. With -j4 it's 0.62s. > > Without more thorough checking it's 1.24s and 0.30s with -j4. Okay. That sounds

Re: New Object Access Type hooks

2022-04-04 Thread Tom Lane
Mark Dilger writes: > On Apr 4, 2022, at 1:47 PM, Tom Lane wrote: >> Perhaps libpq should be trying harder to make those cases look alike, but >> this test is about server behavior not libpq behavior, so I'm inclined >> to just make it lax. > +1. > I've gotten this test failure only a few

Re: PATCH: add "--config-file=" option to pg_rewind

2022-04-04 Thread Michael Paquier
On Sun, Apr 03, 2022 at 09:31:32PM +0200, Michael Banck wrote: > The patch applies, make is ok, make check is ok, pg_rewind TAP tests > are ok. + appendPQExpBufferStr(postgres_cmd, " --config_file="); + appendShellString(postgres_cmd, config_file); Nit. This is a valid option for the

Re: shared-memory based stats collector - v68

2022-04-04 Thread David G. Johnston
On Mon, Apr 4, 2022 at 3:44 PM Andres Freund wrote: > Hi, > > On 2022-04-04 15:24:24 -0700, David G. Johnston wrote: > > Replacing the existing assert(!kind->fixed_amount) with > > assert(!kind->accessed_across_databases) produces the same result as the > > later presently implies the former. >

Re: Handle infinite recursion in logical replication setup

2022-04-04 Thread vignesh C
On Mon, Apr 4, 2022 at 6:50 PM Ashutosh Bapat wrote: > > I didn't find this in https://commitfest.postgresql.org/37/. Is this > somewhere in the commitfest? I missed adding it earlier, I have added it to commitfest today: https://commitfest.postgresql.org/38/3610/ Regards, Vignesh

Re: Skipping logical replication transactions on subscriber side

2022-04-04 Thread Noah Misch
On Tue, Apr 05, 2022 at 10:13:06AM +0900, Masahiko Sawada wrote: > On Tue, Apr 5, 2022 at 9:21 AM Noah Misch wrote: > > On Mon, Apr 04, 2022 at 06:55:45PM +0900, Masahiko Sawada wrote: > > > On Mon, Apr 4, 2022 at 3:26 PM Noah Misch wrote: > > > > On Mon, Apr 04, 2022 at 08:20:08AM +0530, Amit

Re: Showing I/O timings spent reading/writing temp buffers in EXPLAIN

2022-04-04 Thread Masahiko Sawada
On Tue, Apr 5, 2022 at 1:31 AM Julien Rouhaud wrote: > > On Tue, Apr 05, 2022 at 12:51:12AM +0900, Masahiko Sawada wrote: > > On Mon, Apr 4, 2022 at 1:30 PM Julien Rouhaud wrote: > > > > > > Hmm, but AFAICS the json format would be stable as the counters are always > > > shown even if zero. So

Re: Add LZ4 compression in pg_dump

2022-04-04 Thread Michael Paquier
On Fri, Apr 01, 2022 at 03:06:40PM +, gkokola...@pm.me wrote: > I understand the itch. Indeed when LZ4 is added as compression method, this > block changes slightly. I went with the minimum amount changed. Please find > in 0001 of the attached this variable renamed as $gzip_program_exist. I >

Re: Skipping logical replication transactions on subscriber side

2022-04-04 Thread Masahiko Sawada
On Tue, Apr 5, 2022 at 9:21 AM Noah Misch wrote: > > On Mon, Apr 04, 2022 at 06:55:45PM +0900, Masahiko Sawada wrote: > > On Mon, Apr 4, 2022 at 3:26 PM Noah Misch wrote: > > > On Mon, Apr 04, 2022 at 08:20:08AM +0530, Amit Kapila wrote: > > > > How about a comment like: "It has to be kept at

Re: Handle infinite recursion in logical replication setup

2022-04-04 Thread Peter Smith
Here are my comments for the latest patch v6-0001. (I will post my v6-0002 review comments separately) PATCH v6-0001 comments == 1.1 General - Option name I still feel like the option name is not ideal. Unfortunately, this is important because any name change would impact

Re: Run pg_amcheck in 002_pg_upgrade.pl and 027_stream_regress.pl?

2022-04-04 Thread Andres Freund
Hi, On 2022-04-05 08:46:06 +0900, Michael Paquier wrote: > On Sun, Apr 03, 2022 at 11:53:03AM -0700, Andres Freund wrote: > > It seems $subject would have a chance of catching some of these bugs, as > > well > > as exposing amcheck to a database with a bit more varied content? > > Makes sense

Re: Extensible Rmgr for Table AMs

2022-04-04 Thread Jeff Davis
On Mon, 2022-04-04 at 11:15 +0530, Bharath Rupireddy wrote: > 1) Why can't rmid be chosen by the extensions in sequential order > from > (129 till 255), say, to start with a columnar extension can choose > 129, another extension can register 130 and so on right? I'm not sure what you mean by

Re: Skipping logical replication transactions on subscriber side

2022-04-04 Thread Noah Misch
On Mon, Apr 04, 2022 at 06:55:45PM +0900, Masahiko Sawada wrote: > On Mon, Apr 4, 2022 at 3:26 PM Noah Misch wrote: > > On Mon, Apr 04, 2022 at 08:20:08AM +0530, Amit Kapila wrote: > > > How about a comment like: "It has to be kept at 8-byte alignment > > > boundary so as to be accessed directly

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-04-04 Thread Mark Dilger
> On Apr 4, 2022, at 5:12 PM, Tom Lane wrote: > > Wrote it already, no need for you to do it. Thanks! — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-04-04 Thread Tom Lane
Mark Dilger writes: > On Apr 4, 2022, at 2:26 PM, Tom Lane wrote: >> So I think that instead of what you've got here, we should >> (1) apply the map_old_guc_names[] mapping, which is constant >> (for any one PG release anyway) >> (2) smash to lower case >> (3) verify validity per

Re: Extensible Rmgr for Table AMs

2022-04-04 Thread Jeff Davis
On Sun, 2022-04-03 at 21:33 -0700, Andres Freund wrote: > I still think the easiest and fastest would be to just make RmgrTable > longer, > and not const. When registering, copy the caller provided struct into > the > respective RmgrData element. Yes, we'd waste a bit of space at the > end of the

Re: Window Function "Run Conditions"

2022-04-04 Thread David Rowley
Thanks for having a look at this. On Wed, 30 Mar 2022 at 11:16, Andres Freund wrote: > On 2022-03-29 15:11:52 +1300, David Rowley wrote: > > One thing which I'm not sure about with the patch is how I'm handling > > the evaluation of the runcondition in nodeWindowAgg.c. Instead of > > having

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-04-04 Thread Mark Dilger
> On Apr 4, 2022, at 2:26 PM, Tom Lane wrote: > > Thanks. As I'm working through this, I'm kind of inclined to drop > the has_parameter_privilege() variants that take an OID as object > identifier. This gets back to the fact that I don't think > pg_parameter_acl OIDs have any outside use;

Re: Run pg_amcheck in 002_pg_upgrade.pl and 027_stream_regress.pl?

2022-04-04 Thread Michael Paquier
On Sun, Apr 03, 2022 at 11:53:03AM -0700, Andres Freund wrote: > It seems $subject would have a chance of catching some of these bugs, as well > as exposing amcheck to a database with a bit more varied content? Makes sense to me to extend that. > Depending on the cost it might make sense to do

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-04-04 Thread Tatsuo Ishii
Hi Fabien, > Hello Tatsuo-san, > >> interval_start num_transactions sum_latency sum_latency_2 min_latency >> max_latency >> sum_lag sum_lag_2 min_lag max_lag skipped >> failures serialization_failures deadlock_failures retried retries > > I would suggest to reorder the last chunk to: > >

Re: shared-memory based stats collector - v67

2022-04-04 Thread Andres Freund
Hi, On 2022-03-23 10:42:03 -0700, Andres Freund wrote: > On 2022-03-23 17:27:50 +0900, Kyotaro Horiguchi wrote: > > + /* > > +* When starting with crash recovery, reset pgstat data - it might not > > be > > +* valid. Otherwise restore pgstat data. It's safe to do this here, > > +*

Re: pushdown of joinquals beyond group by/distinct on

2022-04-04 Thread David Rowley
On Tue, 5 Apr 2022 at 07:40, Arne Roland wrote: > can someone point out to me, why we don't consider pushdowns of the joinqual > for these queries beyond the distinct on? > > When the qual matches the distinct clause, it should be possible to generate > both parametrized and non parametrized

Re: shared-memory based stats collector - v68

2022-04-04 Thread Andres Freund
Hi, On 2022-04-04 15:24:24 -0700, David G. Johnston wrote: > Replacing the existing assert(!kind->fixed_amount) with > assert(!kind->accessed_across_databases) produces the same result as the > later presently implies the former. I wasn't proposing to replace, but to add... > Now I start to

Re: wrong fds used for refilenodes after pg_upgrade relfilenode changes Reply-To:

2022-04-04 Thread Thomas Munro
On Tue, Apr 5, 2022 at 2:18 AM Robert Haas wrote: > On Fri, Apr 1, 2022 at 5:03 PM Thomas Munro wrote: > > Another idea would be to call a new function DropPendingWritebacks(), > > and also tell all the SMgrRelation objects to close all their internal > > state (ie the fds + per-segment objects)

Re: shared-memory based stats collector - v68

2022-04-04 Thread David G. Johnston
On Mon, Apr 4, 2022 at 2:54 PM Andres Freund wrote: > > > As the existing function only handles functions and relations why not > just > > perform a specific Kind check for them? Generalizing to assert on > whether > > or not the function works on fixed or variable Kinds seems beyond its > >

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-04-04 Thread Andrew Dunstan
On 4/4/22 17:26, Tom Lane wrote: > Mark Dilger writes: >> [ v15 patch ] > Thanks. As I'm working through this, I'm kind of inclined to drop > the has_parameter_privilege() variants that take an OID as object > identifier. This gets back to the fact that I don't think > pg_parameter_acl OIDs

Re: shared-memory based stats collector - v68

2022-04-04 Thread Andres Freund
Hi, On 2022-04-04 14:25:57 -0700, David G. Johnston wrote: > > You mentioned this as a restriction above - I'm not seeing it as such? I'd > > like to write out stats more often in the future (e.g. in the > > checkpointer), > > but then it'd not be written out with this function... > > > > >

Re: psql - add SHOW_ALL_RESULTS option

2022-04-04 Thread Peter Eisentraut
On 02.04.22 15:26, Fabien COELHO wrote: Again, after the SendQuery refactoring extraction. I'm doing this locally, so don't feel obliged to send more of these. ;-) Good for me :-) This has been committed. I reduced some of your stylistic changes in order to keep the surface area of

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-04-04 Thread Tom Lane
Mark Dilger writes: > [ v15 patch ] Thanks. As I'm working through this, I'm kind of inclined to drop the has_parameter_privilege() variants that take an OID as object identifier. This gets back to the fact that I don't think pg_parameter_acl OIDs have any outside use; we wouldn't even have

Re: shared-memory based stats collector - v68

2022-04-04 Thread David G. Johnston
On Mon, Apr 4, 2022 at 2:06 PM Andres Freund wrote: > > > My first encounter with pg_stat_exists_stat() didn't draw my attention as > > being problematic so I'd say we just stick with it. As a SQL user > reading: > > WHERE exists (...) is somewhat natural; using "have" or back-to-back > >

Re: shared-memory based stats collector - v68

2022-04-04 Thread Andres Freund
Hi, On 2022-04-04 13:45:40 -0700, David G. Johnston wrote: > I didn't take the time to fixup all the various odd typos in the general > code comments; none of them reduced comprehension appreciably. I may do so > when/if I do another pass. Cool. > My first encounter with pg_stat_exists_stat()

Re: New Object Access Type hooks

2022-04-04 Thread Mark Dilger
> On Apr 4, 2022, at 1:47 PM, Tom Lane wrote: > > Yeah, it's plausible to get a failure on either the write or read side > depending on timing. > > Perhaps libpq should be trying harder to make those cases look alike, but > this test is about server behavior not libpq behavior, so I'm

Re: New Object Access Type hooks

2022-04-04 Thread Tom Lane
Mark Dilger writes: >> On Apr 4, 2022, at 12:05 PM, Tom Lane wrote: >> So scratch that. Maybe we'd better add "could not send data to server" >> to the regex? > If it fails in pqsecure_raw_write(), you get either "server closed the > connection unexpectedly" or "could not send data to

Re: shared-memory based stats collector - v68

2022-04-04 Thread David G. Johnston
On Sun, Apr 3, 2022 at 9:16 PM Andres Freund wrote: > > Please take a look! > > I didn't take the time to fixup all the various odd typos in the general code comments; none of them reduced comprehension appreciably. I may do so when/if I do another pass. I did skim over the entire patch set

Re: Mingw task for Cirrus CI

2022-04-04 Thread Andres Freund
Hi, On 2022-03-30 17:26:18 -0700, Andres Freund wrote: > Hi, > > On 2022-03-22 19:00:42 +0300, Melih Mutlu wrote: > > Rebased it. > > I also removed the temp installation task and > > used NoDefaultCurrentDirectoryInExePath env variable instead. > > Hm. But you're still using a separate build

Re: New Object Access Type hooks

2022-04-04 Thread Mark Dilger
> On Apr 4, 2022, at 12:05 PM, Tom Lane wrote: > > I wrote: >> The "terminating connection" warning absolutely should get through, > > ... oh, no, that's not guaranteed at all, since it's sent from quickdie(). > So scratch that. Maybe we'd better add "could not send data to server" > to the

Re: Pluggable toaster

2022-04-04 Thread Robert Haas
On Mon, Apr 4, 2022 at 4:05 PM Nikita Malakhov wrote: > - Is 'git apply' not a valid way to apply such patches? I have found that it never works. This case is no exception: [rhaas pgsql]$ git apply ~/Downloads/1_toaster_interface_v4.patch /Users/rhaas/Downloads/1_toaster_interface_v4.patch:253:

Re: Pluggable toaster

2022-04-04 Thread Nikita Malakhov
Hi, Thank you very much for your review, I'd like to get it much earlier. I'm currently working on cleaning up code, and would try to comment as much as possible. This patch set is really a large set of functionality, it was divided into 4 logically complete parts, but anyway these parts contain a

pushdown of joinquals beyond group by/distinct on

2022-04-04 Thread Arne Roland
Hi, can someone point out to me, why we don't consider pushdowns of the joinqual for these queries beyond the distinct on? When the qual matches the distinct clause, it should be possible to generate both parametrized and non parametrized subplans for the same query. The same should hold

Re: Higher level questions around shared memory stats

2022-04-04 Thread Andres Freund
Hi, On 2022-03-30 14:08:41 -0700, Andres Freund wrote: > On 2022-03-30 14:42:23 -0400, Robert Haas wrote: > > On Tue, Mar 29, 2022 at 5:01 PM Andres Freund wrote: > > > I can think of these different times: > > > > > > - Last time stats were removed due to starting up in crash recovery > > > -

Re: Pluggable toaster

2022-04-04 Thread Greg Stark
Thanks for the review Robert. I think that gives some pretty actionable advice on how to improve the patch and it doesn't seem likely to get much more in this cycle. I'll mark the patch Returned with Feedback. Hope to see it come back with improvements in the next release.

Re: shared-memory based stats collector - v68

2022-04-04 Thread Andres Freund
Hi, On 2022-04-03 21:15:16 -0700, Andres Freund wrote: > - collect who reviewed earlier revisions I found reviews by - Tomas Vondra - Arthur Zakirov - Antonin Houska There's also reviews by Fujii and Alvaro, but afaics just for parts that were separately committed. Greetings, Andres Freund

Re: New Object Access Type hooks

2022-04-04 Thread Tom Lane
I wrote: > The "terminating connection" warning absolutely should get through, ... oh, no, that's not guaranteed at all, since it's sent from quickdie(). So scratch that. Maybe we'd better add "could not send data to server" to the regex? regards, tom lane

Re: New Object Access Type hooks

2022-04-04 Thread Tom Lane
Mark Dilger writes: > # Running: pg_ctl kill QUIT 2083 > ok 4 - killed process with SIGQUIT > # pump_until: process terminated unexpectedly when searching for > "(?^m:WARNING: terminating connection because of crash of another server > process|server closed the connection

Re: New Object Access Type hooks

2022-04-04 Thread Mark Dilger
> On Apr 4, 2022, at 10:44 AM, Mark Dilger wrote: > > I'm writing a parallel test just for this. Will get back to you. Ok, that experiment didn't accomplish anything, beyond refreshing my memory regarding Cluster.pm preferring sockets over ports. — Mark Dilger EnterpriseDB:

Re: Run pg_amcheck in 002_pg_upgrade.pl and 027_stream_regress.pl?

2022-04-04 Thread Robert Haas
On Mon, Apr 4, 2022 at 2:16 PM Andres Freund wrote: > On 2022-04-04 10:02:37 -0400, Robert Haas wrote: > > It does a good job, I think, checking all the things that a human being > > could potentially spot just by looking at an individual page. > > I think there's a few more things that'd be good

Re: JSON constructors and window functions

2022-04-04 Thread Andrew Dunstan
On 4/4/22 11:43, Tom Lane wrote: > Andrew Dunstan writes: >> On 4/3/22 22:46, Andrew Dunstan wrote: >>> On 4/3/22 20:11, Andres Freund wrote: I don't think you're allowed to free stuff in a finalfunc - we might reuse the transition state for further calls to the aggregate. >>>

Re: JSON constructors and window functions

2022-04-04 Thread Andrew Dunstan
On 4/4/22 12:33, Andres Freund wrote: > Hi, > > On 2022-04-04 11:54:23 -0400, Greg Stark wrote: >> Are we missing regression tests using these functions as window functions? > So far, yes. > > ISTM that 4eb97988796 should have at least included the crashing statement as > a test... The statement

Re: Run pg_amcheck in 002_pg_upgrade.pl and 027_stream_regress.pl?

2022-04-04 Thread Andres Freund
Hi, On 2022-04-04 10:02:37 -0400, Robert Haas wrote: > It does a good job, I think, checking all the things that a human being > could potentially spot just by looking at an individual page. I think there's a few more things that'd be good to check. For example amcheck doesn't verify that HOT

Re: New Object Access Type hooks

2022-04-04 Thread Mark Dilger
> On Apr 4, 2022, at 11:07 AM, Tom Lane wrote: > > I was hoping to see regress_log_013_crash_restart, though. regress_log_013_crash_restart Description: Binary data — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: New Object Access Type hooks

2022-04-04 Thread Tom Lane
Mark Dilger writes: > The test logs are attached. Server log looks as-expected: 2022-04-04 09:11:51.087 PDT [2084] 013_crash_restart.pl LOG: statement: SELECT pg_sleep(3600); 2022-04-04 09:11:51.094 PDT [2083] 013_crash_restart.pl WARNING: terminating connection because of unexpected

Re: New Object Access Type hooks

2022-04-04 Thread Mark Dilger
> On Apr 4, 2022, at 10:51 AM, Tom Lane wrote: > > Oh, I'm barking up the wrong tree. This test must have been run > against HEAD between 6da65a3f9 (23 Feb) and 2beb4acff (31 Mar), when > pump_until indeed didn't print this essential information :-( > > If you just got this failure, could

Re: shared-memory based stats collector - v68

2022-04-04 Thread Andres Freund
Hi, On 2022-04-05 01:16:04 +1200, Thomas Munro wrote: > On Mon, Apr 4, 2022 at 4:16 PM Andres Freund wrote: > > Please take a look! > > A few superficial comments: > > > [PATCH v68 01/31] pgstat: consistent function header formatting. > > [PATCH v68 02/31] pgstat: remove some superflous

Re: New Object Access Type hooks

2022-04-04 Thread Tom Lane
I wrote: > Is our CI setup failing to capture stderr from TAP tests?? Oh, I'm barking up the wrong tree. This test must have been run against HEAD between 6da65a3f9 (23 Feb) and 2beb4acff (31 Mar), when pump_until indeed didn't print this essential information :-( If you just got this failure,

Re: New Object Access Type hooks

2022-04-04 Thread Mark Dilger
> On Apr 4, 2022, at 10:41 AM, Tom Lane wrote: > > Is our CI setup failing to capture stderr from TAP tests?? I'm looking into the way our TAP test infrastructure assigns port numbers to nodes, and whether that is reasonable during parallel test runs with nodes stopping and starting again.

Re: New Object Access Type hooks

2022-04-04 Thread Tom Lane
I wrote: > Usually the test would succeed anyway because of matching the > second or third regex alternative, but I wonder if there is > some other spelling of libpq's complaint that shows up > occasionally. It'd be nice if we could see the contents of > $killme_stderr upon failure. OK, now I'm

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

2022-04-04 Thread David Steele
On 4/4/22 11:42 AM, Nathan Bossart wrote: I noticed a couple of other things that can be removed. Since we no longer wait on exclusive backup mode during smart shutdown, we can change connsAllowed (in postmaster.c) to a boolean and remove CAC_SUPERUSER. We can also remove a couple of related

Re: New Object Access Type hooks

2022-04-04 Thread Tom Lane
Mark Dilger writes: > I just got a crash in this test again. Are you still interested? I still > have the logs. No core file appears to have been generated. > The test failure is > not ok 5 - psql query died successfully after SIGQUIT Hmm ... I can see one problem with that test: ok(

Re: A test for replay of regression tests

2022-04-04 Thread Tom Lane
I wrote: > I think the fundamental instability here is that this TAP test is > setting shared_buffers small enough to allow the syncscan logic > to kick in where it does not in normal testing. Maybe we should > just disable syncscan in this test script? Did that, we'll see how much it helps.

Re: Pluggable toaster

2022-04-04 Thread Robert Haas
On Mon, Apr 4, 2022 at 12:13 PM Nikita Malakhov wrote: > I'm really sorry. Messed up some code while merging rebased branches with > previous (v1) > patches issued in December and haven't noticed that seg fault because of > corrupted code > while running check-world. > I've fixed messed code in

Re: JSON constructors and window functions

2022-04-04 Thread Andres Freund
Hi, On 2022-04-04 11:54:23 -0400, Greg Stark wrote: > Are we missing regression tests using these functions as window functions? So far, yes. ISTM that 4eb97988796 should have at least included the crashing statement as a test... The statement can be simpler too: SELECT json_objectagg(k : v

Re: Showing I/O timings spent reading/writing temp buffers in EXPLAIN

2022-04-04 Thread Julien Rouhaud
On Tue, Apr 05, 2022 at 12:51:12AM +0900, Masahiko Sawada wrote: > On Mon, Apr 4, 2022 at 1:30 PM Julien Rouhaud wrote: > > > > Hmm, but AFAICS the json format would be stable as the counters are always > > shown even if zero. So just doing the json format first and then the text > > format

Re: Run pg_amcheck in 002_pg_upgrade.pl and 027_stream_regress.pl?

2022-04-04 Thread Mark Dilger
> On Apr 4, 2022, at 9:27 AM, Peter Geoghegan wrote: > > I'd really like it if amcheck had HOT chain verification. That's the > other area where catching bugs passively with assertions and whatnot > is clearly not good enough. I agree, and was hoping to get around to this in the postgres 15

Re: Run pg_amcheck in 002_pg_upgrade.pl and 027_stream_regress.pl?

2022-04-04 Thread Peter Geoghegan
On Mon, Apr 4, 2022 at 7:02 AM Robert Haas wrote: > Yeah, I was very excited about verify_heapam(). There is a lot more > stuff that we could check, but a lot of those things would be much > more expensive to check. If anything I understated the value of verify_heapam() with this kind of work

Re: JSON constructors and window functions

2022-04-04 Thread Andres Freund
Hi, On 2022-04-04 11:43:31 -0400, Tom Lane wrote: > Andrew Dunstan writes: > > On 4/3/22 22:46, Andrew Dunstan wrote: > >> On 4/3/22 20:11, Andres Freund wrote: > >>> I don't think you're allowed to free stuff in a finalfunc - we might > >>> reuse the > >>> transition state for further calls to

Re: New Object Access Type hooks

2022-04-04 Thread Mark Dilger
> On Mar 21, 2022, at 10:03 PM, Thomas Munro wrote: > > On Fri, Mar 18, 2022 at 4:22 PM Mark Dilger > wrote: >> (FYI, I got a test failure from src/test/recovery/t/013_crash_restart.pl >> when testing v1-0001. I'm not sure yet what that is about.) > > Doesn't look like 0001 has anything

Re: JSON constructors and window functions

2022-04-04 Thread Greg Stark
Are we missing regression tests using these functions as window functions? Hm. I suppose it's possible to write a general purpose regression test that loops over all aggregate functions and runs them as window functions and aggregates over the same data sets and compares results. At least for the

Re: Showing I/O timings spent reading/writing temp buffers in EXPLAIN

2022-04-04 Thread Masahiko Sawada
On Mon, Apr 4, 2022 at 1:30 PM Julien Rouhaud wrote: > > Hi, > > On Tue, Mar 01, 2022 at 11:35:32AM +0900, Masahiko Sawada wrote: > > On Wed, Jan 19, 2022 at 5:52 PM Julien Rouhaud wrote: > > > > > > It seems that the regression tests aren't entirely stable, per cfbot: > > >

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-04-04 Thread Mark Dilger
> On Apr 4, 2022, at 8:36 AM, Tom Lane wrote: > > Mark Dilger writes: >> If we want to backtrack to v8, that's fine. I can rebase that, port >> some of the other changes from v14 to it, and repost it as v15. > > Are you working on that? I've set aside time this week to hopefully > get

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2022-04-04 Thread Dilip Kumar
On Mon, Apr 4, 2022 at 2:25 PM Kyotaro Horiguchi wrote: > > At Mon, 04 Apr 2022 17:29:48 +0900 (JST), Kyotaro Horiguchi > wrote in > > I haven't found how the patch caused creation of a relation file that > > is to be removed soon. However, I find that v19 patch fails by maybe > > due to some

Re: JSON constructors and window functions

2022-04-04 Thread Tom Lane
Andrew Dunstan writes: > On 4/3/22 22:46, Andrew Dunstan wrote: >> On 4/3/22 20:11, Andres Freund wrote: >>> I don't think you're allowed to free stuff in a finalfunc - we might reuse >>> the >>> transition state for further calls to the aggregate. >> Doh! Of course! I'll fix it in the morning.

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

2022-04-04 Thread Nathan Bossart
I noticed a couple of other things that can be removed. Since we no longer wait on exclusive backup mode during smart shutdown, we can change connsAllowed (in postmaster.c) to a boolean and remove CAC_SUPERUSER. We can also remove a couple of related notes in the documentation. I've done all

Re: PATCH: add "--config-file=" option to pg_rewind

2022-04-04 Thread Greg Stark
Just reposting the previous patches to straighten out the cfbot. --- /usr/lib/python3/dist-packages/patroni/postgresql/__init__.py 2022-02-18 13:16:15.0 + +++ __init__.py 2022-04-03 19:17:29.952665383 + @@ -798,7 +798,8 @@ return True def get_guc_value(self, name):

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-04-04 Thread Tom Lane
Mark Dilger writes: > If we want to backtrack to v8, that's fine. I can rebase that, port > some of the other changes from v14 to it, and repost it as v15. Are you working on that? I've set aside time this week to hopefully get this over the finish line, but I don't want to find out that I've

Re: Add non-blocking version of PQcancel

2022-04-04 Thread Jelte Fennema
Hereby what I consider the final version of this patch. I don't have any changes planned myself (except for ones that come up during review). Things that changed since the previous iteration: 1. postgres_fdw now uses the non-blocking cancellation API (including test). 2. Added some extra sleeps

Re: JSON constructors and window functions

2022-04-04 Thread Andrew Dunstan
On 4/3/22 22:46, Andrew Dunstan wrote: > On 4/3/22 20:11, Andres Freund wrote: >> Hi, >> >> On 2022-04-03 18:56:39 -0400, Andrew Dunstan wrote: >>> Haven't found the issue yet :-( It happens on the second call for the >>> partition to  json_check_unique_key(). >>> >>> Here's a more idiomatic and

Re: basebackup/lz4 crash

2022-04-04 Thread Robert Haas
On Thu, Mar 31, 2022 at 3:19 AM Dipesh Pandit wrote: > Patch attached. Committed. Thanks. -- Robert Haas EDB: http://www.enterprisedb.com

Re: use has_privs_of_role() for pg_hba.conf

2022-04-04 Thread Nathan Bossart
On Mon, Apr 04, 2022 at 09:36:13AM -0400, Joshua Brindle wrote: > Good catch, I think this is a logical followup to the previous > has_privs_of_role patch. > > Reviewed and +1 Thanks! I created a commitfest entry for this: https://commitfest.postgresql.org/38/3609/ -- Nathan Bossart

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

2022-04-04 Thread Nathan Bossart
On Mon, Apr 04, 2022 at 09:56:26AM -0400, David Steele wrote: > Minor typo in the docs: > > + * capable of doing an online backup, but exclude then just in case. > > Should be: > > capable of doing an online backup, but exclude them just in case. fixed -- Nathan Bossart Amazon Web

Re: wrong fds used for refilenodes after pg_upgrade relfilenode changes Reply-To:

2022-04-04 Thread Robert Haas
On Fri, Apr 1, 2022 at 5:03 PM Thomas Munro wrote: > Another idea would be to call a new function DropPendingWritebacks(), > and also tell all the SMgrRelation objects to close all their internal > state (ie the fds + per-segment objects) but not free the main > SMgrRelationData object, and for

Re: Run pg_amcheck in 002_pg_upgrade.pl and 027_stream_regress.pl?

2022-04-04 Thread Robert Haas
On Sun, Apr 3, 2022 at 10:10 PM Peter Geoghegan wrote: > I meant to tell the authors of verify_heapam() (also CC'd) that it > really helped with my recent VACUUM project. While the assertions that > I wrote in vacuumlazy.c might catch certain bugs like this, > verify_heapam() is much more

  1   2   >