Re: Assertion failure with replication origins and PREPARE TRANSACTIOn

2021-12-12 Thread Masahiko Sawada
On Mon, Dec 13, 2021 at 12:44 PM Michael Paquier wrote: > > Hi all, > (CCing some folks who worked on this area lately) > > The following sequence of commands generates an assertion failure, as > of $subject: > select pg_replication_origin_create('popo'); > select

Re: parallel vacuum comments

2021-12-12 Thread Masahiko Sawada
On Mon, Dec 13, 2021 at 2:09 PM Amit Kapila wrote: > > On Mon, Dec 13, 2021 at 10:33 AM Masahiko Sawada > wrote: > > > > On Fri, Dec 10, 2021 at 9:08 PM Amit Kapila wrote: > > > > > > On Thu, Dec 9, 2021 at 6:05 PM Masahiko Sawada > > > wrote: > > > > > > > > On Thu, Dec 9, 2021 at 7:44 PM

Re: Add client connection check during the execution of the query

2021-12-12 Thread Thomas Munro
On Mon, Dec 13, 2021 at 5:51 PM Thomas Munro wrote: > [...] Everywhere else calls > with nevents == 1, so that's hypothetical. Erm, I forgot about ExecAppendAsyncEventWait(), so I'd have to update the commit message on that point, but it's hard to worry too much about that case -- it's creating

Re: parallel vacuum comments

2021-12-12 Thread Amit Kapila
On Mon, Dec 13, 2021 at 10:33 AM Masahiko Sawada wrote: > > On Fri, Dec 10, 2021 at 9:08 PM Amit Kapila wrote: > > > > On Thu, Dec 9, 2021 at 6:05 PM Masahiko Sawada > > wrote: > > > > > > On Thu, Dec 9, 2021 at 7:44 PM Amit Kapila > > > wrote: > > > > > > Agreed with the above two points. >

Re: parallel vacuum comments

2021-12-12 Thread Masahiko Sawada
On Fri, Dec 10, 2021 at 9:08 PM Amit Kapila wrote: > > On Thu, Dec 9, 2021 at 6:05 PM Masahiko Sawada wrote: > > > > On Thu, Dec 9, 2021 at 7:44 PM Amit Kapila wrote: > > > > Agreed with the above two points. > > > > I've attached updated patches that incorporated the above comments > > too.

Re: parallel vacuum comments

2021-12-12 Thread Masahiko Sawada
On Mon, Dec 13, 2021 at 12:03 PM Amit Kapila wrote: > > On Sat, Dec 11, 2021 at 8:30 PM Masahiko Sawada wrote: > > > > On Sat, Dec 11, 2021 at 2:32 PM Andres Freund wrote: > > > > > > Hi, > > > > > > On 2021-10-30 14:21:01 -0700, Andres Freund wrote: > > > > Due to bug #17245: [1] I spent a

Re: Add client connection check during the execution of the query

2021-12-12 Thread Thomas Munro
On Sat, Dec 11, 2021 at 7:09 PM Thomas Munro wrote: > On Sat, Dec 11, 2021 at 6:11 PM Andres Freund wrote: > > Yuck. Is there really no better way to deal with this? What kind of errors > > is > > this trying to handle transparently? Afaics this still changes when we'd > > e.g. detect

Re: Skipping logical replication transactions on subscriber side

2021-12-12 Thread Amit Kapila
On Mon, Dec 13, 2021 at 8:28 AM Masahiko Sawada wrote: > > On Sat, Dec 11, 2021 at 3:29 PM Amit Kapila wrote: > > > > 3. > > + * Also, we don't skip receiving the changes in streaming cases, > > since we decide > > + * whether or not to skip applying the changes when starting to apply > >

Assertion failure with replication origins and PREPARE TRANSACTIOn

2021-12-12 Thread Michael Paquier
Hi all, (CCing some folks who worked on this area lately) The following sequence of commands generates an assertion failure, as of $subject: select pg_replication_origin_create('popo'); select pg_replication_origin_session_setup('popo'); begin; select txid_current(); prepare transaction 'popo';

Re: Skipping logical replication transactions on subscriber side

2021-12-12 Thread Greg Nancarrow
On Fri, Dec 10, 2021 at 4:44 PM Masahiko Sawada wrote: > > I've attached an updated patch. The new syntax is like "ALTER > SUBSCRIPTION testsub SKIP (xid = '123')". > I have some review comments: (1) Patch comment - some suggested wording improvements BEFORE: If incoming change violates any

Re: Replication slot drop message is sent after pgstats shutdown.

2021-12-12 Thread Kyotaro Horiguchi
At Fri, 10 Dec 2021 18:13:31 +0900, Masahiko Sawada wrote in > I agreed with Andres and Horiguchi-san and attached an updated patch. Thanks for the new version. It seems fine, but I have some comments from a cosmetic viewpoint. + /* +* Register callback to make sure cleanup and

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-12-12 Thread Ashutosh Sharma
+ /* +* If the relation is from the default tablespace then we need to +* create it in the destinations db's default tablespace. Otherwise, +* we need to create in the same tablespace as it is in the source +* database. +*/ This comment looks a bit

Re: parallel vacuum comments

2021-12-12 Thread Amit Kapila
On Sat, Dec 11, 2021 at 8:30 PM Masahiko Sawada wrote: > > On Sat, Dec 11, 2021 at 2:32 PM Andres Freund wrote: > > > > Hi, > > > > On 2021-10-30 14:21:01 -0700, Andres Freund wrote: > > > Due to bug #17245: [1] I spent a considerably amount of time looking at > > > vacuum > > > related code.

Re: Skipping logical replication transactions on subscriber side

2021-12-12 Thread Masahiko Sawada
On Sat, Dec 11, 2021 at 3:29 PM Amit Kapila wrote: > > On Fri, Dec 10, 2021 at 11:14 AM Masahiko Sawada > wrote: > > > > On Thu, Dec 9, 2021 at 6:16 PM Amit Kapila wrote: > > > > > > On Thu, Dec 9, 2021 at 2:24 PM Masahiko Sawada > > > wrote: > > > > > > > > On Thu, Dec 9, 2021 at 11:47 AM

Re: Use extended statistics to estimate (Var op Var) clauses

2021-12-12 Thread Zhihong Yu
On Sun, Dec 12, 2021 at 6:04 PM Tomas Vondra wrote: > On 8/31/21 00:14, Zhihong Yu wrote: > > Hi, > > For patch 0002, > > > > + s1 = statext_clauselist_selectivity(root, clauses, > > varRelid, > > + jointype, > > sjinfo,

Re: Use extended statistics to estimate (Var op Var) clauses

2021-12-12 Thread Tomas Vondra
Hi, I finally got around to this patch again, focusing mostly on the first part that simply returns either 1.0 or 0.0 for Var op Var conditions (i.e. the part not really using extended statistics). I have been unhappy about using examine_variable, which does various expensive things like

Re: Make pg_waldump report replication origin ID, LSN, and timestamp.

2021-12-12 Thread Michael Paquier
On Thu, Dec 09, 2021 at 05:42:56PM +0900, Masahiko Sawada wrote: > Thank you for updating the patch. The patch looks good to me. Done this way. Please note that while testing this patch, I have found a completely different issue. I'll spawn a thread about that in a minute.. -- Michael

Re: Use extended statistics to estimate (Var op Var) clauses

2021-12-12 Thread Tomas Vondra
On 8/31/21 00:14, Zhihong Yu wrote: Hi, For patch 0002, +                   s1 = statext_clauselist_selectivity(root, clauses, varRelid, +                                                       jointype, sjinfo, rel, + , false); + +      

Defining (and possibly skipping) useless VACUUM operations

2021-12-12 Thread Peter Geoghegan
Robert Haas has written on the subject of useless vacuuming, here: http://rhaas.blogspot.com/2020/02/useless-vacuuming.html I'm sure at least a few of us have thought about the problem at some point. I would like to discuss how we can actually avoid useless vacuuming, and what our goals should

Re: Atomic rename feature for Windows.

2021-12-12 Thread Michael Paquier
On Thu, Dec 09, 2021 at 11:33:17PM -0500, Tom Lane wrote: > My general approach to platform compatibility is that when we > break compatibility with old versions of something, we should do so > because it will bring concrete benefits. If we can plausibly > drop support for Windows versions that

Re: Logical replication error "no record found" /* shouldn't happen */

2021-12-12 Thread Alvaro Herrera
On 2021-Jul-23, Andrey Borodin wrote: > Hi! > > From time to time I observe $subj on clusters using logical replication. > I most of cases there are a lot of other errors. Probably $subj condition > should be kind of impossible without other problems. > I propose to enhance error logging of

Re: [PATCH] pg_stat_toast

2021-12-12 Thread Andres Freund
Hi, On 2021-12-13 00:00:23 +0100, Gunnar "Nick" Bluth wrote: > Regarding stats size; it adds one PgStat_BackendToastEntry > (PgStat_BackendAttrIdentifier + PgStat_ToastCounts, should be 56-64 bytes or > something in that ballpark) per TOASTable attribute, I can't see that make > any system break

Re: can we add subscription TAP test option "vcregress subscriptioncheck" for MSVC builds?

2021-12-12 Thread Juan José Santamaría Flecha
On Thu, Oct 21, 2021 at 7:21 AM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > > Here's the documentation v1 patch that I've come up with. Please review it. > > There's a typo: + To run an arbitrary TAP test set, run vcregress taptest + comamnd. For example, use the

Re: [PATCH] pg_stat_toast

2021-12-12 Thread Gunnar "Nick" Bluth
Am 12.12.21 um 22:52 schrieb Andres Freund: Hi, On 2021-12-12 17:20:58 +0100, Gunnar "Nick" Bluth wrote: Please have a look at the attached patch, which implements some statistics for TOAST. The idea (and patch) have been lurking here for quite a while now, so I decided to dust it off, rebase

Re: extended stats on partitioned tables

2021-12-12 Thread Tomas Vondra
On 12/12/21 22:32, Justin Pryzby wrote: On Sun, Dec 12, 2021 at 05:17:10AM +0100, Tomas Vondra wrote: The one thing bugging me a bit is that the regression test checks only a GROUP BY query. It'd be nice to add queries testing MCV/dependencies too, but that seems tricky because most queries

Re: [PATCH] pg_stat_toast

2021-12-12 Thread Andres Freund
Hi, On 2021-12-12 17:20:58 +0100, Gunnar "Nick" Bluth wrote: > Please have a look at the attached patch, which implements some statistics > for TOAST. > > The idea (and patch) have been lurking here for quite a while now, so I > decided to dust it off, rebase it to HEAD and send it out for

Re: extended stats on partitioned tables

2021-12-12 Thread Justin Pryzby
On Sun, Dec 12, 2021 at 10:29:39PM +0100, Tomas Vondra wrote: > On 12/12/21 18:52, Justin Pryzby wrote: > That's mostly a conscious choice, so that I don't have to include > parsetree.h. But maybe that'd be better ... > > > The regression tests changed as a result of not populating stx_data; I

Re: extended stats on partitioned tables

2021-12-12 Thread Justin Pryzby
On Sun, Dec 12, 2021 at 05:17:10AM +0100, Tomas Vondra wrote: > The one thing bugging me a bit is that the regression test checks only a > GROUP BY query. It'd be nice to add queries testing MCV/dependencies > too, but that seems tricky because most queries will use per-partitions > stats. You

Re: Windows now has fdatasync()

2021-12-12 Thread Thomas Munro
On Sun, Dec 12, 2021 at 3:48 PM Thomas Munro wrote: > [...] I > tried out a quick POC patch and it runs a bit faster than fsync(), as > expected. I'm not sure if it's worth bothering with or not given the > other options, but figured it was worth sharing. One reason to consider developing this

Re: extended stats on partitioned tables

2021-12-12 Thread Tomas Vondra
On 12/12/21 18:52, Justin Pryzby wrote: > + * XXX Can't we simply look at rte->inh? > + */ > + inh = root->append_rel_array == NULL ? false : > + root->append_rel_array[onerel->relid]->parent_relid != 0; > > I think so. That's what I came up with while

Re: extended stats on partitioned tables

2021-12-12 Thread Tom Lane
Tomas Vondra writes: > On 12/12/21 16:37, Zhihong Yu wrote: >> Since the rte (RangeTblEntry*) doesn't seem to be used beyond checking >> inh, I think it would be better if the above style of checking is used >> throughout the patch (without introducing rte variable). > It's mostly a matter of

Re: extended stats on partitioned tables

2021-12-12 Thread Justin Pryzby
+ * XXX Can't we simply look at rte->inh? + */ + inh = root->append_rel_array == NULL ? false : + root->append_rel_array[onerel->relid]->parent_relid != 0; I think so. That's what I came up with while trying to figured this out, and it's no great

Re: extended stats on partitioned tables

2021-12-12 Thread Tomas Vondra
On 12/12/21 16:37, Zhihong Yu wrote: Hi, For patch 1, minor comment: +           if (planner_rt_fetch(onerel->relid, root)->inh) Since the rte (RangeTblEntry*) doesn't seem to be used beyond checking inh, I think it would be better if the above style of checking is used throughout the patch

Re: extended stats on partitioned tables

2021-12-12 Thread Tomas Vondra
On 12/12/21 14:47, Zhihong Yu wrote: On Sat, Dec 11, 2021 at 9:14 PM Tomas Vondra mailto:tomas.von...@enterprisedb.com>> wrote: > ... > +       /* skip statistics with mismatching stxdinherit value */ > +       if (stat->inherit != rte->inh) > > Should a log be added

[PATCH] pg_stat_toast

2021-12-12 Thread Gunnar "Nick" Bluth
Hello -hackers! Please have a look at the attached patch, which implements some statistics for TOAST. The idea (and patch) have been lurking here for quite a while now, so I decided to dust it off, rebase it to HEAD and send it out for review today. A big shoutout to Georgios Kokolatos,

Re: extended stats on partitioned tables

2021-12-12 Thread Zhihong Yu
On Sat, Dec 11, 2021 at 8:17 PM Tomas Vondra wrote: > Hi, > > Attached is a rebased and cleaned-up version of these patches, with more > comments, refactorings etc. Justin and Zhihong, can you take a look? > > > 0001 - Ignore extended statistics for inheritance trees > > 0002 - Build inherited

Re: extended stats on partitioned tables

2021-12-12 Thread Zhihong Yu
On Sat, Dec 11, 2021 at 9:14 PM Tomas Vondra wrote: > > > On 12/12/21 05:38, Zhihong Yu wrote: > > > > > > On Sat, Dec 11, 2021 at 8:17 PM Tomas Vondra > > mailto:tomas.von...@enterprisedb.com>> > > wrote: > > > > Hi, > > > > Attached is a rebased and cleaned-up version of these patches,

回复:Re: Re: 回复:Re: Is it worth pushing conditions to sublink/subplan?

2021-12-12 Thread 曾文旌(义从)
--原始邮件 -- 发件人:Zhihong Yu 发送时间:Sun Dec 12 01:13:11 2021 收件人:曾文旌(义从) 抄送:Tomas Vondra , wjzeng , PostgreSQL Hackers , shawn wang , ggys...@gmail.com 主题:Re: Re: 回复:Re: Is it worth pushing conditions to sublink/subplan? On Sat, Dec 11, 2021 at 7:31 AM 曾文旌(义从)

Re: add recovery, backup, archive, streaming etc. activity messages to server logs along with ps display

2021-12-12 Thread Bharath Rupireddy
On Thu, Dec 9, 2021 at 9:28 PM Alvaro Herrera wrote: > > On 2021-Dec-09, Bharath Rupireddy wrote: > > > On Thu, Dec 9, 2021 at 6:00 PM Alvaro Herrera > > wrote: > > > > > > On 2021-Dec-09, Bharath Rupireddy wrote: > > > > > > > I just want to call this out: an insertion of 10 million rows in

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-12-12 Thread Dilip Kumar
On Fri, Dec 10, 2021 at 7:39 AM Ashutosh Sharma wrote: >> >> Logfile Snippet: >> 2021-12-09 17:49:18.110 +04 [18151] PANIC: could not fsync file >> "base/116398/116400": No such file or directory >> 2021-12-09 17:49:19.105 +04 [18150] LOG: checkpointer process (PID 18151) >> was terminated by