Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-07 Thread Ankit Kumar Pandey
On 08/01/23 04:06, David Rowley wrote: On Sun, 8 Jan 2023 at 05:45, Ankit Kumar Pandey wrote: Attached patch with test cases. I can look at this in a bit more detail if you find a way to fix the case you mentioned earlier. i.e, push the sort down to the deepest WindowAgg that has

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-07 Thread Ankit Kumar Pandey
On 08/01/23 03:56, David Rowley wrote: > (your email client still seems broken) I am looking at this again, will be changing client for here onward. You might need to have another loop before the foreach loop that loops backwards through the WindowClauses and remembers the index of the

Re: [BUG] Logical replica crash if there was an error in a function.

2023-01-07 Thread Anton A. Melnikov
Thanks for your remarks. On 07.01.2023 15:27, vignesh C wrote: Few suggestions: 1) There is a warning: +# This would crash on the subscriber if not fixed +$node_publisher->safe_psql('postgres', "INSERT INTO tab1 VALUES (3, 4)"); + +my $result = $node_subscriber->wait_for_log( + "ERROR:

Re: drop postmaster symlink

2023-01-07 Thread Karl O. Pinc
On Sat, 7 Jan 2023 19:56:08 -0600 "Karl O. Pinc" wrote: > On Sat, 07 Jan 2023 18:38:25 -0500 > Tom Lane wrote: > > > "Karl O. Pinc" writes: > > > This is a review of Peter's 2 patches. I see only 1 small > > > problem. ... > > Hmm ... I thought this patch was about getting rid of the > >

Re: pgsql: Delay commit status checks until freezing executes.

2023-01-07 Thread Andres Freund
Hi, On 2023-01-07 15:41:29 -0800, Peter Geoghegan wrote: > Do we need to do anything about this to the "pg_xact and pg_subtrans" > section of the transam README? Probably a good idea, although it doesn't neatly fit right now. > Also, does amcheck's get_xid_status() need a reference to these

Re: BUG: Postgres 14 + vacuum_defer_cleanup_age + FOR UPDATE + UPDATE

2023-01-07 Thread Andres Freund
Hi, On 2023-01-07 16:29:23 -0800, Andres Freund wrote: > It's probably not too hard to fix specifically in this one place - we could > just clamp vacuum_defer_cleanup_age to be smaller than latest_completed, but > it strikes me as as a somewhat larger issue for the 64it xid infrastructure. I >

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-07 Thread Amit Kapila
On Sat, Jan 7, 2023 at 2:25 PM Dilip Kumar wrote: > Today, I was analyzing this patch w.r.t recent commit c6e1f62e2c and found that pa_set_xact_state() should set the latch (wake up) for the leader worker as the leader could be waiting in pa_wait_for_xact_state(). What do you think? But

Re: drop postmaster symlink

2023-01-07 Thread Karl O. Pinc
On Sat, 7 Jan 2023 19:33:38 -0500 Joe Conway wrote: > On 1/7/23 18:38, Tom Lane wrote: > > "Karl O. Pinc" writes: > >> This is a review of Peter's 2 patches. I see only 1 small > >> problem. The small problem is a reference to a deleted file. Regards, Karl Free Software: "You don't

Re: drop postmaster symlink

2023-01-07 Thread Karl O. Pinc
On Sat, 07 Jan 2023 18:38:25 -0500 Tom Lane wrote: > "Karl O. Pinc" writes: > > This is a review of Peter's 2 patches. I see only 1 small problem. > > > > > Looking at the documentation, a "postmaster" in the glossary is > > defined as the controlling process. This works; it needs to be >

Re: BUG: Postgres 14 + vacuum_defer_cleanup_age + FOR UPDATE + UPDATE

2023-01-07 Thread Andres Freund
Hi, On 2023-01-07 21:06:06 +0300, Michail Nikolaev wrote: > 2) It is not an issue at table creation time. Issue is reproducible if > vacuum_defer_cleanup_age set after table preparation. > > 3) To reproduce the issue, vacuum_defer_cleanup_age should flip xid > over zero (be >= txid_current()). >

Re: drop postmaster symlink

2023-01-07 Thread Joe Conway
On 1/7/23 18:38, Tom Lane wrote: "Karl O. Pinc" writes: This is a review of Peter's 2 patches. I see only 1 small problem. Looking at the documentation, a "postmaster" in the glossary is defined as the controlling process. This works; it needs to be called something. There is still a

Re: BUG: Postgres 14 + vacuum_defer_cleanup_age + FOR UPDATE + UPDATE

2023-01-07 Thread Andres Freund
Hi, Thomas, CCing you because of the 64bit xid representation aspect. On 2023-01-06 00:39:44 +0300, Michail Nikolaev wrote: > I apologize for the direct ping, but I think your snapshot scalability > work in PG14 could be related to the issue. Good call! > The TransactionIdRetreatedBy

Re: pgsql: Delay commit status checks until freezing executes.

2023-01-07 Thread Peter Geoghegan
On Sat, Jan 7, 2023 at 1:47 PM Andres Freund wrote: > > What do you think of the attached patch, which revises comments over > > TransactionIdDidAbort, and adds something about it to the top of > > heapam_visbility.c? > > Mostly looks good to me. I think it'd be good to add a reference to the >

Re: drop postmaster symlink

2023-01-07 Thread Tom Lane
"Karl O. Pinc" writes: > This is a review of Peter's 2 patches. I see only 1 small problem. > Looking at the documentation, a "postmaster" in the glossary is > defined as the controlling process. This works; it needs to be called > something. There is still a postmaster.pid (etc.) in the data

FYI: 2022-10 thorntail failures from coreutils FICLONE

2023-01-07 Thread Noah Misch
thorntail failed some recovery tests in 2022-10: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=thorntail=2022-11-02%2004%3A25%3A43 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=thorntail=2022-10-31%2013%3A32%3A42

Re: drop postmaster symlink

2023-01-07 Thread Karl O. Pinc
Hello, This is a review of Peter's 2 patches. I see only 1 small problem. +++ Looking at the documentation, a "postmaster" in the glossary is defined as the controlling process. This works; it needs to be called something. There is still a postmaster.pid (etc.) in

Re: Using WaitEventSet in the postmaster

2023-01-07 Thread Andres Freund
Hi, On 2023-01-07 18:08:11 +1300, Thomas Munro wrote: > On Sat, Jan 7, 2023 at 12:25 PM Andres Freund wrote: > > On 2023-01-07 11:08:36 +1300, Thomas Munro wrote: > > > 3. Is it OK to clobber the shared pending flag for SIGQUIT, SIGTERM, > > > SIGINT? If you send all of these extremely

Re: Infinite Interval

2023-01-07 Thread Joseph Koshakow
On Sat, Jan 7, 2023 at 3:05 PM Joseph Koshakow wrote: > > On Sat, Jan 7, 2023 at 3:04 PM Joseph Koshakow wrote: > > > > I think this patch is just about ready for review, except for the > > following two questions: > > 1. Should finite checks on intervals only look at months or all three > >

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-07 Thread David Rowley
On Sun, 8 Jan 2023 at 05:45, Ankit Kumar Pandey wrote: > Attached patch with test cases. I can look at this in a bit more detail if you find a way to fix the case you mentioned earlier. i.e, push the sort down to the deepest WindowAgg that has pathkeys contained in the query's ORDER BY pathkeys.

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-07 Thread David Rowley
(your email client still seems broken) On Sun, 8 Jan 2023 at 05:27, Ankit Kumar Pandey wrote: > > > While writing test cases, I found that optimization do not happen for > case #1 > > (which is prime candidate for such operation) like > > EXPLAIN (COSTS OFF) > SELECT empno, > depname, >

Re: delayed initialization in worktable scan

2023-01-07 Thread Tom Lane
Andres Freund writes: > Basically the issue is that in queries with two CTEs we can, at least > currently, end up with a WorkTable scans on a CTE we've not yet initialized, > due to the WorkTable scan of one CTE appearing in the other. Thus > ExecInitRecursiveUnion() hasn't yet set up the param

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-07 Thread David Rowley
On Sun, 8 Jan 2023 at 01:52, Ankit Kumar Pandey wrote: > I am just wondering though, why can we not do top-N sort > in optimized version if we include limit clause? Is top-N sort is > limited to non strict sorting or > cases last operation before limit is sort? . Maybe the sort bound can be

Re: pgsql: Delay commit status checks until freezing executes.

2023-01-07 Thread Andres Freund
Hi, On 2023-01-06 11:16:00 -0800, Peter Geoghegan wrote: > On Tue, Jan 3, 2023 at 10:52 PM Peter Geoghegan wrote: > > > And it'd make sense to have > > > the explanation of why TransactionIdDidAbort() isn't the same as > > > !TransactionIdDidCommit(), even for !TransactionIdIsInProgress() xacts,

delayed initialization in worktable scan

2023-01-07 Thread Andres Freund
Hi, while looking at fixing [1], I again came across the fact that we don't initialize the projection etc during ExecInitWorkTableScan(), but do so during the first call to ExecWorkTableScan(). This is explained with the following comment: /* * On the first call, find the

Re: How to define template types in PostgreSQL

2023-01-07 Thread Nikita Malakhov
Hi! I'd suggest creating an API that defines a general function set with variable input, and calling implementation defined on the input type? On Sat, Jan 7, 2023 at 12:32 PM Esteban Zimanyi wrote: > Dear all > > MobilityDB (https://github.com/MobilityDB/MobilityDB) defines at the C > level

Re: Infinite Interval

2023-01-07 Thread Joseph Koshakow
On Sat, Jan 7, 2023 at 3:04 PM Joseph Koshakow wrote: > > On Thu, Jan 5, 2023 at 11:30 PM jian he wrote: > > > > > > > > On Fri, Jan 6, 2023 at 6:54 AM Joseph Koshakow wrote: > >> > >> Looks like some of the error messages have changed and we > >> have some issues with parsing "+infinity" after

Re: Infinite Interval

2023-01-07 Thread Joseph Koshakow
On Thu, Jan 5, 2023 at 11:30 PM jian he wrote: > > > > On Fri, Jan 6, 2023 at 6:54 AM Joseph Koshakow wrote: >> >> Looks like some of the error messages have changed and we >> have some issues with parsing "+infinity" after rebasing. > > > There is a commit

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-01-07 Thread Tom Lane
=?UTF-8?B?w5ZuZGVyIEthbGFjxLE=?= writes: > Attached v22. I took a very brief look through this. I'm not too pleased with this whole line of development TBH. It seems to me that the core design of execReplication.c and related code is "let's build our own half-baked executor and

Re: [RFC] Add jit deform_counter

2023-01-07 Thread Pavel Stehule
so 7. 1. 2023 v 16:48 odesílatel Dmitry Dolgov <9erthali...@gmail.com> napsal: > > On Fri, Jan 06, 2023 at 09:42:09AM +0100, Pavel Stehule wrote: > > The explain part is working, the part of pg_stat_statements doesn't > > > > set jit_above_cost to 10; > > set jit_optimize_above_cost to 10; > >

Re: BUG: Postgres 14 + vacuum_defer_cleanup_age + FOR UPDATE + UPDATE

2023-01-07 Thread Michail Nikolaev
Hello. The few things I have got so far: 1) It is not required to order by random() to reproduce the issue - it could be done using queries like: BEGIN; SELECT omg.* FROM something_is_wrong_here AS omg ORDER BY value -- change is here LIMIT 1 FOR

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-07 Thread Ankit Kumar Pandey
On 07/01/23 21:57, Ankit Kumar Pandey wrote: On 07/01/23 09:58, David Rowley wrote: > > The attached patch has no tests added. It's going to need some of > those. While writing test cases, I found that optimization do not happen for case #1 (which is prime candidate for such operation) like

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-07 Thread Ankit Kumar Pandey
On 07/01/23 09:58, David Rowley wrote: The attached patch has no tests added. It's going to need some of those. While writing test cases, I found that optimization do not happen for case #1 (which is prime candidate for such operation) like EXPLAIN (COSTS OFF) SELECT empno,   

Re: [RFC] Add jit deform_counter

2023-01-07 Thread Dmitry Dolgov
> On Fri, Jan 06, 2023 at 09:42:09AM +0100, Pavel Stehule wrote: > The explain part is working, the part of pg_stat_statements doesn't > > set jit_above_cost to 10; > set jit_optimize_above_cost to 10; > set jit_inline_above_cost to 10; > > (2023-01-06 09:08:59) postgres=# explain analyze select >

Re: [PATCH] Add function to_oct

2023-01-07 Thread Tom Lane
vignesh C writes: > Few suggestions > 1) We could use to_oct instead of to_oct32 as we don't have multiple > implementations for to_oct That seems (a) shortsighted and (b) inconsistent with the naming pattern used for to_hex, so I doubt it'd be an improvement. regards,

Re: postgres_fdw: using TABLESAMPLE to collect remote sample

2023-01-07 Thread Tom Lane
Tomas Vondra writes: > However, maybe views are not the best / most common example to think > about. I'd imagine it's much more common to reference a regular table, > but the table gets truncated / populated quickly, and/or the autovacuum > workers are busy so it takes time to update reltuples.

Re: postgres_fdw: using TABLESAMPLE to collect remote sample

2023-01-07 Thread Tomas Vondra
On 1/6/23 23:41, Tomas Vondra wrote: > On 1/6/23 17:58, Tom Lane wrote: >> Tomas Vondra writes: >>> The one difference is that I realized the relkind check does not >>> actually say we can't do sampling - it just means we can't use >>> TABLESAMPLE to do it. We could still use "random()" ... >>

Re: MERGE ... WHEN NOT MATCHED BY SOURCE

2023-01-07 Thread Dean Rasheed
On Thu, 5 Jan 2023 at 13:21, Dean Rasheed wrote: > > On Thu, 5 Jan 2023 at 11:03, Alvaro Herrera wrote: > > > > > + /* Join type required */ > > > + if (left_join && right_join) > > > + qry->mergeJoinType = JOIN_FULL; > > > + else if (left_join) > > > +

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-07 Thread Ankit Kumar Pandey
On 07/01/23 17:28, David Rowley wrote: Your email client seems to be adding additional vertical space to your emails. I've removed the additional newlines in the quotes. Are you able to fix the client so it does not do that? I have adjusted my mail client, hope it is better now? On Sun, 8

Re: [BUG] Logical replica crash if there was an error in a function.

2023-01-07 Thread vignesh C
On Sun, 11 Dec 2022 at 09:21, Anton A. Melnikov wrote: > > > On 07.12.2022 21:03, Andres Freund wrote: > > > > > This CF entry causes tests to fail on all platforms: > > https://cirrus-ci.com/build/5755408111894528 > > > > E.g. > >

Re: Improve tab completion for ALTER FUNCTION/PROCEDURE/ROUTINE

2023-01-07 Thread vignesh C
On Fri, 6 Jan 2023 at 15:33, Dean Rasheed wrote: > > On Fri, 6 Jan 2023 at 02:38, vignesh C wrote: > > > > On Thu, 5 Jan 2023 at 18:22, Dean Rasheed wrote: > > > > > > That leads to the attached, which barring objections, I'll push shortly. > > > > The changes look good to me. > > > > Pushed.

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-07 Thread David Rowley
Your email client seems to be adding additional vertical space to your emails. I've removed the additional newlines in the quotes. Are you able to fix the client so it does not do that? On Sun, 8 Jan 2023 at 00:10, Ankit Kumar Pandey wrote: > > On 07/01/23 09:58, David Rowley wrote: > > You also

Re: add \dpS to psql

2023-01-07 Thread Dean Rasheed
On Sat, 7 Jan 2023 at 00:36, Nathan Bossart wrote: > > On Fri, Jan 06, 2023 at 06:52:33PM +, Dean Rasheed wrote: > > > > So I think we should use the same SQL clauses as every other psql > > command that supports "S", namely: > > > > if (!showSystem && !pattern) > >

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-07 Thread Ankit Kumar Pandey
On 07/01/23 07:59, David Rowley wrote: On Thu, 5 Jan 2023 at 04:11, Ankit Kumar Pandey wrote: Attaching test cases for this (+ small change in doc). Tested this in one of WIP branch where I had modified select_active_windows and it failed as expected. Please let me know if something can

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-07 Thread Ankit Kumar Pandey
Thanks for looking into this. On 07/01/23 09:58, David Rowley wrote: On Sat, 7 Jan 2023 at 02:11, Ankit Kumar Pandey wrote: On 05/01/23 12:53, David Rowley wrote: We *can* reuse Sorts where a more strict or equivalent sort order is available. The question is how do we get the final

Re: [PATCH] Add function to_oct

2023-01-07 Thread vignesh C
On Thu, 22 Dec 2022 at 23:11, Eric Radman wrote: > > On Thu, Dec 22, 2022 at 10:08:17AM +, Dag Lem wrote: > > > > The calculation of quotient and remainder can be replaced by less costly > > masking and shifting. > > > > Defining > > > > #define OCT_DIGIT_BITS 3 > > #define OCT_DIGIT_BITMASK

Re: Generating code for query jumbling through gen_node_support.pl

2023-01-07 Thread Michael Paquier
On Sat, Jan 07, 2023 at 07:37:49AM +0100, Peter Eisentraut wrote: > The generation script already has a way to identify location fields, by ($t > eq 'int' && $f =~ 'location$'), so you could use that as well. I recall that some of the nodes may need renames to map with this choice. That could be

Re: Notify downstream to discard the streamed transaction which was aborted due to crash.

2023-01-07 Thread Amit Kapila
On Fri, Jan 6, 2023 at 11:18 AM Dilip Kumar wrote: > > > > > To fix it, One idea is to send a stream abort message when we are cleaning > > up > > crashed transaction on publisher(e.g. in ReorderBufferAbortOld()). And here > > is > > a tiny patch which changes the same. I have confirmed that

Allow DISTINCT to use Incremental Sort

2023-01-07 Thread David Rowley
While working on the regression tests added in a14a58329, I noticed that DISTINCT does not make use of Incremental Sort. It'll only ever do full sorts on the cheapest input path or make use of a path that's already got the required pathkeys. Also, I see that create_final_distinct_paths() is a

How to define template types in PostgreSQL

2023-01-07 Thread Esteban Zimanyi
Dear all MobilityDB (https://github.com/MobilityDB/MobilityDB) defines at the C level four template types: Set, Span, SpanSet, and Temporal. The type Set is akin to PostgreSQL's ArrayType restricted to one dimension, but enforces the constraint that sets do not have duplicates, the types Span and

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-07 Thread Dilip Kumar
On Sat, Jan 7, 2023 at 11:13 AM houzj.f...@fujitsu.com wrote: > > On Saturday, January 7, 2023 12:50 PM Dilip Kumar > > > > On Fri, Jan 6, 2023 at 3:38 PM houzj.f...@fujitsu.com > > > > wrote: > > > > > > > Looks good, but I feel in pa_process_spooled_messages_if_required() > > function after