Re: Removing unneeded self joins

2024-07-21 Thread Andrei Lepikhov
to working with the query tree before the planning phase. SJE works right in the middle of the planning process. So, it may be more practical to keep it separate as a walker, as Dean has proposed. If the optimisation stuff changes, the walker code will be changed, too. -- regards, Andrei Lepikhov

Re: Expand applicability of aggregate's sortop optimization

2024-07-18 Thread Andrei Lepikhov
On 18/7/2024 19:49, Matthias van de Meent wrote: On Wed, 17 Jul 2024 at 16:09, Andrei Lepikhov wrote: On 17/7/2024 16:33, Matthias van de Meent wrote: On Wed, 17 Jul 2024 at 05:29, Andrei Lepikhov wrote: Because the @<@ and @>@ operators are not registered as commutative, it couldn't

Re: Expand applicability of aggregate's sortop optimization

2024-07-17 Thread Andrei Lepikhov
On 17/7/2024 16:33, Matthias van de Meent wrote: On Wed, 17 Jul 2024 at 05:29, Andrei Lepikhov wrote: As I see, the code: aggsortop = fetch_agg_sort_op(aggref->aggfnoid); if (!OidIsValid(aggsortop)) return false;/* not a MIN/MAX aggregate */ used twice and can be evalua

Re: Expand applicability of aggregate's sortop optimization

2024-07-17 Thread Andrei Lepikhov
On 17/7/2024 16:33, Matthias van de Meent wrote: On Wed, 17 Jul 2024 at 05:29, Andrei Lepikhov wrote: Thanks for the job! I guess you could be more brave and push down also FILTER statement. While probably not impossible, I wasn't planning on changing this code with new optimizations; just

Re: Expand applicability of aggregate's sortop optimization

2024-07-16 Thread Andrei Lepikhov
function separately. How do you think to avoid it? Also, I don't clearly understand the case you mentioned here - does it mean that you want to nullify orders for other aggregate types if they are the same as the incoming order? -- regards, Andrei Lepikhov

Re: Expand applicability of aggregate's sortop optimization

2024-07-16 Thread Andrei Lepikhov
ooking through the btree classes. Maybe just to check the commutator to the sortop, like in the diff attached? Or could you provide an example to support your approach? -- regards, Andrei Lepikhov diff --git a/src/backend/optimizer/plan/planagg.c b/src/backend/optimizer/plan/planagg.c index afb5

Re: Removing unneeded self joins

2024-07-15 Thread Andrei Lepikhov
On 7/15/24 14:35, jian he wrote: On Mon, Jul 15, 2024 at 2:08 PM Andrei Lepikhov wrote: On 7/15/24 12:31, jian he wrote: hi. Here is the latest patch (v6), I've made the following changes. * disallow original Query->resultRelation participate in SJE. for SELECT, nothing is chan

Re: Removing unneeded self joins

2024-07-15 Thread Andrei Lepikhov
is not? Why not to skip this pair of relids? Or, at least, insert an assertion to check that you filtered it earlier. -- regards, Andrei Lepikhov

Re: Check lateral references within PHVs for memoize cache keys

2024-07-11 Thread Andrei Lepikhov
On 7/11/24 16:18, Richard Guo wrote: On Fri, Jun 28, 2024 at 10:14 PM Andrei Lepikhov wrote: I got the point about Memoize over join, but as a join still calls replace_nestloop_params to replace parameters in its clauses, why not to invent something similar to find Memoize keys inside specific

Re: Removing unneeded self joins

2024-07-11 Thread Andrei Lepikhov
On 7/11/24 14:43, jian he wrote: On Tue, Jul 9, 2024 at 2:06 PM Andrei Lepikhov wrote: On 7/2/24 07:25, jian he wrote: to make sure it's correct, I have added a lot of tests, Some of this may be contrived, maybe some of the tests are redundant. Thanks for your job! I passed through

Re: MergeJoin beats HashJoin in the case of multiple hash clauses

2024-07-08 Thread Andrei Lepikhov
ergejoin?r=34q1yy_campaign=post_medium=web -- regards, Andrei Lepikhov From b4b007970b1d9b99602b8422f2122c8e5738828e Mon Sep 17 00:00:00 2001 From: "Andrei V. Lepikhov" Date: Mon, 13 May 2024 16:15:02 +0700 Subject: [PATCH] Use extended statistics for precise estimation of bucket size in has

Re: Pluggable cumulative statistics

2024-07-03 Thread Andrei Lepikhov
be loaded on startup, stay backend-local and utilise shared resources. It was a tremendous win for me. Is it possible to design this extension in the same way? -- regards, Andrei Lepikhov

Re: A new strategy for pull-up correlated ANY_SUBLINK

2024-07-03 Thread Andrei Lepikhov
On 7/1/24 16:17, Andrei Lepikhov wrote: On 10/12/23 14:52, Andy Fan wrote: Here the sublink can't be pulled up because of its reference to the  LHS of left join, the original logic is that no matter the 'b.t in ..' returns the true or false,  the rows in LHS will be returned.  If we pull

Re: A new strategy for pull-up correlated ANY_SUBLINK

2024-07-01 Thread Andrei Lepikhov
://www.postgresql.org/message-id/6531.1218473967%40sss.pgh.pa.us [2] https://www.postgresql.org/message-id/BANLkTikGFtGnAaXVh5%3DntRdN%2B4w%2Br%3DNPuw%40mail.gmail.com [3] https://www.vldb.org/conf/1992/P091.PDF -- regards, Andrei Lepikhov

Re: Check lateral references within PHVs for memoize cache keys

2024-06-28 Thread Andrei Lepikhov
)) cache_purge_all(node); is a good place to check an assertion: is it really the parent query parameters that make a difference between memoize keys and node list of parameters? Generally, this patch looks good for me to be committed. -- regards, Andrei Lepikhov

Incremental Sort Cost Estimation Instability

2024-06-26 Thread Andrei Lepikhov
Hi, While designing an improvement for the cost sort model, I discovered that the query plan can vary if we slightly change the query text without pushing semantic differences. See the example below: CREATE TABLE test(x integer, y integer,z text); INSERT INTO test (x,y) SELECT x, 1 FROM

Re: Missing docs for new enable_group_by_reordering GUC

2024-06-18 Thread Andrei Lepikhov
. Would someone please add docs for this? Thanks. It is my mistake, sorry for that. See the patch in attachment. -- regards, Andrei Lepikhov From b2d5aca0547bf912032ad60a30a55ef86a28708e Mon Sep 17 00:00:00 2001 From: "Andrei V. Lepikhov" Date: Tue, 18 Jun 2024 13:11:52 +0700 Subject: [

Re: POC, WIP: OR-clause support for indexes

2024-06-16 Thread Andrei Lepikhov
reasons why we shifted the code to the earlier stages in the previous incarnation. So, going this way we should recheck all the fields of this structure and analyse how the transformation can [potentially] affect their values. -- regards, Andrei Lepikhov Postgres Professional diff --git a/src

Re: Proposal: Job Scheduler

2024-06-06 Thread Andrei Lepikhov
On 6/6/2024 16:04, Wang Cheng wrote: Noted. Thanks for suggestions. We will open-source it as an extension. It would be nice! `For me doesn't matter where to contribute: to PostgreSQL core or to its extension if it is published under BSD license. -- regards, Andrei Lepikhov

Re: POC: GROUP BY optimization

2024-05-29 Thread Andrei Lepikhov
On 5/29/24 19:53, Alexander Korotkov wrote: Hi, Andrei! Thank you for your feedback. On Wed, May 29, 2024 at 11:08 AM Andrei Lepikhov wrote: On 5/27/24 19:41, Alexander Korotkov wrote: Any thoughts? About 0001: Having overviewed it, I don't see any issues (but I'm the author), except

About 0001:,Having overviewed it, I don't see any issues (but I'm the author), except grammatical ones - but I'm not a native to judge it.,Also, the sentence 'turning GROUP BY clauses into pathkeys' i

2024-05-29 Thread Andrei Lepikhov
costs for alternative column orderings. So, I'm okay with the code. But why don't you use the same approach with foreach_ptr as before? -- regards, Andrei Lepikhov Postgres Professional

Re: using extended statistics to improve join estimates

2024-05-22 Thread Andrei Lepikhov
On 5/23/24 09:04, Andy Fan wrote: Andrei Lepikhov writes: * c) No extended stats with MCV. If there are multiple join clauses, * we can try using ndistinct coefficients and do what eqjoinsel does. OK, I didn't pay enough attention to this comment before. and yes, I get the same conclusion

Re: POC: GROUP BY optimization

2024-05-21 Thread Andrei Lepikhov
of sorted columns. -- regards, Andrei Lepikhov Postgres Professional

Re: using extended statistics to improve join estimates

2024-05-21 Thread Andrei Lepikhov
On 5/20/24 16:40, Andrei Lepikhov wrote: On 20/5/2024 15:52, Andy Fan wrote: +    if (clauselist_selectivity_hook) +    *return* clauselist_selectivity_hook(root, clauses, ..) Of course - library may estimate not all the clauses - it is a reason, why I added input/output parameter

Re: using extended statistics to improve join estimates

2024-05-20 Thread Andrei Lepikhov
he clauses - it is a reason, why I added input/output parameter 'estimatedclauses' by analogy with statext_clauselist_selectivity. -- regards, Andrei Lepikhov Postgres Professional

Re: using extended statistics to improve join estimates

2024-05-20 Thread Andrei Lepikhov
. -- regards, Andrei Lepikhov Postgres Professional diff --git a/src/backend/optimizer/path/clausesel.c b/src/backend/optimizer/path/clausesel.c index 0ab021c1e8..271d36a522 100644 --- a/src/backend/optimizer/path/clausesel.c +++ b/src/backend/optimizer/path/clausesel.c @@ -128,6 +128,9

Re: POC: GROUP BY optimization

2024-05-16 Thread Andrei Lepikhov
ebuke - does behavior still the same?. 2. Could you try to find the reason? -- regards, Andrei Lepikhov Postgres Professional

Re: query_id, pg_stat_activity, extended query protocol

2024-05-15 Thread Andrei Lepikhov
se to move pgstat_report_query_id to the ExecutorRun routine? -- regards, Andrei Lepikhov

Re: query_id, pg_stat_activity, extended query protocol

2024-05-15 Thread Andrei Lepikhov
hat's what I precisely wanted to understand: queryId doesn't have semantics to show the job that consumes resources right now—it is mostly about convenience to know that the backend processes nothing except (probably) this query. -- regards, Andrei Lepikhov

Re: query_id, pg_stat_activity, extended query protocol

2024-05-08 Thread Andrei Lepikhov
reset on abortion. -- regards, Andrei Lepikhov diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c index 4d7c92d63c..a4d38a157f 100644 --- a/src/backend/executor/execMain.c +++ b/src/backend/executor/execMain.c @@ -470,6 +470,12 @@ ExecutorEnd(QueryDesc *queryDesc

Re: Removing unneeded self joins

2024-05-06 Thread Andrei Lepikhov
On 6/5/2024 21:44, Robert Haas wrote: On Sat, May 4, 2024 at 10:46 PM Andrei Lepikhov wrote: Having no objective negative feedback, we have no reason to change anything in the design or any part of the code. It looks regrettable and unusual. To me, this sounds like you think it's someone

Re: Removing unneeded self joins

2024-05-04 Thread Andrei Lepikhov
migration policy in some countries, it would be better to work through the online presence as equivalent to offline. Without an online part of the conference, the only way to communicate and discuss is through this mailing list. -- regards, Andrei Lepikhov

Re: Removing unneeded self joins

2024-05-02 Thread Andrei Lepikhov
ements in the planner. As a result, I vote to stay with the feature. But remember, as an author, I'm not entirely objective, so let's wait for alternative opinions. [1] Make Vars be outer-join-aware [2] Improve performance of ORDER BY / DISTINCT aggregates -- regards, Andrei Lepikhov Postgres Professional

Re: Removing unneeded self joins

2024-05-02 Thread Andrei Lepikhov
this subquery and don't execute SJE. Do we need more code, as you have written in the first patch? -- regards, Andrei Lepikhov Postgres Professional From dac8afd2095586921dfcf5564e7f2979e89b77de Mon Sep 17 00:00:00 2001 From: "Andrei V. Lepikhov" Date: Thu, 2 May 2024 16:17:52 +07

Re: query_id, pg_stat_activity, extended query protocol

2024-04-27 Thread Andrei Lepikhov
be considered invalid at that point and raise an error to the user? I don't think so. It may be any object, even stored procedure, that can be changed. IMO, the right option here is to report zero (like the undefined value of queryId) until the end of the parsing stage. -- regards, Andrei Lepikhov

Re: query_id, pg_stat_activity, extended query protocol

2024-04-23 Thread Andrei Lepikhov
On 4/23/24 12:49, Michael Paquier wrote: On Tue, Apr 23, 2024 at 11:42:41AM +0700, Andrei Lepikhov wrote: On 23/4/2024 11:16, Imseih (AWS), Sami wrote: + pgstat_report_query_id(linitial_node(Query, psrc->query_list)->queryId, true); set_ps_display("BIND"); @@

Re: query_id, pg_stat_activity, extended query protocol

2024-04-22 Thread Andrei Lepikhov
hedPlan(), which will validate and lock the plan? What if some OIDs were altered in the middle? -- regards, Andrei Lepikhov

Re: POC: GROUP BY optimization

2024-04-21 Thread Andrei Lepikhov
don't need any ordering procedure at all. And the last thing here — this code introduces the basics needed to add more sophisticated strategies, like ordering according to uniqueness or preferring to set constant-width columns first in grouping. -- regards, Andrei Lepikhov Postgres Professional

Re: POC: GROUP BY optimization

2024-04-18 Thread Andrei Lepikhov
of code more resistant to blunders is another story. -- regards, Andrei Lepikhov Postgres Professional diff --git a/src/backend/optimizer/path/pathkeys.c b/src/backend/optimizer/path/pathkeys.c index aa80f6486c..9c079270ec 100644 --- a/src/backend/optimizer/path/pathkeys.c +++ b/src/backend/o

Re: POC: GROUP BY optimization

2024-04-17 Thread Andrei Lepikhov
ly one reason to do this: adjust_group_pathkeys_for_groupagg. Curious about how much profit we get here, I think we can discover it later with no hurry. A good outcome here will be a test case that can show the importance of arranging GROUP-BY and ORDER-BY at an early stage. -- regards, Andrei Lepikhov Postgre

Re: POC: GROUP BY optimization

2024-04-16 Thread Andrei Lepikhov
. -- regards, Andrei Lepikhov Postgres Professional diff --git a/src/backend/optimizer/path/pathkeys.c b/src/backend/optimizer/path/pathkeys.c index 5d9597adcd..aa80f6486c 100644 --- a/src/backend/optimizer/path/pathkeys.c +++ b/src/backend/optimizer/path/pathkeys.c @@ -380,15 +380,18

Re: POC: GROUP BY optimization

2024-04-16 Thread Andrei Lepikhov
style. To avoid possible usage issues (in extensions, for example), I propose setting it up afterwards, explicitly forcing this action by input parameter - see my attempt in the attachment. -- regards, Andrei Lepikhov Postgres Professional diff --git a/src/backend/optimizer/path/equivclass.c b

Re: POC: GROUP BY optimization

2024-04-11 Thread Andrei Lepikhov
passed through the code being steered by the issues explained in detail. I see seven issues. Two of them definitely should be scrutinised right now, and I'm ready to do that. -- regards, Andrei Lepikhov Postgres Professional

Re: post-freeze damage control

2024-04-09 Thread Andrei Lepikhov
On 9/4/2024 12:55, Tom Lane wrote: Andrei Lepikhov writes: * I really, really dislike jamming this logic into prepqual.c, where it has no business being. I note that it was shoved into process_duplicate_ors without even the courtesy of expanding the header comment: Yeah, I preferred to do

Re: post-freeze damage control

2024-04-08 Thread Andrei Lepikhov
any bugs or strong performance issues, and all the issues can be resolved further; on the other hand, I've got your good review and some ideas to work out. -- regards, Andrei Lepikhov Postgres Professional

Re: type cache cleanup improvements

2024-04-03 Thread Andrei Lepikhov
and reducing the hash table's size might be more efficient. In toto, the 0001-* patch looks good, and I would be glad to see it in the core. [1] https://www.postgresql.org/message-id/flat/CAKcux6ktu-8tefLWtQuuZBYFaZA83vUzuRd7c1YHC-yEWyYFpg%40mail.gmail.com -- regards, Andrei Lepikhov Postgres

Re: Asymmetric partition-wise JOIN

2024-04-01 Thread Andrei Lepikhov
. But as I can predict, it is easier to implement and looks more natural for the architecture. What do you think about that? -- regards, Andrei Lepikhov Postgres Professional

Re: Table AM Interface Enhancements

2024-04-01 Thread Andrei Lepikhov
and avoid huge underestimation. This makes sense, especially for multicolumn filters/clauses. Having a probing AM method, we may invent something for this challenge. -- regards, Andrei Lepikhov Postgres Professional

Re: POC, WIP: OR-clause support for indexes

2024-04-01 Thread Andrei Lepikhov
rray element as an OR clause; we can also provide a BitmapOr path, where SAOP is covered with a minimal number of partial indexes (likewise, previous version). -- regards, Andrei Lepikhov Postgres Professional From e42a7111a12ef82eecdb2e692d65e7ba6e43ad79 Mon Sep 17 00:00:00 2001 From: Alena Rybakina Date

Re: POC, WIP: OR-clause support for indexes

2024-03-18 Thread Andrei Lepikhov
On 14/3/2024 16:31, Alexander Korotkov wrote: On Wed, Mar 13, 2024 at 2:16 PM Andrei Lepikhov As you can see this case is not related to partial indexes.  Just no index selective for the whole query.  However, splitting scan by the OR qual lets use a combination of two selective indexes. I

Re: POC, WIP: OR-clause support for indexes

2024-03-14 Thread Andrei Lepikhov
ust no > index selective for the whole query. However, splitting scan by the > OR qual lets use a combination of two selective indexes. Thanks for the case. I will try to resolve it. -- regards, Andrei Lepikhov Postgres Professional From 156c00c820a38e5e1856f07363af87b3109b5d77 Mon Sep 17

Re: POC, WIP: OR-clause support for indexes

2024-03-14 Thread Andrei Lepikhov
On 14/3/2024 16:31, Alexander Korotkov wrote: On Wed, Mar 13, 2024 at 2:16 PM Andrei Lepikhov mailto:a.lepik...@postgrespro.ru>> wrote: > On 13/3/2024 18:05, Alexander Korotkov wrote: > > On Wed, Mar 13, 2024 at 7:52 AM Andrei Lepikhov > > Given all of the

Re: POC, WIP: OR-clause support for indexes

2024-03-13 Thread Andrei Lepikhov
On 13/3/2024 18:05, Alexander Korotkov wrote: On Wed, Mar 13, 2024 at 7:52 AM Andrei Lepikhov Given all of the above, I think moving transformation to the canonicalize_qual() would be the right way to go. Ok, I will try to move the code. I have no idea about the timings so far. I recall

Re: POC, WIP: OR-clause support for indexes

2024-03-12 Thread Andrei Lepikhov
On 12/3/2024 22:20, Alexander Korotkov wrote: On Mon, Mar 11, 2024 at 2:43 PM Andrei Lepikhov I think you are right. It is probably a better place than any other to remove duplicates in an array. I just think we should sort and remove duplicates from entry->consts in one pass. T

Re: a wrong index choose when statistics is out of date

2024-03-12 Thread Andrei Lepikhov
-picking technique less dependent on the ordering of index lists [1]. [1] https://www.postgresql.org/message-id/9b3dbf6d-776a-4953-a5a4-609929393...@postgrespro.ru -- regards, Andrei Lepikhov Postgres Professional diff --git a/src/backend/utils/adt/like_support.c b/src/backend/utils/adt

Re: POC, WIP: OR-clause support for indexes

2024-03-11 Thread Andrei Lepikhov
kly as possible to increase the effect of the optimization. -- regards, Andrei Lepikhov Postgres Professional

Re: POC, WIP: OR-clause support for indexes

2024-03-10 Thread Andrei Lepikhov
On 7/3/2024 21:51, Alexander Korotkov wrote: Hi! On Tue, Mar 5, 2024 at 9:59 AM Andrei Lepikhov mailto:a.lepik...@postgrespro.ru>> wrote: > On 5/3/2024 12:30, Andrei Lepikhov wrote: > > On 4/3/2024 09:26, jian he wrote: > ... and the new version of the patchset is atta

Re: a wrong index choose when statistics is out of date

2024-03-08 Thread Andrei Lepikhov
On 7/3/2024 17:32, David Rowley wrote: On Thu, 7 Mar 2024 at 21:17, Andrei Lepikhov wrote: I would like to ask David why the var_eq_const estimator doesn't have an option for estimation with a histogram. Having that would relieve a problem with skewed data. Detecting the situation

Re: a wrong index choose when statistics is out of date

2024-03-07 Thread Andrei Lepikhov
al rows=3501 EXPLAIN ANALYZE SELECT * FROM norm_test WHERE val = 700; -- result: planned rows=8604, actual rows=1705 EXPLAIN ANALYZE SELECT * FROM norm_test WHERE val = 1000; -- result: planned rows=8604, actual rows=91 -- regards, Andrei Lepikhov Postgres Professional

Re: "type with xxxx does not exist" when doing ExecMemoize()

2024-03-05 Thread Andrei Lepikhov
On 6/3/2024 10:10, Tender Wang wrote: Andrei Lepikhov <mailto:a.lepik...@postgrespro.ru>> 于2024年3月5日周二 17:36写道: On 1/3/2024 14:18, Tender Wang wrote: > During debug, I learned that numeric_add doesn't have type check like > rangetype, so aboved query will n

Re: Hooking into ExplainOneQuery() complicated by missing standard_ExplainOneQuery

2024-03-05 Thread Andrei Lepikhov
%40garret.ru -- regards, Andrei Lepikhov Postgres Professional

Re: "type with xxxx does not exist" when doing ExecMemoize()

2024-03-05 Thread Andrei Lepikhov
On 1/3/2024 14:18, Tender Wang wrote: During debug, I learned that numeric_add doesn't have type check like rangetype, so aboved query will not report "type with xxx does not exist". And I realize that  the test case added by Andrei Lepikhov  in v3 is right. So in v6 patch I

Re: POC, WIP: OR-clause support for indexes

2024-03-04 Thread Andrei Lepikhov
On 5/3/2024 12:30, Andrei Lepikhov wrote: On 4/3/2024 09:26, jian he wrote: ... and the new version of the patchset is attached. -- regards, Andrei Lepikhov Postgres Professional From 1c3ac3e006cd66ff40f1ddaaa09e3fc0f3a75ca5 Mon Sep 17 00:00:00 2001 From: Alena Rybakina Date: Fri, 2 Feb 2024

Re: POC, WIP: OR-clause support for indexes

2024-03-04 Thread Andrei Lepikhov
On 4/3/2024 09:26, jian he wrote: On Thu, Feb 29, 2024 at 4:59 PM Andrei Lepikhov Feel free to add, change or totally rewrite these changes. On replacement of static ScalarArrayOpExpr dest with dynamic allocated one: After discussion [1] I agree with that replacement. Some style (and language

Re: a wrong index choose when statistics is out of date

2024-03-03 Thread Andrei Lepikhov
of the boundaries when an index shows us that we have min/max outside known statistics? Because it would be used for the values out of the histogram, it should only add an overhead with a reason. -- regards, Andrei Lepikhov Postgres Professional

Re: a wrong index choose when statistics is out of date

2024-03-03 Thread Andrei Lepikhov
in pg_type and a lot of additional code for such a rare case. I'm looking forward to the demo patch. -- regards, Andrei Lepikhov Postgres Professional

Re: POC, WIP: OR-clause support for indexes

2024-03-03 Thread Andrei Lepikhov
more cycles after detection of TYPCATEGORY_ARRAY. I haven't done it yet because have a second thought: what if to combine arrays into the larger one? I'm unsure on that, so we can forbid it too. -- regards, Andrei Lepikhov Postgres Professional

Re: POC, WIP: OR-clause support for indexes

2024-02-29 Thread Andrei Lepikhov
othly. All modifications are integrated into the two new patches. Feel free to add, change or totally rewrite these changes. -- regards, Andrei Lepikhov Postgres Professional From 015a564cc784139c806a7004f25bf5f7a4b4a29d Mon Sep 17 00:00:00 2001 From: Alena Rybakina Date: Fri, 2 Feb 2024 22:01:09 +03

Re: POC, WIP: OR-clause support for indexes

2024-02-28 Thread Andrei Lepikhov
transformBoolExprOr and generate_saop_pathlist (including cross-referencing each other). These are starting points to understand the transformation and, therefore, a good place for a detailed explanation. -- regards, Andrei Lepikhov Postgres Professional

Re: "type with xxxx does not exist" when doing ExecMemoize()

2024-02-27 Thread Andrei Lepikhov
. Also, You don't need to initialize tts_values[i] at all if tts_isnull[i] set to true. -- regards, Andrei Lepikhov Postgres Professional

Re: POC, WIP: OR-clause support for indexes

2024-02-27 Thread Andrei Lepikhov
. We should describe the second part of the feature, where the optimiser can split an array to fit the optimal BitmapOr scan path. -- regards, Andrei Lepikhov Postgres Professional

Re: The const expression evaluation routine should always return a copy

2024-02-27 Thread Andrei Lepikhov
On 28/2/2024 04:19, Tom Lane wrote: Andrei Lepikhov writes: IMO, the routine eval_const_expressions_mutator contains some stale code: I'd like to push back against the idea that eval_const_expressions is expected to return a freshly-copied tree. Its API specification contains no such claim

The const expression evaluation routine should always return a copy

2024-02-26 Thread Andrei Lepikhov
and, as a result, allow to return a Const instead of SubPlan? But at least we can return a flat copy of the SubPplan node just for the convention — the same thing for the AlternativeSubPlan. See the patch in the attachment. -- regards, Andrei Lepikhov Postgres ProfessionalFrom

Re: "type with xxxx does not exist" when doing ExecMemoize()

2024-02-26 Thread Andrei Lepikhov
On 26/2/2024 18:34, Richard Guo wrote: On Mon, Feb 26, 2024 at 3:54 PM Andrei Lepikhov mailto:a.lepik...@postgrespro.ru>> wrote: On 26/2/2024 12:44, Tender Wang wrote: > Make sense. I found MemoizeState already has a MemoryContext, so I used it. > I upda

Re: "type with xxxx does not exist" when doing ExecMemoize()

2024-02-25 Thread Andrei Lepikhov
On 26/2/2024 12:44, Tender Wang wrote: Andrei Lepikhov <mailto:a.lepik...@postgrespro.ru>> 于2024年2月26日周一 10:57写道: On 25/2/2024 20:32, Tender Wang wrote: > I think in prepare_probe_slot(), should called datumCopy as the attached > patch does. >

Re: "type with xxxx does not exist" when doing ExecMemoize()

2024-02-25 Thread Andrei Lepikhov
On 26/2/2024 09:52, Andrei Lepikhov wrote: On 25/2/2024 20:32, Tender Wang wrote: I think in prepare_probe_slot(), should called datumCopy as the attached patch does. Any thoughts? Thanks. Thanks for the report. I think it is better to invent a Runtime Memory Context; likewise

Re: "type with xxxx does not exist" when doing ExecMemoize()

2024-02-25 Thread Andrei Lepikhov
. Here, you just allocate the value in some upper memory context. Also, I'm curious why such a trivial error hasn't been found for a long time -- regards, Andrei Lepikhov Postgres Professional

Re: Removing unneeded self joins

2024-02-22 Thread Andrei Lepikhov
style, but technically, it's still OK. -- regards, Andrei Lepikhov Postgres Professional

Re: POC: GROUP BY optimization

2024-02-21 Thread Andrei Lepikhov
Maxim will answer that it's enough to resolve his issue, why not? -- regards, Andrei Lepikhov Postgres Professional

Re: Unlinking Parallel Hash Join inner batch files sooner

2024-02-21 Thread Andrei Lepikhov
On 22/2/2024 06:42, Thomas Munro wrote: On Wed, Feb 21, 2024 at 7:34 PM Andrei Lepikhov wrote: I see in [1] that the reporter mentioned a delay between the error message in parallel HashJoin and the return control back from PSQL. Your patch might reduce this delay. Also, I have the same

Re: POC: GROUP BY optimization

2024-02-21 Thread Andrei Lepikhov
org/message-id/CACG=ezaYM1tr6Lmp8PRH1aeZq=rbkxeotwgzmclad5mphfw...@mail.gmail.com -- regards, Andrei Lepikhov Postgres Professional

Re: Removing unneeded self joins

2024-02-21 Thread Andrei Lepikhov
need two different subtrees for the same query. I will look into your fix. -- regards, Andrei Lepikhov Postgres Professional

Re: Unlinking Parallel Hash Join inner batch files sooner

2024-02-20 Thread Andrei Lepikhov
stuck into the unlink of tons of temporary files. So, are you going to do something with this code? [1] https://www.postgresql.org/message-id/18349-83d33dd3d0c855c3%40postgresql.org -- regards, Andrei Lepikhov Postgres Professional

Re: [POC] Allow flattening of subquery with a link to upper query

2024-02-20 Thread Andrei Lepikhov
On 20/2/2024 17:43, David Rowley wrote: On Tue, 20 Feb 2024 at 22:57, Andrei Lepikhov wrote: I agree that it would be nice to teach the planner how to do this, but I think it just has to be a cost-based decision. Imagine how the transformed query would perform of pg_am had a billion rows

Re: [POC] Allow flattening of subquery with a link to upper query

2024-02-20 Thread Andrei Lepikhov
links from a non-parent query block. So, in my opinion, the reason for this patch still exists, and we can continue this work further, maybe elaborating on flattening LATERAL references - this needs some research. [1] https://www.postgresql.org/message-id/35c8a3e8-d080-dfa8-2be3-cf5fe702010a%40postgr

Re: Optimize planner memory consumption for huge arrays

2024-02-19 Thread Andrei Lepikhov
design small memory contexts - one per scalable direction of memory utilization, like selectivity or partitioning (appending ?). My coding experience shows that short-lived GEQO memory context forces people to learn on Postgres internals more precisely :). -- regards, Andrei Lepikhov Postgres

Re: POC, WIP: OR-clause support for indexes

2024-02-19 Thread Andrei Lepikhov
eliminate many calls of the equal() routine, too. `leftop operator rightop` the operator can also be volatile. Do we need to check (op_volatile(opno) == PROVOLATILE_VOLATILE) within transformBoolExprOr? As usual, could you provide a test case to discuss it more objectively? -- regards, Andrei

Re: Optimize planner memory consumption for huge arrays

2024-02-19 Thread Andrei Lepikhov
for the next array element. It finds this piece more quickly than before that optimization. -- regards, Andrei Lepikhov Postgres Professional From 2e89dc8b743953068174c777d7a014e1ea71f659 Mon Sep 17 00:00:00 2001 From: "Andrey V. Lepikhov" Date: Tue, 20 Feb 2024 11:05:51 +0700 Subject: [PATC

Re: POC, WIP: OR-clause support for indexes

2024-02-19 Thread Andrei Lepikhov
ate in this operation */ + continue; + arrayconst = lsecond_node(Const, saop->args); + dest = makeNode(ScalarArrayOpExpr); Thanks for the review! I'm not sure I understand you clearly. Does the patch in attachment fix the issue you raised? -- regards, Andrei Lepikhov Postgres Professional diff --git

Re: Memory consumed by paths during partitionwise join planning

2024-02-19 Thread Andrei Lepikhov
On 19/2/2024 19:25, Ashutosh Bapat wrote: On Fri, Feb 16, 2024 at 8:42 AM Andrei Lepikhov wrote: Live example: right now, I am working on the code like MSSQL has - a combination of NestLoop and HashJoin paths and switching between them in real-time. It requires both paths in the path list

Re: POC, WIP: OR-clause support for indexes

2024-02-19 Thread Andrei Lepikhov
. As I see it, the only general explanation of the idea is not addressed. I'm not sure how deeply we should explain it. -- regards, Andrei Lepikhov Postgres Professional From 3a3b6aa36320a06b64f2f608e3526255e53ed655 Mon Sep 17 00:00:00 2001 From: Alena Rybakina Date: Fri, 2 Feb 2024 22:01:09 +0300

Re: Removing unneeded self joins

2024-02-18 Thread Andrei Lepikhov
, please, recheck? I reviewed this patch. Why do you check only the target list? I guess these links can be everywhere. See the patch in the attachment with the elaborated test and slightly changed code. -- regards, Andrei Lepikhov Postgres Professional From 7f94a3c96fd410522b87e570240cdb96b300dd31

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2024-02-18 Thread Andrei Lepikhov
partitions, we should introduce per-partition memory context and reset it in between. GEQO already has a short-lived memory context, making designing extensions a bit more challenging but nothing too painful. -- regards, Andrei Lepikhov Postgres Professional

Re: POC, WIP: OR-clause support for indexes

2024-02-15 Thread Andrei Lepikhov
On 16/2/2024 07:00, jian he wrote: On Wed, Feb 14, 2024 at 11:21 AM Andrei Lepikhov wrote: My OS: Ubuntu 22.04.3 LTS I already set the max_parallel_workers_per_gather to 10. So for all cases, it should use parallelism first? a better question would be: how to make the number of OR less than 29

Re: Memory consumed by paths during partitionwise join planning

2024-02-15 Thread Andrei Lepikhov
On 15/2/2024 19:06, Ashutosh Bapat wrote: On Thu, Feb 15, 2024 at 9:41 AM Andrei Lepikhov But I'm not sure about freeing unreferenced paths. I would have to see alternatives in the pathlist. I didn't understand this. Can you please elaborate? A path in any pathlist is referenced

Re: planner chooses incremental but not the best one

2024-02-15 Thread Andrei Lepikhov
On 15/2/2024 18:10, Tomas Vondra wrote: On 2/15/24 07:50, Andrei Lepikhov wrote: On 18/12/2023 19:53, Tomas Vondra wrote: On 12/18/23 11:40, Richard Guo wrote: The challenge is where to get usable information about correlation between columns. I only have a couple very rought ideas of what

Re: planner chooses incremental but not the best one

2024-02-14 Thread Andrei Lepikhov
directly checking appendrel->tuples correction. -- regards, Andrei Lepikhov Postgres Professional

Re: planner chooses incremental but not the best one

2024-02-14 Thread Andrei Lepikhov
f grouping? -- regards, Andrei Lepikhov Postgres Professional

Re: Memory consumed by paths during partitionwise join planning

2024-02-14 Thread Andrei Lepikhov
/CAOP8fzaVL_2SCJayLL9kj5pCA46PJOXXjuei6-3aFUV45j4LJQ%40mail.gmail.com [2] https://www.postgresql.org/message-id/flat/CAMbWs496%2BN%3DUAjOc%3DrcD3P7B6oJe4rZw08e_TZRUsWbPxZW3Tw%40mail.gmail.com -- regards, Andrei Lepikhov Postgres Professional

Re: Memory consumed by child SpecialJoinInfo in partitionwise join planning

2024-02-13 Thread Andrei Lepikhov
On 14/2/2024 13:32, Ashutosh Bapat wrote: On Wed, Feb 14, 2024 at 9:50 AM Andrei Lepikhov wrote: On 30/1/2024 12:44, Ashutosh Bapat wrote: Thanks Vignesh. PFA patches rebased on the latest HEAD. The patch addressing Amit's comments is still a separate patch for him to review. Thanks

  1   2   >