Re: Implement missing join selectivity estimation for range types

2024-01-22 Thread jian he
I cannot figure out why it aborts. as Tom mentioned in upthread about the test cases. similar to src/test/regress/sql/stats_ext.sql check_estimated_rows function. we can test it by something: create or replace function check_estimated_rows(text) returns table (ok bool) language plpgsql as $$

Re: Implement missing join selectivity estimation for range types

2024-01-17 Thread vignesh C
On Fri, 5 Jan 2024 at 23:09, Schoemans Maxime wrote: > > On 05/01/2024 11:37, vignesh C wrote: > > One of the tests was aborted at [1], kindly post an updated patch for > the same: > > Thank you for notifying us. > I believe I fixed the issue but it is hard to be certain as the issue > did not

Re: Implement missing join selectivity estimation for range types

2024-01-05 Thread Schoemans Maxime
On 05/01/2024 11:37, vignesh C wrote: > One of the tests was aborted at [1], kindly post an updated patch for the same: Thank you for notifying us. I believe I fixed the issue but it is hard to be certain as the issue did not arise when running the regression tests locally. Regards,

Re: Implement missing join selectivity estimation for range types

2024-01-05 Thread vignesh C
On Tue, 21 Nov 2023 at 01:47, Schoemans Maxime wrote: > > On 14/11/2023 20:46, Tom Lane wrote: > > I took a brief look through this very interesting work. I concur > > with Tomas that it feels a little odd that range join selectivity > > would become smarter than scalar inequality join

Re: Implement missing join selectivity estimation for range types

2023-11-30 Thread Alena Rybakina
Hi! Thank you for your work on the subject, I think it's a really useful feature and it allows optimizer to estimate more correctly clauses with such type of operator. I rewieved your patch and noticed that some comments are repeated into multirangejoinsel functions, I suggest combining

Re: Implement missing join selectivity estimation for range types

2023-11-20 Thread Schoemans Maxime
On 14/11/2023 20:46, Tom Lane wrote: > I took a brief look through this very interesting work. I concur > with Tomas that it feels a little odd that range join selectivity > would become smarter than scalar inequality join selectivity, and > that we really ought to prioritize applying these

Re: Implement missing join selectivity estimation for range types

2023-11-14 Thread Tom Lane
Schoemans Maxime writes: > You can find attached a new version of the patch that can be applied on > the current master branch of postgres. I took a brief look through this very interesting work. I concur with Tomas that it feels a little odd that range join selectivity would become smarter

Re: Implement missing join selectivity estimation for range types

2023-07-07 Thread Schoemans Maxime
Hi, Thank you for picking up this patch. > The patch doesn't apply to the current postgres version. Could you please update it? Indeed, the code was initially written on pg15. You can find attached a new version of the patch that can be applied on the current master branch of postgres.

Re: Implement missing join selectivity estimation for range types

2023-07-07 Thread Damir Belyalov
Hello! Thank you for the patch, very interesting article. The patch doesn't apply to the current postgres version. Could you please update it? Regards, Damir Belyalov, Postgres Professional

Re: Implement missing join selectivity estimation for range types

2023-06-19 Thread Schoemans Maxime
This is a quick correction as the last patch contained a missing semicolon. Regards, Maxime SchoemansFrom ebd62356210eff2f38772a9c46a0a8792c0e9ce3 Mon Sep 17 00:00:00 2001 From: Maxime Schoemans Date: Mon, 20 Mar 2023 11:48:05 -0400 Subject: [PATCH v2] Apply division before adding remainder ---

Re: Implement missing join selectivity estimation for range types

2023-06-19 Thread Schoemans Maxime
Hi, In the selectivity algorithm, the division was applied after adding the remaining histogram buckets of histogram2 that don't overlap with histogram1. This could lead to reducing selectivity by half, e.g., in the case that histogram2 is completely right of histogram1. The correct calculation

Re: Implement missing join selectivity estimation for range types

2023-03-20 Thread Schoemans Maxime
Hi Tomas, As a quick update, the paper related to this work has finally been published in Mathematics (https://www.mdpi.com/2227-7390/11/6/1383). During revision we also added a figure showing a comparison of our algorithm vs the existing algorithms in Oracle, SQL Server, MySQL and PostgreSQL,

Re: Implement missing join selectivity estimation for range types

2023-01-25 Thread Mahmoud Sakr
Hi Tomas, > I finally had time to properly read the paper today - the general > approach mostly matches how I imagined the estimation would work for > inequalities, but it's definitely nice to see the algorithm properly > formalized and analyzed. Awesome, thanks for this interest! > What seems

Re: Implement missing join selectivity estimation for range types

2023-01-21 Thread Tomas Vondra
Hi Mahmoud, I finally had time to properly read the paper today - the general approach mostly matches how I imagined the estimation would work for inequalities, but it's definitely nice to see the algorithm properly formalized and analyzed. What seems a bit strange to me is that the patch only

Re: Implement missing join selectivity estimation for range types

2023-01-20 Thread Tomas Vondra
On 1/18/23 20:23, Mahmoud Sakr wrote: > Hi Tomas, > Thanks for picking up the patch and for the interesting discussions that > you bring ! > >> Interesting. Are there any particular differences compared to how we >> estimate for example range clauses on regular columns? > > The theory is the same

Re: Implement missing join selectivity estimation for range types

2023-01-18 Thread Mahmoud Sakr
Hi Tomas, Thanks for picking up the patch and for the interesting discussions that you bring ! > Interesting. Are there any particular differences compared to how we > estimate for example range clauses on regular columns? The theory is the same for scalar types. Yet, the statistics that are

Re: Implement missing join selectivity estimation for range types

2023-01-18 Thread Tomas Vondra
Also, calc_hist_selectivity_contains in multirangetypes_selfuncs.c needs a proper comment, not just "this is a copy from rangetypes". However, it seems the two functions are exactly the same. Would the functions diverge in the future? If not, maybe there should be just a single shared function?

Re: Implement missing join selectivity estimation for range types

2023-01-18 Thread Tomas Vondra
Hello Mahmoud, Thanks for the patch and sorry for not taking a look earlier. On 6/30/22 16:31, Mahmoud Sakr wrote: > Hi, > Given a query: > SELECT * FROM t1, t2 WHERE t1.r << t2.r > where t1.r, t2.r are of range type, > currently PostgreSQL will estimate a constant selectivity for the << >

Implement missing join selectivity estimation for range types

2022-06-30 Thread Mahmoud Sakr
Hi, Given a query: SELECT * FROM t1, t2 WHERE t1.r << t2.r where t1.r, t2.r are of range type, currently PostgreSQL will estimate a constant selectivity for the << predicate, which is equal to 0.005, not utilizing the statistics that the optimizer collects for range attributes. We have worked out