Re: Multi-Column List Partitioning

2022-01-14 Thread Julien Rouhaud
Hi, The cfbot reports some clang warning on the last version of the patchset: https://cirrus-ci.com/task/6721617647632384 [16:35:24.444] partprune.c:2775:8: error: explicitly assigning value of variable of type 'int' to itself [-Werror,-Wself-assign] [16:35:24.444] off

Re: Multi-Column List Partitioning

2022-01-02 Thread Amul Sul
On Wed, Dec 29, 2021 at 7:26 PM Nitin Jadhav wrote: > > > > -* For range partitioning, we must only perform pruning with values > > -* for either all partition keys or a prefix thereof. > > +* For range partitioning and list partitioning, we must only > > perform > > +

Re: Multi-Column List Partitioning

2021-12-23 Thread Amul Sul
ioning key & pass an additional argument. > > Also, no information about the length of the "isnulls" array. > > This is required during merge_list_bounds(). AFAIK partition key > information is not available here. > You can get that as an argument, see merge_range_bo

Re: Multi-Column List Partitioning

2021-12-21 Thread Amit Langote
---+--+- a | integer | | | Check constraints: "foo_a_check" CHECK (a = 1 AND a = 2) Maybe partitioning should be looked at differently than the free-form CHECK constraints, but I'm not so sure. Or if others insist that it may be worthwhile to improve the user experience in such cases, we could do that as a separate patch than the patch to implement multi-column list partitioning. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Multi-Column List Partitioning

2021-12-20 Thread Ashutosh Sharma
On Mon, Dec 20, 2021 at 7:04 PM Amit Langote wrote: > Hi, > > On Mon, Dec 13, 2021 at 11:37 PM Ashutosh Sharma > wrote: > > > > Hi, > > > > Is this okay? > > > > postgres=# CREATE TABLE t1 (a int, b int) PARTITION BY LIST ( a, a, a ); > > CREATE TABLE > > > > postgres=# CREATE TABLE t1_1

Re: Multi-Column List Partitioning

2021-12-20 Thread Amit Langote
Hi, On Mon, Dec 13, 2021 at 11:37 PM Ashutosh Sharma wrote: > > Hi, > > Is this okay? > > postgres=# CREATE TABLE t1 (a int, b int) PARTITION BY LIST ( a, a, a ); > CREATE TABLE > > postgres=# CREATE TABLE t1_1 PARTITION OF t1 FOR VALUES IN ((1, 2, 3), (4, 5, > 6)); > CREATE TABLE > >

Re: Multi-Column List Partitioning

2021-12-13 Thread Ashutosh Sharma
Hi, Is this okay? postgres=# CREATE TABLE t1 (a int, b int) PARTITION BY LIST ( a, a, a ); CREATE TABLE postgres=# CREATE TABLE t1_1 PARTITION OF t1 FOR VALUES IN ((1, 2, 3), (4, 5, 6)); CREATE TABLE postgres=# \d t1 Partitioned table "public.t1" Column | Type | Collation |

Re: Multi-Column List Partitioning

2021-12-09 Thread Amul Sul
On Thu, Dec 9, 2021 at 12:43 PM Amul Sul wrote: > > On Thu, Dec 9, 2021 at 12:03 PM Amit Langote wrote: > > > > On Thu, Dec 9, 2021 at 3:12 PM Amul Sul wrote: > > > On Thu, Dec 9, 2021 at 11:24 AM Amit Langote > > > wrote: > > > > > > > [] > > > > On Mon, Dec 6, 2021 at 10:57 PM Nitin

Re: Multi-Column List Partitioning

2021-12-08 Thread Amul Sul
On Thu, Dec 9, 2021 at 12:03 PM Amit Langote wrote: > > On Thu, Dec 9, 2021 at 3:12 PM Amul Sul wrote: > > On Thu, Dec 9, 2021 at 11:24 AM Amit Langote > > wrote: > > > > > [] > > > On Mon, Dec 6, 2021 at 10:57 PM Nitin Jadhav > > > wrote: > > > > > Looks difficult to understand at first

Re: Multi-Column List Partitioning

2021-12-08 Thread Amit Langote
On Thu, Dec 9, 2021 at 3:12 PM Amul Sul wrote: > On Thu, Dec 9, 2021 at 11:24 AM Amit Langote wrote: > > > [] > > On Mon, Dec 6, 2021 at 10:57 PM Nitin Jadhav > > wrote: > > > > Looks difficult to understand at first glance, how about the following: > > > > > > > > if (b1->isnulls !=

Re: Multi-Column List Partitioning

2021-12-08 Thread Amul Sul
On Thu, Dec 9, 2021 at 11:24 AM Amit Langote wrote: > [] > On Mon, Dec 6, 2021 at 10:57 PM Nitin Jadhav > wrote: > > > Looks difficult to understand at first glance, how about the following: > > > > > > if (b1->isnulls != b2->isnulls) > > >return false; > > I don't think having this

Re: Multi-Column List Partitioning

2021-12-08 Thread Amit Langote
On Thu, Dec 9, 2021 at 2:54 PM Amit Langote wrote: > > Hi Nitin, > > Was looking at warnings generated by v8: > > partbounds.c:971:17: warning: unused variable 'b1_isnull' [-Wunused-variable] > boolb1_isnull = false; >

Re: Multi-Column List Partitioning

2021-12-08 Thread Amit Langote
Hi Nitin, Was looking at warnings generated by v8: partbounds.c:971:17: warning: unused variable 'b1_isnull' [-Wunused-variable] boolb1_isnull = false; ^ partbounds.c:972:17: warning: unused variable 'b2_isnull'

Re: Multi-Column List Partitioning

2021-12-06 Thread Amul Sul
On Mon, Dec 6, 2021 at 7:27 PM Nitin Jadhav wrote: > > Thank you for reviewing the patch. > > > partbounds.c: In function ‘get_qual_for_list.isra.18’: > > partbounds.c:4284:29: warning: ‘boundinfo’ may be used uninitialized > > in this function [-Wmaybe-uninitialized] > >

Re: Multi-Column List Partitioning

2021-12-03 Thread Amul Sul
Hi, Few comments for v7 patch, note that I haven't been through the previous discussion, if any of the review comments that has been already discussed & overridden, then please ignore here too: partbounds.c: In function ‘get_qual_for_list.isra.18’: partbounds.c:4284:29: warning: ‘boundinfo’ may

Re: Multi-Column List Partitioning

2021-11-02 Thread Nitin Jadhav
> I noticed that there's no commitfest entry for this. Will you please > add this to the next one? I have added it to Nov commitfest. Thanks & Regards, Nitin Jadhav On Fri, Oct 29, 2021 at 1:40 PM Amit Langote wrote: > > Hi Nitin, > > On Fri, Oct 22, 2021 at 6:48 PM Nitin Jadhav > wrote: > >

Re: Multi-Column List Partitioning

2021-10-29 Thread Amit Langote
Hi Nitin, On Fri, Oct 22, 2021 at 6:48 PM Nitin Jadhav wrote: > Thanks for sharing. I have fixed the issue in the attached patch. I noticed that there's no commitfest entry for this. Will you please add this to the next one? -- Amit Langote EDB: http://www.enterprisedb.com

Re: Multi-Column List Partitioning

2021-10-22 Thread Zhihong Yu
On Fri, Oct 22, 2021 at 3:50 AM Zhihong Yu wrote: > > > On Fri, Oct 22, 2021 at 2:48 AM Nitin Jadhav < > nitinjadhavpostg...@gmail.com> wrote: > >> > While testing further I got a crash with partition wise join enabled >> for multi-col list partitions. please find test case & stack-trace below.

Re: Multi-Column List Partitioning

2021-10-22 Thread Zhihong Yu
On Fri, Oct 22, 2021 at 2:48 AM Nitin Jadhav wrote: > > While testing further I got a crash with partition wise join enabled for > multi-col list partitions. please find test case & stack-trace below. > > Thanks for sharing. I have fixed the issue in the attached patch. > > Thanks & Regards, >

Re: Multi-Column List Partitioning

2021-10-15 Thread Amit Langote
Hi Rajkumar, On Mon, Oct 11, 2021 at 2:36 PM Rajkumar Raghuwanshi wrote: > > Thanks for the patch, it applied cleanly and fixed the reported issue. I > observed another case where > In case of multi-col list partition on the same column query is not picking > partition wise join. Is this

Re: Multi-Column List Partitioning

2021-10-11 Thread Rajkumar Raghuwanshi
rtition by list has only 1 >>> column but we are giving 2 values. >>> also if you see \d+ showing plt1_p1 partition value as ‘(0001,0001)’ >>> instead of ('0001','0001'). >>> >>> Thanks & Regards, >>> Rajkumar Raghuwanshi >>> >&g

Re: Multi-Column List Partitioning

2021-10-10 Thread Rajkumar Raghuwanshi
;> >>> > > On PG head + Nitin's v3 patch + Amit's Delta patch. Make check is >>> failing with below errors. >>> > >>> > Thanks Rajkumar for testing. >>> > >>> > Here's a v2 of the delta patch that should fix both of these test >

Re: Multi-Column List Partitioning

2021-10-07 Thread Rajkumar Raghuwanshi
> > themselves. Especially, I hadn't bothered to investigate various /* > > TODO: handle multi-column list partitioning */ sites to deal with my > > own changes. > > Thanks Rajkumar for testing and Thank you Amit for working on v2 of > the delta patch. Actually I had

Re: Multi-Column List Partitioning

2021-10-01 Thread Zhihong Yu
elta patch fixed what > I think were problems in Nitin's v3 patch but were not complete by > themselves. Especially, I hadn't bothered to investigate various /* > TODO: handle multi-column list partitioning */ sites to deal with my > own changes. > > In the attached updated version, I've

Re: Multi-Column List Partitioning

2021-10-01 Thread Amit Langote
artitionwise_join set to true. Here's a v2 of the delta patch that should fix both of these test failures. As I mentioned in my last reply, my delta patch fixed what I think were problems in Nitin's v3 patch but were not complete by themselves. Especially, I hadn't bothered to investigat

Re: Multi-Column List Partitioning

2021-09-13 Thread Rajkumar Raghuwanshi
On PG head + Nitin's v3 patch + Amit's Delta patch. Make check is failing with below errors. --inherit.sql is failing with error :"ERROR: negative bitmapset member not allowed" update mlparted_tab mlp set c = 'xxx' from (select a from some_tab union all select a+1 from some_tab) ss (a) where

Re: Multi-Column List Partitioning

2021-09-03 Thread Amit Langote
On Wed, Sep 1, 2021 at 2:31 PM Amit Langote wrote: > On Tue, Aug 31, 2021 at 8:02 PM Nitin Jadhav > wrote: > > The attached patch also fixes the above comments. > > I noticed that multi-column list partitions containing NULLs don't > work correctly with partition pruning yet. > > create table p0

Re: Multi-Column List Partitioning

2021-08-31 Thread Amit Langote
Hi Nitin, On Tue, Aug 31, 2021 at 8:02 PM Nitin Jadhav wrote: > The attached patch also fixes the above comments. I noticed that multi-column list partitions containing NULLs don't work correctly with partition pruning yet. create table p0 (a int, b text, c bool) partition by list (a, b, c);

Re: Multi-Column List Partitioning

2021-08-30 Thread Amit Langote
On Mon, Aug 30, 2021 at 4:51 PM Rajkumar Raghuwanshi wrote: > > Hi Nitin. > > I have been testing these patches. Patches applied cleanly on the head. While > testing I found below a case where update row movement is not working > properly. > Please find the test case below. > > postgres=#

Re: Multi-Column List Partitioning

2021-08-30 Thread Rajkumar Raghuwanshi
s worth the complexity of the implementation? That is, is the use > >> > case to search for only a subset of partition key columns common > >> > enough with list partitioning? > >> > > >> > If we do decide to implement the special case, remember that t

Re: Multi-Column List Partitioning

2021-08-27 Thread Nitin Jadhav
_MATCH_CLAUSE, >> PARTCLAUSE_MATCH_NULLNESS, PARTCLAUSE_NOMATCH, etc. >> 2. In case of PARTCLAUSE_MATCH_CLAUSE, we generate steps using >> gen_prune_steps_from_opexps() (strategy-2) which generate and return a >> list of PartitionPruneStepOp that are based on OpExpr and Bool

Re: Multi-Column List Partitioning

2021-08-25 Thread Zhihong Yu
ums are > > sorted. That is, match all partitions when the query only contains a > > clause for b when the partition key is (a, b, c), but engage the > > special case of pruning if the query contains clauses for a, or for a > > and b. > > Thanks for the suggestion. Below is th

Re: Multi-Column List Partitioning

2021-06-11 Thread Zhihong Yu
tation > > details. > > Thanks for the updated version. > > > 1. Regarding syntax, the existing syntax will work fine for the > > single-column list partitioning. However I have used the new syntax > > for the multi-column list partitioning as we discussed earlier.

Re: Multi-Column List Partitioning

2021-06-11 Thread Amit Langote
On Fri, Jun 11, 2021 at 12:37 PM Amit Langote wrote: > I will look at other parts of the patch next week hopefully. For > now, attached is a delta patch that applies on top of your v1, which > does: > > * Simplify partition_list_bsearch() and partition_lbound_datum_cmp() > * Make

Re: Multi-Column List Partitioning

2021-06-10 Thread Amit Langote
rk fine for the > single-column list partitioning. However I have used the new syntax > for the multi-column list partitioning as we discussed earlier. I have > used a combination of 'AND' and 'OR' logic for the partition > constraints as given in the below example. > > postgres@1750

Re: Multi-Column List Partitioning

2021-06-03 Thread Nitin Jadhav
d patch contains more changes compared to the previous patch. Following are the implementation details. 1. Regarding syntax, the existing syntax will work fine for the single-column list partitioning. However I have used the new syntax for the multi-column list partitioning as we discussed earlier. I ha

Re: Multi-Column List Partitioning

2021-05-25 Thread Amit Langote
on't put the NULL value in the 'datums' > field of 'PartitionBoundInfoData' structure [2]. Since there can be > only one NULL value, we directly store the corresponding index value > in the 'null_index' field. Now we have to handle multiple NULL values > in case of Multi-Column L

Re: Multi-Column List Partitioning

2021-05-23 Thread Nitin Jadhav
structure [2]. Since there can be only one NULL value, we directly store the corresponding index value in the 'null_index' field. Now we have to handle multiple NULL values in case of Multi-Column List Partitioning. So the question is how to handle this scenario. Following are the 2 approaches t

Re: Multi-Column List Partitioning

2021-05-21 Thread Amit Langote
On Fri, May 21, 2021 at 1:02 PM Amit Langote wrote: > I will now take a look at the patch itself. Some quick observations: * I get a lot of instances of the following 2 warnings when compiling the patched code: Warning #1: partprune.c: In function ‘get_matching_list_bounds’:

Re: Multi-Column List Partitioning

2021-05-20 Thread Amit Langote
ng on this feature. I also feel that > 'Multi-Column List Partitioning' can be benefited to the Postgres users in > future. Yes, it would be nice to have this. Thanks for picking this up. > I am attaching the WIP patch for this feature here. It supports 'Multi-Column > List Par

Re: Multi-Column List Partitioning

2021-05-08 Thread Nitin Jadhav
n (port=0x55779e358380) at > postmaster.c:4488 > #24 0x55779d066d8c in BackendStartup (port=0x55779e358380) at > postmaster.c:4210 > #25 0x55779d062f9b in ServerLoop () at postmaster.c:1742 > #26 0x55779d062734 in PostmasterMain (argc=3, argv=0x55779e3308b0) at > postmaste

Re: Multi-Column List Partitioning

2021-05-07 Thread Jeevan Ladhe
> While reviewing one of the 'Table partitioning' related patches, > I found that Postgres does not support multiple column based LIST > partitioning. Based on this understanding, I have started working on > this feature. I also feel that 'Multi-Column List Partitioning' can &g

Multi-Column List Partitioning

2021-05-06 Thread Nitin Jadhav
Hi, While reviewing one of the 'Table partitioning' related patches, I found that Postgres does not support multiple column based LIST partitioning. Based on this understanding, I have started working on this feature. I also feel that 'Multi-Column List Partitioning' can be benefited