[jira] [Updated] (DRILL-6896) Extraneous columns being projected past a join
[ https://issues.apache.org/jira/browse/DRILL-6896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pritesh Maker updated DRILL-6896: - Fix Version/s: (was: 1.16.0) Future > Extraneous columns being projected past a join > -- > > Key: DRILL-6896 > URL: https://issues.apache.org/jira/browse/DRILL-6896 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.15.0 >Reporter: Karthikeyan Manivannan >Assignee: Aman Sinha >Priority: Major > Fix For: Future > > > [~rhou] noted that TPCH13 on Drill 1.15 was running slower than Drill 1.14. > Analysis revealed that an extra column was being projected in 1.15 and the > slowdown was because the extra column was being unnecessarily pushed across > an exchange. > Here is a simplified query written by [~amansinha100] that exhibits the same > problem : > In first plan, o_custkey and o_comment are both extraneous projections. > In the second plan (on 1.14.0), also, there is an extraneous projection: > o_custkey but not o_comment. > On 1.15.0: > - > {noformat} > explain plan without implementation for > select > c.c_custkey > from >cp.`tpch/customer.parquet` c > left outer join cp.`tpch/orders.parquet` o > on c.c_custkey = o.o_custkey > and o.o_comment not like '%special%requests%' >; > DrillScreenRel > DrillProjectRel(c_custkey=[$0]) > DrillProjectRel(c_custkey=[$2], o_custkey=[$0], o_comment=[$1]) > DrillJoinRel(condition=[=($2, $0)], joinType=[right]) > DrillFilterRel(condition=[NOT(LIKE($1, '%special%requests%'))]) > DrillScanRel(table=[[cp, tpch/orders.parquet]], > groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath > [path=classpath:/tpch/orders.parquet]], > selectionRoot=classpath:/tpch/orders.parquet, numFiles=1, numRowGroups=1, > usedMetadataFile=false, columns=[`o_custkey`, `o_comment`]]]) > DrillScanRel(table=[[cp, tpch/customer.parquet]], > groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath > [path=classpath:/tpch/customer.parquet]], > selectionRoot=classpath:/tpch/customer.parquet, numFiles=1, numRowGroups=1, > usedMetadataFile=false, columns=[`c_custkey`]]]) > {noformat} > On 1.14.0: > - > {noformat} > DrillScreenRel > DrillProjectRel(c_custkey=[$0]) > DrillProjectRel(c_custkey=[$1], o_custkey=[$0]) > DrillJoinRel(condition=[=($1, $0)], joinType=[right]) > DrillProjectRel(o_custkey=[$0]) > DrillFilterRel(condition=[NOT(LIKE($1, '%special%requests%'))]) > DrillScanRel(table=[[cp, tpch/orders.parquet]], > groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath > [path=classpath:/tpch/orders.parquet]], > selectionRoot=classpath:/tpch/orders.parquet, numFiles=1, numRowGroups=1, > usedMetadataFile=false, columns=[`o_custkey`, `o_comment`]]]) > DrillScanRel(table=[[cp, tpch/customer.parquet]], > groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath > [path=classpath:/tpch/customer.parquet]], > selectionRoot=classpath:/tpch/customer.parquet, numFiles=1, numRowGroups=1, > usedMetadataFile=false, columns=[`c_custkey`]]]) > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6896) Extraneous columns being projected past a join
[ https://issues.apache.org/jira/browse/DRILL-6896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aman Sinha updated DRILL-6896: -- Fix Version/s: 1.16.0 > Extraneous columns being projected past a join > -- > > Key: DRILL-6896 > URL: https://issues.apache.org/jira/browse/DRILL-6896 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.15.0 >Reporter: Karthikeyan Manivannan >Assignee: Aman Sinha >Priority: Major > Fix For: 1.16.0 > > > [~rhou] noted that TPCH13 on Drill 1.15 was running slower than Drill 1.14. > Analysis revealed that an extra column was being projected in 1.15 and the > slowdown was because the extra column was being unnecessarily pushed across > an exchange. > Here is a simplified query written by [~amansinha100] that exhibits the same > problem : > In first plan, o_custkey and o_comment are both extraneous projections. > In the second plan (on 1.14.0), also, there is an extraneous projection: > o_custkey but not o_comment. > On 1.15.0: > - > {noformat} > explain plan without implementation for > select > c.c_custkey > from >cp.`tpch/customer.parquet` c > left outer join cp.`tpch/orders.parquet` o > on c.c_custkey = o.o_custkey > and o.o_comment not like '%special%requests%' >; > DrillScreenRel > DrillProjectRel(c_custkey=[$0]) > DrillProjectRel(c_custkey=[$2], o_custkey=[$0], o_comment=[$1]) > DrillJoinRel(condition=[=($2, $0)], joinType=[right]) > DrillFilterRel(condition=[NOT(LIKE($1, '%special%requests%'))]) > DrillScanRel(table=[[cp, tpch/orders.parquet]], > groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath > [path=classpath:/tpch/orders.parquet]], > selectionRoot=classpath:/tpch/orders.parquet, numFiles=1, numRowGroups=1, > usedMetadataFile=false, columns=[`o_custkey`, `o_comment`]]]) > DrillScanRel(table=[[cp, tpch/customer.parquet]], > groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath > [path=classpath:/tpch/customer.parquet]], > selectionRoot=classpath:/tpch/customer.parquet, numFiles=1, numRowGroups=1, > usedMetadataFile=false, columns=[`c_custkey`]]]) > {noformat} > On 1.14.0: > - > {noformat} > DrillScreenRel > DrillProjectRel(c_custkey=[$0]) > DrillProjectRel(c_custkey=[$1], o_custkey=[$0]) > DrillJoinRel(condition=[=($1, $0)], joinType=[right]) > DrillProjectRel(o_custkey=[$0]) > DrillFilterRel(condition=[NOT(LIKE($1, '%special%requests%'))]) > DrillScanRel(table=[[cp, tpch/orders.parquet]], > groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath > [path=classpath:/tpch/orders.parquet]], > selectionRoot=classpath:/tpch/orders.parquet, numFiles=1, numRowGroups=1, > usedMetadataFile=false, columns=[`o_custkey`, `o_comment`]]]) > DrillScanRel(table=[[cp, tpch/customer.parquet]], > groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath > [path=classpath:/tpch/customer.parquet]], > selectionRoot=classpath:/tpch/customer.parquet, numFiles=1, numRowGroups=1, > usedMetadataFile=false, columns=[`c_custkey`]]]) > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6896) Extraneous columns being projected past a join
[ https://issues.apache.org/jira/browse/DRILL-6896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aman Sinha updated DRILL-6896: -- Description: [~rhou] noted that TPCH13 on Drill 1.15 was running slower than Drill 1.14. Analysis revealed that an extra column was being projected in 1.15 and the slowdown was because the extra column was being unnecessarily pushed across an exchange. Here is a simplified query written by [~amansinha100] that exhibits the same problem : In first plan, o_custkey and o_comment are both extraneous projections. In the second plan (on 1.14.0), also, there is an extraneous projection: o_custkey but not o_comment. On 1.15.0: - {noformat} explain plan without implementation for select c.c_custkey from cp.`tpch/customer.parquet` c left outer join cp.`tpch/orders.parquet` o on c.c_custkey = o.o_custkey and o.o_comment not like '%special%requests%' ; DrillScreenRel DrillProjectRel(c_custkey=[$0]) DrillProjectRel(c_custkey=[$2], o_custkey=[$0], o_comment=[$1]) DrillJoinRel(condition=[=($2, $0)], joinType=[right]) DrillFilterRel(condition=[NOT(LIKE($1, '%special%requests%'))]) DrillScanRel(table=[[cp, tpch/orders.parquet]], groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=classpath:/tpch/orders.parquet]], selectionRoot=classpath:/tpch/orders.parquet, numFiles=1, numRowGroups=1, usedMetadataFile=false, columns=[`o_custkey`, `o_comment`]]]) DrillScanRel(table=[[cp, tpch/customer.parquet]], groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=classpath:/tpch/customer.parquet]], selectionRoot=classpath:/tpch/customer.parquet, numFiles=1, numRowGroups=1, usedMetadataFile=false, columns=[`c_custkey`]]]) {noformat} On 1.14.0: - {noformat} DrillScreenRel DrillProjectRel(c_custkey=[$0]) DrillProjectRel(c_custkey=[$1], o_custkey=[$0]) DrillJoinRel(condition=[=($1, $0)], joinType=[right]) DrillProjectRel(o_custkey=[$0]) DrillFilterRel(condition=[NOT(LIKE($1, '%special%requests%'))]) DrillScanRel(table=[[cp, tpch/orders.parquet]], groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=classpath:/tpch/orders.parquet]], selectionRoot=classpath:/tpch/orders.parquet, numFiles=1, numRowGroups=1, usedMetadataFile=false, columns=[`o_custkey`, `o_comment`]]]) DrillScanRel(table=[[cp, tpch/customer.parquet]], groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=classpath:/tpch/customer.parquet]], selectionRoot=classpath:/tpch/customer.parquet, numFiles=1, numRowGroups=1, usedMetadataFile=false, columns=[`c_custkey`]]]) {noformat} was: [~rhou] noted that TPCH13 on Drill 1.15 was running slower than Drill 1.14. Analysis revealed that an extra column was being projected in 1.15 and the slowdown was because the extra column was being unnecessarily pushed across an exchange. Here is a simplified query written by [~amansinha100] that exhibits the same problem : In first plan, o_custkey and o_comment are both extraneous projections. In the second plan (on 1.14.0), also, there is an extraneous projection: o_custkey but not o_comment. On 1.15.0: - explain plan without implementation for select c.c_custkey from cp.`tpch/customer.parquet` c left outer join cp.`tpch/orders.parquet` o on c.c_custkey = o.o_custkey and o.o_comment not like '%special%requests%' ; DrillScreenRel DrillProjectRel(c_custkey=[$0]) DrillProjectRel(c_custkey=[$2], o_custkey=[$0], o_comment=[$1]) DrillJoinRel(condition=[=($2, $0)], joinType=[right]) DrillFilterRel(condition=[NOT(LIKE($1, '%special%requests%'))]) DrillScanRel(table=[[cp, tpch/orders.parquet]], groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=classpath:/tpch/orders.parquet]], selectionRoot=classpath:/tpch/orders.parquet, numFiles=1, numRowGroups=1, usedMetadataFile=false, columns=[`o_custkey`, `o_comment`]]]) DrillScanRel(table=[[cp, tpch/customer.parquet]], groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=classpath:/tpch/customer.parquet]], selectionRoot=classpath:/tpch/customer.parquet, numFiles=1, numRowGroups=1, usedMetadataFile=false, columns=[`c_custkey`]]]) On 1.14.0: - DrillScreenRel DrillProjectRel(c_custkey=[$0]) DrillProjectRel(c_custkey=[$1], o_custkey=[$0]) DrillJoinRel(condition=[=($1, $0)], joinType=[right]) DrillProjectRel(o_custkey=[$0]) DrillFilterRel(condition=[NOT(LIKE($1, '%special%requests%'))]) DrillScanRel(table=[[cp, tpch/orders.parquet]], groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=classpath:/tpch/orders.parquet]], selectionRoot=classpath:/tpch/orders.parquet, numFiles=1, numRowGroups=1, usedMetadataFile=false, columns=[`o_custkey`, `o_comment`]]]) DrillScanRel(table=[[cp, tpch/customer.parquet]], groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath
[jira] [Updated] (DRILL-6896) Extraneous columns being projected past a join
[ https://issues.apache.org/jira/browse/DRILL-6896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aman Sinha updated DRILL-6896: -- Summary: Extraneous columns being projected past a join (was: Extraneous columns being projected in Drill 1.15) > Extraneous columns being projected past a join > -- > > Key: DRILL-6896 > URL: https://issues.apache.org/jira/browse/DRILL-6896 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.15.0 >Reporter: Karthikeyan Manivannan >Assignee: Aman Sinha >Priority: Major > > [~rhou] noted that TPCH13 on Drill 1.15 was running slower than Drill 1.14. > Analysis revealed that an extra column was being projected in 1.15 and the > slowdown was because the extra column was being unnecessarily pushed across > an exchange. > Here is a simplified query written by [~amansinha100] that exhibits the same > problem : > In first plan, o_custkey and o_comment are both extraneous projections. > In the second plan (on 1.14.0), also, there is an extraneous projection: > o_custkey but not o_comment. > On 1.15.0: > - > explain plan without implementation for > select > c.c_custkey > from > cp.`tpch/customer.parquet` c > left outer join cp.`tpch/orders.parquet` o > on c.c_custkey = o.o_custkey > and o.o_comment not like '%special%requests%' > ; > DrillScreenRel > DrillProjectRel(c_custkey=[$0]) > DrillProjectRel(c_custkey=[$2], o_custkey=[$0], o_comment=[$1]) > DrillJoinRel(condition=[=($2, $0)], joinType=[right]) > DrillFilterRel(condition=[NOT(LIKE($1, '%special%requests%'))]) > DrillScanRel(table=[[cp, tpch/orders.parquet]], groupscan=[ParquetGroupScan > [entries=[ReadEntryWithPath [path=classpath:/tpch/orders.parquet]], > selectionRoot=classpath:/tpch/orders.parquet, numFiles=1, numRowGroups=1, > usedMetadataFile=false, columns=[`o_custkey`, `o_comment`]]]) > DrillScanRel(table=[[cp, tpch/customer.parquet]], groupscan=[ParquetGroupScan > [entries=[ReadEntryWithPath [path=classpath:/tpch/customer.parquet]], > selectionRoot=classpath:/tpch/customer.parquet, numFiles=1, numRowGroups=1, > usedMetadataFile=false, columns=[`c_custkey`]]]) > On 1.14.0: > - > DrillScreenRel > DrillProjectRel(c_custkey=[$0]) > DrillProjectRel(c_custkey=[$1], o_custkey=[$0]) > DrillJoinRel(condition=[=($1, $0)], joinType=[right]) > DrillProjectRel(o_custkey=[$0]) > DrillFilterRel(condition=[NOT(LIKE($1, '%special%requests%'))]) > DrillScanRel(table=[[cp, tpch/orders.parquet]], groupscan=[ParquetGroupScan > [entries=[ReadEntryWithPath [path=classpath:/tpch/orders.parquet]], > selectionRoot=classpath:/tpch/orders.parquet, numFiles=1, numRowGroups=1, > usedMetadataFile=false, columns=[`o_custkey`, `o_comment`]]]) > DrillScanRel(table=[[cp, tpch/customer.parquet]], groupscan=[ParquetGroupScan > [entries=[ReadEntryWithPath [path=classpath:/tpch/customer.parquet]], > selectionRoot=classpath:/tpch/customer.parquet, numFiles=1, numRowGroups=1, > usedMetadataFile=false, columns=[`c_custkey`]]]) -- This message was sent by Atlassian JIRA (v7.6.3#76005)