[jira] [Commented] (DRILL-3412) Projections are not getting push down below Window operator
[
https://issues.apache.org/jira/browse/DRILL-3412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14744588#comment-14744588
]
Victoria Markman commented on DRILL-3412:
-
Verified fixed in 1.2.0
#Fri Sep 11 05:38:24 UTC 2015
git.commit.id.abbrev=b525692
Added bunch of test cases under:
Functional/Passing/window_functions/optimization
> Projections are not getting push down below Window operator
> ---
>
> Key: DRILL-3412
> URL: https://issues.apache.org/jira/browse/DRILL-3412
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Reporter: Aman Sinha
>Assignee: Sean Hsuan-Yi Chu
>Priority: Blocker
> Labels: window_function
> Fix For: 1.2.0
>
>
> The plan below shows that the 'star' column is being produced by the Scan and
> subsequent Project. This indicates projection pushdown is not working as
> desired when window function is present. The query produces correct results.
> {code}
> explain plan for select min(n_nationkey) over (partition by n_regionkey) from
> cp.`tpch/nation.parquet` ;
> 00-00Screen
> 00-01 Project(EXPR$0=[$0])
> 00-02Project(w0$o0=[$3])
> 00-03 Window(window#0=[window(partition {2} order by [] range
> between UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [MIN($1)])])
> 00-04SelectionVectorRemover
> 00-05 Sort(sort0=[$2], dir0=[ASC])
> 00-06Project(T1¦¦*=[$0], n_nationkey=[$1], n_regionkey=[$2])
> 00-07 Scan(groupscan=[ParquetGroupScan
> [entries=[ReadEntryWithPath [path=classpath:/tpch/nation.parquet]],
> selectionRoot=/tpch/nation.parquet, numFiles=1, columns=[`*`]]])
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3412) Projections are not getting push down below Window operator
[
https://issues.apache.org/jira/browse/DRILL-3412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14707598#comment-14707598
]
Sean Hsuan-Yi Chu commented on DRILL-3412:
--
https://reviews.apache.org/r/37697
Projections are not getting push down below Window operator
---
Key: DRILL-3412
URL: https://issues.apache.org/jira/browse/DRILL-3412
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Reporter: Aman Sinha
Assignee: Sean Hsuan-Yi Chu
Priority: Blocker
Labels: window_function
Fix For: 1.2.0
The plan below shows that the 'star' column is being produced by the Scan and
subsequent Project. This indicates projection pushdown is not working as
desired when window function is present. The query produces correct results.
{code}
explain plan for select min(n_nationkey) over (partition by n_regionkey) from
cp.`tpch/nation.parquet` ;
00-00Screen
00-01 Project(EXPR$0=[$0])
00-02Project(w0$o0=[$3])
00-03 Window(window#0=[window(partition {2} order by [] range
between UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [MIN($1)])])
00-04SelectionVectorRemover
00-05 Sort(sort0=[$2], dir0=[ASC])
00-06Project(T1¦¦*=[$0], n_nationkey=[$1], n_regionkey=[$2])
00-07 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=classpath:/tpch/nation.parquet]],
selectionRoot=/tpch/nation.parquet, numFiles=1, columns=[`*`]]])
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3412) Projections are not getting push down below Window operator
[
https://issues.apache.org/jira/browse/DRILL-3412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703875#comment-14703875
]
Deneche A. Hakim commented on DRILL-3412:
-
A quick workaround until this issue is fixed is to add a constant inside the
window function. The following query will work properly:
{noformat}
explain plan for select min(n_nationkey + 0) over (partition by n_regionkey)
from cp.`tpch/nation.parquet` ;
00-00Screen
00-01 Project(EXPR$0=[$0])
00-02Project($0=[$2])
00-03 Window(window#0=[window(partition {0} order by [] range between
UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [MIN($1)])])
00-04SelectionVectorRemover
00-05 Sort(sort0=[$0], dir0=[ASC])
00-06Project(n_regionkey=[$0], $1=[+($1, 0)])
00-07 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=classpath:/tpch/nation.parquet]],
selectionRoot=classpath:/tpch/nation.parquet, numFiles=1,
columns=[`n_regionkey`, `n_nationkey`]]])
{noformat}
Projections are not getting push down below Window operator
---
Key: DRILL-3412
URL: https://issues.apache.org/jira/browse/DRILL-3412
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Reporter: Aman Sinha
Assignee: Sean Hsuan-Yi Chu
Priority: Blocker
Labels: window_function
Fix For: 1.2.0
The plan below shows that the 'star' column is being produced by the Scan and
subsequent Project. This indicates projection pushdown is not working as
desired when window function is present. The query produces correct results.
{code}
explain plan for select min(n_nationkey) over (partition by n_regionkey) from
cp.`tpch/nation.parquet` ;
00-00Screen
00-01 Project(EXPR$0=[$0])
00-02Project(w0$o0=[$3])
00-03 Window(window#0=[window(partition {2} order by [] range
between UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [MIN($1)])])
00-04SelectionVectorRemover
00-05 Sort(sort0=[$2], dir0=[ASC])
00-06Project(T1¦¦*=[$0], n_nationkey=[$1], n_regionkey=[$2])
00-07 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=classpath:/tpch/nation.parquet]],
selectionRoot=/tpch/nation.parquet, numFiles=1, columns=[`*`]]])
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3412) Projections are not getting push down below Window operator
[
https://issues.apache.org/jira/browse/DRILL-3412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696256#comment-14696256
]
Sean Hsuan-Yi Chu commented on DRILL-3412:
--
We can reproduce this issue on Calcite master:
https://issues.apache.org/jira/browse/CALCITE-844
Projections are not getting push down below Window operator
---
Key: DRILL-3412
URL: https://issues.apache.org/jira/browse/DRILL-3412
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Reporter: Aman Sinha
Assignee: Sean Hsuan-Yi Chu
Priority: Blocker
Labels: window_function
Fix For: 1.2.0
The plan below shows that the 'star' column is being produced by the Scan and
subsequent Project. This indicates projection pushdown is not working as
desired when window function is present. The query produces correct results.
{code}
explain plan for select min(n_nationkey) over (partition by n_regionkey) from
cp.`tpch/nation.parquet` ;
00-00Screen
00-01 Project(EXPR$0=[$0])
00-02Project(w0$o0=[$3])
00-03 Window(window#0=[window(partition {2} order by [] range
between UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [MIN($1)])])
00-04SelectionVectorRemover
00-05 Sort(sort0=[$2], dir0=[ASC])
00-06Project(T1¦¦*=[$0], n_nationkey=[$1], n_regionkey=[$2])
00-07 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=classpath:/tpch/nation.parquet]],
selectionRoot=/tpch/nation.parquet, numFiles=1, columns=[`*`]]])
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3412) Projections are not getting push down below Window operator
[
https://issues.apache.org/jira/browse/DRILL-3412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14642849#comment-14642849
]
Deneche A. Hakim commented on DRILL-3412:
-
Interestingly, adding a constant in one of the window functions will push the
projections below the Window operator:
Query without constant:
{noformat}
0: jdbc:drill:zk=local explain plan for SELECT RANK() OVER (PARTITION BY
ss.ss_store_sk ORDER BY ss.ss_store_sk) FROM store_sales ss LIMIT 20;
00-00Screen
00-01 Project(EXPR$0=[$0])
00-02SelectionVectorRemover
00-03 Limit(fetch=[20])
00-04UnionExchange
01-01 Project(w0$o0=[$2])
01-02Window(window#0=[window(partition {1} order by [1] range
between UNBOUNDED PRECEDING and CURRENT ROW aggs [RANK()])])
01-03 SelectionVectorRemover
01-04Sort(sort0=[$1], sort1=[$1], dir0=[ASC], dir1=[ASC])
01-05 Project(T3¦¦*=[$0], ss_store_sk=[$1])
01-06HashToRandomExchange(dist0=[[$1]])
02-01 UnorderedMuxExchange
03-01Project(T3¦¦*=[$0], ss_store_sk=[$1],
E_X_P_R_H_A_S_H_F_I_E_L_D=[castInt(hash64AsDouble($1))])
03-02 Project(T3¦¦*=[$0], ss_store_sk=[$1])
03-03Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath
[path=file:/Users/hakim/MapR/data/tpcds100/parquet/store_sales]],
selectionRoot=file:/Users/hakim/MapR/data/tpcds100/parquet/store_sales,
numFiles=1, columns=[`*`]]])
{noformat}
Query with constant in ORDER BY clause (the query is still the same because we
were ordering on the partition clause):
{noformat}
0: jdbc:drill:zk=local explain plan for SELECT RANK() OVER (PARTITION BY
ss.ss_store_sk ORDER BY 1) FROM store_sales ss LIMIT 20;
00-00Screen
00-01 Project(EXPR$0=[$0])
00-02SelectionVectorRemover
00-03 Limit(fetch=[20])
00-04UnionExchange
01-01 Project($0=[$1])
01-02Window(window#0=[window(partition {0} order by [] range
between UNBOUNDED PRECEDING and CURRENT ROW aggs [RANK()])])
01-03 SelectionVectorRemover
01-04Sort(sort0=[$0], dir0=[ASC])
01-05 Project(ss_store_sk=[$0])
01-06HashToRandomExchange(dist0=[[$0]])
02-01 UnorderedMuxExchange
03-01Project(ss_store_sk=[$0],
E_X_P_R_H_A_S_H_F_I_E_L_D=[castInt(hash64AsDouble($0))])
03-02 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath
[path=file:/Users/hakim/MapR/data/tpcds100/parquet/store_sales]],
selectionRoot=file:/Users/hakim/MapR/data/tpcds100/parquet/store_sales,
numFiles=1, columns=[`ss_store_sk`]]])
{noformat}
Query with a constant in a different window function COUNT(1):
{noformat}
0: jdbc:drill:zk=local explain plan for SELECT COUNT(1) OVER(PARTITION BY
ss.ss_store_sk ORDER BY ss.ss_store_sk), RANK() OVER (PARTITION BY
ss.ss_store_sk ORDER BY ss.ss_store_sk) FROM store_sales ss LIMIT 20;
00-00Screen
00-01 Project(EXPR$0=[$0], EXPR$1=[$1])
00-02SelectionVectorRemover
00-03 Limit(fetch=[20])
00-04UnionExchange
01-01 Project($0=[$1], $1=[$2])
01-02Window(window#0=[window(partition {0} order by [0] range
between UNBOUNDED PRECEDING and CURRENT ROW aggs [COUNT($1), RANK()])])
01-03 SelectionVectorRemover
01-04Sort(sort0=[$0], sort1=[$0], dir0=[ASC], dir1=[ASC])
01-05 Project(ss_store_sk=[$0])
01-06HashToRandomExchange(dist0=[[$0]])
02-01 UnorderedMuxExchange
03-01Project(ss_store_sk=[$0],
E_X_P_R_H_A_S_H_F_I_E_L_D=[castInt(hash64AsDouble($0))])
03-02 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath
[path=file:/Users/hakim/MapR/data/tpcds100/parquet/store_sales]],
selectionRoot=file:/Users/hakim/MapR/data/tpcds100/parquet/store_sales,
numFiles=1, columns=[`ss_store_sk`]]])
{noformat}
Projections are not getting push down below Window operator
---
Key: DRILL-3412
URL: https://issues.apache.org/jira/browse/DRILL-3412
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Reporter: Aman Sinha
Assignee: Jinfeng Ni
Priority: Blocker
Labels: window_function
Fix For: 1.2.0
The plan below shows that the 'star' column is being produced by the Scan and
subsequent Project. This indicates projection pushdown is not working as
desired when window function is present. The query produces
[jira] [Commented] (DRILL-3412) Projections are not getting push down below Window operator
[
https://issues.apache.org/jira/browse/DRILL-3412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643600#comment-14643600
]
Deneche A. Hakim commented on DRILL-3412:
-
I tried multiple commits, the commit right before 3304 didn't have this issue
but 3304 did. I may be mistaken as I tried lot's of commits building them each
time.
Projections are not getting push down below Window operator
---
Key: DRILL-3412
URL: https://issues.apache.org/jira/browse/DRILL-3412
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Reporter: Aman Sinha
Assignee: Jinfeng Ni
Priority: Blocker
Labels: window_function
Fix For: 1.2.0
The plan below shows that the 'star' column is being produced by the Scan and
subsequent Project. This indicates projection pushdown is not working as
desired when window function is present. The query produces correct results.
{code}
explain plan for select min(n_nationkey) over (partition by n_regionkey) from
cp.`tpch/nation.parquet` ;
00-00Screen
00-01 Project(EXPR$0=[$0])
00-02Project(w0$o0=[$3])
00-03 Window(window#0=[window(partition {2} order by [] range
between UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [MIN($1)])])
00-04SelectionVectorRemover
00-05 Sort(sort0=[$2], dir0=[ASC])
00-06Project(T1¦¦*=[$0], n_nationkey=[$1], n_regionkey=[$2])
00-07 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=classpath:/tpch/nation.parquet]],
selectionRoot=/tpch/nation.parquet, numFiles=1, columns=[`*`]]])
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3412) Projections are not getting push down below Window operator
[
https://issues.apache.org/jira/browse/DRILL-3412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643589#comment-14643589
]
Jinfeng Ni commented on DRILL-3412:
---
Why does DRILL-3304 have anything to with project pushdown? Seems to me
DRILL-3412 and DRILL-3304 are dealing with two different issues.
Projections are not getting push down below Window operator
---
Key: DRILL-3412
URL: https://issues.apache.org/jira/browse/DRILL-3412
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Reporter: Aman Sinha
Assignee: Jinfeng Ni
Priority: Blocker
Labels: window_function
Fix For: 1.2.0
The plan below shows that the 'star' column is being produced by the Scan and
subsequent Project. This indicates projection pushdown is not working as
desired when window function is present. The query produces correct results.
{code}
explain plan for select min(n_nationkey) over (partition by n_regionkey) from
cp.`tpch/nation.parquet` ;
00-00Screen
00-01 Project(EXPR$0=[$0])
00-02Project(w0$o0=[$3])
00-03 Window(window#0=[window(partition {2} order by [] range
between UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [MIN($1)])])
00-04SelectionVectorRemover
00-05 Sort(sort0=[$2], dir0=[ASC])
00-06Project(T1¦¦*=[$0], n_nationkey=[$1], n_regionkey=[$2])
00-07 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=classpath:/tpch/nation.parquet]],
selectionRoot=/tpch/nation.parquet, numFiles=1, columns=[`*`]]])
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3412) Projections are not getting push down below Window operator
[
https://issues.apache.org/jira/browse/DRILL-3412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643579#comment-14643579
]
Deneche A. Hakim commented on DRILL-3412:
-
After some testing it looks like this bug was introduced by DRILL-3304, at
least for queries that didn't involve a RANK window function
Projections are not getting push down below Window operator
---
Key: DRILL-3412
URL: https://issues.apache.org/jira/browse/DRILL-3412
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Reporter: Aman Sinha
Assignee: Jinfeng Ni
Priority: Blocker
Labels: window_function
Fix For: 1.2.0
The plan below shows that the 'star' column is being produced by the Scan and
subsequent Project. This indicates projection pushdown is not working as
desired when window function is present. The query produces correct results.
{code}
explain plan for select min(n_nationkey) over (partition by n_regionkey) from
cp.`tpch/nation.parquet` ;
00-00Screen
00-01 Project(EXPR$0=[$0])
00-02Project(w0$o0=[$3])
00-03 Window(window#0=[window(partition {2} order by [] range
between UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [MIN($1)])])
00-04SelectionVectorRemover
00-05 Sort(sort0=[$2], dir0=[ASC])
00-06Project(T1¦¦*=[$0], n_nationkey=[$1], n_regionkey=[$2])
00-07 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=classpath:/tpch/nation.parquet]],
selectionRoot=/tpch/nation.parquet, numFiles=1, columns=[`*`]]])
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3412) Projections are not getting push down below Window operator
[
https://issues.apache.org/jira/browse/DRILL-3412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637550#comment-14637550
]
Victoria Markman commented on DRILL-3412:
-
This bug is blocking performance testing of window functions.
Projections are not getting push down below Window operator
---
Key: DRILL-3412
URL: https://issues.apache.org/jira/browse/DRILL-3412
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Reporter: Aman Sinha
Assignee: Jinfeng Ni
Priority: Blocker
Fix For: 1.2.0
The plan below shows that the 'star' column is being produced by the Scan and
subsequent Project. This indicates projection pushdown is not working as
desired when window function is present. The query produces correct results.
{code}
explain plan for select min(n_nationkey) over (partition by n_regionkey) from
cp.`tpch/nation.parquet` ;
00-00Screen
00-01 Project(EXPR$0=[$0])
00-02Project(w0$o0=[$3])
00-03 Window(window#0=[window(partition {2} order by [] range
between UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [MIN($1)])])
00-04SelectionVectorRemover
00-05 Sort(sort0=[$2], dir0=[ASC])
00-06Project(T1¦¦*=[$0], n_nationkey=[$1], n_regionkey=[$2])
00-07 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=classpath:/tpch/nation.parquet]],
selectionRoot=/tpch/nation.parquet, numFiles=1, columns=[`*`]]])
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3412) Projections are not getting push down below Window operator
[
https://issues.apache.org/jira/browse/DRILL-3412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637580#comment-14637580
]
Victoria Markman commented on DRILL-3412:
-
Jinfeng, this bug was reported this morning as a blocking issue for testing.
We are using TPCDS schema. Should we proceed with views on top of TPCDS tables
that project fewer columns ?
Projections are not getting push down below Window operator
---
Key: DRILL-3412
URL: https://issues.apache.org/jira/browse/DRILL-3412
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Reporter: Aman Sinha
Assignee: Jinfeng Ni
Priority: Blocker
Fix For: 1.2.0
The plan below shows that the 'star' column is being produced by the Scan and
subsequent Project. This indicates projection pushdown is not working as
desired when window function is present. The query produces correct results.
{code}
explain plan for select min(n_nationkey) over (partition by n_regionkey) from
cp.`tpch/nation.parquet` ;
00-00Screen
00-01 Project(EXPR$0=[$0])
00-02Project(w0$o0=[$3])
00-03 Window(window#0=[window(partition {2} order by [] range
between UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [MIN($1)])])
00-04SelectionVectorRemover
00-05 Sort(sort0=[$2], dir0=[ASC])
00-06Project(T1¦¦*=[$0], n_nationkey=[$1], n_regionkey=[$2])
00-07 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=classpath:/tpch/nation.parquet]],
selectionRoot=/tpch/nation.parquet, numFiles=1, columns=[`*`]]])
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3412) Projections are not getting push down below Window operator
[
https://issues.apache.org/jira/browse/DRILL-3412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637569#comment-14637569
]
Jinfeng Ni commented on DRILL-3412:
---
Although there is big impact for window functions without project pushdown, I
do not think it would be a deciding factor for window function performance,
especially if the table does not have too many columns which are not referenced
in the query.
In stead, in many cases, the performance bottleneck for window function would
be in the SORT operator, not in the SCAN operator.
Projections are not getting push down below Window operator
---
Key: DRILL-3412
URL: https://issues.apache.org/jira/browse/DRILL-3412
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Reporter: Aman Sinha
Assignee: Jinfeng Ni
Priority: Blocker
Fix For: 1.2.0
The plan below shows that the 'star' column is being produced by the Scan and
subsequent Project. This indicates projection pushdown is not working as
desired when window function is present. The query produces correct results.
{code}
explain plan for select min(n_nationkey) over (partition by n_regionkey) from
cp.`tpch/nation.parquet` ;
00-00Screen
00-01 Project(EXPR$0=[$0])
00-02Project(w0$o0=[$3])
00-03 Window(window#0=[window(partition {2} order by [] range
between UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [MIN($1)])])
00-04SelectionVectorRemover
00-05 Sort(sort0=[$2], dir0=[ASC])
00-06Project(T1¦¦*=[$0], n_nationkey=[$1], n_regionkey=[$2])
00-07 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=classpath:/tpch/nation.parquet]],
selectionRoot=/tpch/nation.parquet, numFiles=1, columns=[`*`]]])
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3412) Projections are not getting push down below Window operator
[
https://issues.apache.org/jira/browse/DRILL-3412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14605877#comment-14605877
]
Abhishek Girish commented on DRILL-3412:
I'm seeing a ChannelClosedException with the same RANK() window function query.
SELECT RANK() OVER (PARTITION BY ss.ss_store_sk ORDER BY ss.ss_store_sk)
FROM store_sales ss WHERE ss.ss_store_sk is not NULL LIMIT 20;
java.lang.RuntimeException: java.sql.SQLException: SYSTEM ERROR:
ChannelClosedException
Fragment 3:17
[Error Id: b78b4931-da1d-4669-af09-ee1be3636421 on abhi7.qa.lab:31010]
at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73)
at
sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:85)
at sqlline.TableOutputFormat.print(TableOutputFormat.java:116)
at sqlline.SqlLine.print(SqlLine.java:1583)
at sqlline.Commands.execute(Commands.java:852)
at sqlline.Commands.sql(Commands.java:751)
at sqlline.SqlLine.dispatch(SqlLine.java:738)
at sqlline.SqlLine.begin(SqlLine.java:612)
at sqlline.SqlLine.start(SqlLine.java:366)
at sqlline.SqlLine.main(SqlLine.java:259)
Drill Git.CommitID: 60bc945 (Jun 26)
Projections are not getting push down below Window operator
---
Key: DRILL-3412
URL: https://issues.apache.org/jira/browse/DRILL-3412
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Reporter: Aman Sinha
Assignee: Jinfeng Ni
The plan below shows that the 'star' column is being produced by the Scan and
subsequent Project. This indicates projection pushdown is not working as
desired when window function is present. The query produces correct results.
{code}
explain plan for select min(n_nationkey) over (partition by n_regionkey) from
cp.`tpch/nation.parquet` ;
00-00Screen
00-01 Project(EXPR$0=[$0])
00-02Project(w0$o0=[$3])
00-03 Window(window#0=[window(partition {2} order by [] range
between UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [MIN($1)])])
00-04SelectionVectorRemover
00-05 Sort(sort0=[$2], dir0=[ASC])
00-06Project(T1¦¦*=[$0], n_nationkey=[$1], n_regionkey=[$2])
00-07 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=classpath:/tpch/nation.parquet]],
selectionRoot=/tpch/nation.parquet, numFiles=1, columns=[`*`]]])
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3412) Projections are not getting push down below Window operator
[
https://issues.apache.org/jira/browse/DRILL-3412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14606845#comment-14606845
]
Abhishek Girish commented on DRILL-3412:
Turns out the above issue was due to Drill running out of memory - the actual
exceptions were not propagated correctly. After bumping up Direct Memory, the
query completes. The original issue mentioned in the description is still valid
and observed on latest builds. I'll file a separate JIRA for the error message
issue.
Projections are not getting push down below Window operator
---
Key: DRILL-3412
URL: https://issues.apache.org/jira/browse/DRILL-3412
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Reporter: Aman Sinha
Assignee: Jinfeng Ni
The plan below shows that the 'star' column is being produced by the Scan and
subsequent Project. This indicates projection pushdown is not working as
desired when window function is present. The query produces correct results.
{code}
explain plan for select min(n_nationkey) over (partition by n_regionkey) from
cp.`tpch/nation.parquet` ;
00-00Screen
00-01 Project(EXPR$0=[$0])
00-02Project(w0$o0=[$3])
00-03 Window(window#0=[window(partition {2} order by [] range
between UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [MIN($1)])])
00-04SelectionVectorRemover
00-05 Sort(sort0=[$2], dir0=[ASC])
00-06Project(T1¦¦*=[$0], n_nationkey=[$1], n_regionkey=[$2])
00-07 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=classpath:/tpch/nation.parquet]],
selectionRoot=/tpch/nation.parquet, numFiles=1, columns=[`*`]]])
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
