[jira] [Commented] (FLINK-6326) add ProjectMergeRule at logical optimization stage
[
https://issues.apache.org/jira/browse/FLINK-6326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15974484#comment-15974484
]
ASF GitHub Bot commented on FLINK-6326:
---
GitHub user godfreyhe opened a pull request:
https://github.com/apache/flink/pull/3739
[FLINK-6326] [table] add ProjectMergeRule at logical optimization stage
add ProjectMergeRule to merge projections
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/godfreyhe/flink FLINK-6326
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3739.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3739
commit 1c14d02a4776becbc4147b576d5ee20dd8e15d24
Author: godfreyhe
Date: 2017-04-19T11:46:18Z
[FLINK-6326] [table] add ProjectMergeRule at logical optimization stage
> add ProjectMergeRule at logical optimization stage
> --
>
> Key: FLINK-6326
> URL: https://issues.apache.org/jira/browse/FLINK-6326
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: godfrey he
>Assignee: godfrey he
>
> add ProjectMergeRule to merge projections. Some SQLs can not push projection
> into scan without this rule.
> e.g.
> table1: id: int, name: string
> table2: id: string, score: double, first: string, last: string
> {code}
> SELECT a.id, b.score FROM
> (SELECT id FROM table1 WHERE id > 10) a
> LEFT OUTER JOIN
> (SELECT * FROM table2) b
> ON CAST(a.id AS VARCHAR) = b.id
> {code}
> {code}
> == Optimized Logical Plan without ProjectMergeRule ==
> DataSetCalc(select=[id, score])
> DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score, first, last],
> joinType=[LeftOuterJoin])
> DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)])
> BatchTableSourceScan(table=[[table1]], fields=[id])
> BatchTableSourceScan(table=[[table2]], fields=[id, score, first, last])
> {code}
> {code}
> == Optimized Logical Plan with ProjectMergeRule ==
> DataSetCalc(select=[id, score])
> DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score],
> joinType=[LeftOuterJoin])
> DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)])
> BatchTableSourceScan(table=[[table1]], fields=[id])
> BatchTableSourceScan(table=[[table2]], fields=[id, score])
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (FLINK-6326) add ProjectMergeRule at logical optimization stage
[
https://issues.apache.org/jira/browse/FLINK-6326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15974585#comment-15974585
]
ASF GitHub Bot commented on FLINK-6326:
---
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/3739#discussion_r112192763
--- Diff:
flink-libraries/flink-table/src/test/scala/resources/testProjectMerge.out ---
@@ -0,0 +1,156 @@
+== Abstract Syntax Tree ==
--- End diff --
This file can be removed, IMO.
> add ProjectMergeRule at logical optimization stage
> --
>
> Key: FLINK-6326
> URL: https://issues.apache.org/jira/browse/FLINK-6326
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: godfrey he
>Assignee: godfrey he
>
> add ProjectMergeRule to merge projections. Some SQLs can not push projection
> into scan without this rule.
> e.g.
> table1: id: int, name: string
> table2: id: string, score: double, first: string, last: string
> {code}
> SELECT a.id, b.score FROM
> (SELECT id FROM table1 WHERE id > 10) a
> LEFT OUTER JOIN
> (SELECT * FROM table2) b
> ON CAST(a.id AS VARCHAR) = b.id
> {code}
> {code}
> == Optimized Logical Plan without ProjectMergeRule ==
> DataSetCalc(select=[id, score])
> DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score, first, last],
> joinType=[LeftOuterJoin])
> DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)])
> BatchTableSourceScan(table=[[table1]], fields=[id])
> BatchTableSourceScan(table=[[table2]], fields=[id, score, first, last])
> {code}
> {code}
> == Optimized Logical Plan with ProjectMergeRule ==
> DataSetCalc(select=[id, score])
> DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score],
> joinType=[LeftOuterJoin])
> DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)])
> BatchTableSourceScan(table=[[table1]], fields=[id])
> BatchTableSourceScan(table=[[table2]], fields=[id, score])
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (FLINK-6326) add ProjectMergeRule at logical optimization stage
[
https://issues.apache.org/jira/browse/FLINK-6326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15974584#comment-15974584
]
ASF GitHub Bot commented on FLINK-6326:
---
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/3739#discussion_r112192706
--- Diff:
flink-libraries/flink-table/src/test/scala/org/apache/flink/table/api/scala/batch/ExplainTest.scala
---
@@ -119,4 +120,25 @@ class ExplainTest
"../../src/test/scala/resources/testUnion1.out").mkString.replaceAll("\\r\\n",
"\n")
assertEquals(result, source)
}
+
+ @Test
+ def testProjectMerge(): Unit = {
--- End diff --
I don't think we need to add a test to `ExplainTest.scala`. The other plan
tests are sufficient to validate that the desired plans are generated.
> add ProjectMergeRule at logical optimization stage
> --
>
> Key: FLINK-6326
> URL: https://issues.apache.org/jira/browse/FLINK-6326
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: godfrey he
>Assignee: godfrey he
>
> add ProjectMergeRule to merge projections. Some SQLs can not push projection
> into scan without this rule.
> e.g.
> table1: id: int, name: string
> table2: id: string, score: double, first: string, last: string
> {code}
> SELECT a.id, b.score FROM
> (SELECT id FROM table1 WHERE id > 10) a
> LEFT OUTER JOIN
> (SELECT * FROM table2) b
> ON CAST(a.id AS VARCHAR) = b.id
> {code}
> {code}
> == Optimized Logical Plan without ProjectMergeRule ==
> DataSetCalc(select=[id, score])
> DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score, first, last],
> joinType=[LeftOuterJoin])
> DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)])
> BatchTableSourceScan(table=[[table1]], fields=[id])
> BatchTableSourceScan(table=[[table2]], fields=[id, score, first, last])
> {code}
> {code}
> == Optimized Logical Plan with ProjectMergeRule ==
> DataSetCalc(select=[id, score])
> DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score],
> joinType=[LeftOuterJoin])
> DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)])
> BatchTableSourceScan(table=[[table1]], fields=[id])
> BatchTableSourceScan(table=[[table2]], fields=[id, score])
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (FLINK-6326) add ProjectMergeRule at logical optimization stage
[
https://issues.apache.org/jira/browse/FLINK-6326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15979262#comment-15979262
]
ASF GitHub Bot commented on FLINK-6326:
---
Github user fhueske commented on the issue:
https://github.com/apache/flink/pull/3739
Merging
> add ProjectMergeRule at logical optimization stage
> --
>
> Key: FLINK-6326
> URL: https://issues.apache.org/jira/browse/FLINK-6326
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: godfrey he
>Assignee: godfrey he
>
> add ProjectMergeRule to merge projections. Some SQLs can not push projection
> into scan without this rule.
> e.g.
> table1: id: int, name: string
> table2: id: string, score: double, first: string, last: string
> {code}
> SELECT a.id, b.score FROM
> (SELECT id FROM table1 WHERE id > 10) a
> LEFT OUTER JOIN
> (SELECT * FROM table2) b
> ON CAST(a.id AS VARCHAR) = b.id
> {code}
> {code}
> == Optimized Logical Plan without ProjectMergeRule ==
> DataSetCalc(select=[id, score])
> DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score, first, last],
> joinType=[LeftOuterJoin])
> DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)])
> BatchTableSourceScan(table=[[table1]], fields=[id])
> BatchTableSourceScan(table=[[table2]], fields=[id, score, first, last])
> {code}
> {code}
> == Optimized Logical Plan with ProjectMergeRule ==
> DataSetCalc(select=[id, score])
> DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score],
> joinType=[LeftOuterJoin])
> DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)])
> BatchTableSourceScan(table=[[table1]], fields=[id])
> BatchTableSourceScan(table=[[table2]], fields=[id, score])
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (FLINK-6326) add ProjectMergeRule at logical optimization stage
[
https://issues.apache.org/jira/browse/FLINK-6326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15979373#comment-15979373
]
ASF GitHub Bot commented on FLINK-6326:
---
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/3739
> add ProjectMergeRule at logical optimization stage
> --
>
> Key: FLINK-6326
> URL: https://issues.apache.org/jira/browse/FLINK-6326
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: godfrey he
>Assignee: godfrey he
>
> add ProjectMergeRule to merge projections. Some SQLs can not push projection
> into scan without this rule.
> e.g.
> table1: id: int, name: string
> table2: id: string, score: double, first: string, last: string
> {code}
> SELECT a.id, b.score FROM
> (SELECT id FROM table1 WHERE id > 10) a
> LEFT OUTER JOIN
> (SELECT * FROM table2) b
> ON CAST(a.id AS VARCHAR) = b.id
> {code}
> {code}
> == Optimized Logical Plan without ProjectMergeRule ==
> DataSetCalc(select=[id, score])
> DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score, first, last],
> joinType=[LeftOuterJoin])
> DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)])
> BatchTableSourceScan(table=[[table1]], fields=[id])
> BatchTableSourceScan(table=[[table2]], fields=[id, score, first, last])
> {code}
> {code}
> == Optimized Logical Plan with ProjectMergeRule ==
> DataSetCalc(select=[id, score])
> DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score],
> joinType=[LeftOuterJoin])
> DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)])
> BatchTableSourceScan(table=[[table1]], fields=[id])
> BatchTableSourceScan(table=[[table2]], fields=[id, score])
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
