[jira] [Commented] (FLINK-6326) add ProjectMergeRule at logical optimization stage
[ https://issues.apache.org/jira/browse/FLINK-6326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15974484#comment-15974484 ] ASF GitHub Bot commented on FLINK-6326: --- GitHub user godfreyhe opened a pull request: https://github.com/apache/flink/pull/3739 [FLINK-6326] [table] add ProjectMergeRule at logical optimization stage add ProjectMergeRule to merge projections You can merge this pull request into a Git repository by running: $ git pull https://github.com/godfreyhe/flink FLINK-6326 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3739.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3739 commit 1c14d02a4776becbc4147b576d5ee20dd8e15d24 Author: godfreyhe Date: 2017-04-19T11:46:18Z [FLINK-6326] [table] add ProjectMergeRule at logical optimization stage > add ProjectMergeRule at logical optimization stage > -- > > Key: FLINK-6326 > URL: https://issues.apache.org/jira/browse/FLINK-6326 > Project: Flink > Issue Type: New Feature > Components: Table API & SQL >Reporter: godfrey he >Assignee: godfrey he > > add ProjectMergeRule to merge projections. Some SQLs can not push projection > into scan without this rule. > e.g. > table1: id: int, name: string > table2: id: string, score: double, first: string, last: string > {code} > SELECT a.id, b.score FROM > (SELECT id FROM table1 WHERE id > 10) a > LEFT OUTER JOIN > (SELECT * FROM table2) b > ON CAST(a.id AS VARCHAR) = b.id > {code} > {code} > == Optimized Logical Plan without ProjectMergeRule == > DataSetCalc(select=[id, score]) > DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score, first, last], > joinType=[LeftOuterJoin]) > DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)]) > BatchTableSourceScan(table=[[table1]], fields=[id]) > BatchTableSourceScan(table=[[table2]], fields=[id, score, first, last]) > {code} > {code} > == Optimized Logical Plan with ProjectMergeRule == > DataSetCalc(select=[id, score]) > DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score], > joinType=[LeftOuterJoin]) > DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)]) > BatchTableSourceScan(table=[[table1]], fields=[id]) > BatchTableSourceScan(table=[[table2]], fields=[id, score]) > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-6326) add ProjectMergeRule at logical optimization stage
[ https://issues.apache.org/jira/browse/FLINK-6326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15974585#comment-15974585 ] ASF GitHub Bot commented on FLINK-6326: --- Github user fhueske commented on a diff in the pull request: https://github.com/apache/flink/pull/3739#discussion_r112192763 --- Diff: flink-libraries/flink-table/src/test/scala/resources/testProjectMerge.out --- @@ -0,0 +1,156 @@ +== Abstract Syntax Tree == --- End diff -- This file can be removed, IMO. > add ProjectMergeRule at logical optimization stage > -- > > Key: FLINK-6326 > URL: https://issues.apache.org/jira/browse/FLINK-6326 > Project: Flink > Issue Type: New Feature > Components: Table API & SQL >Reporter: godfrey he >Assignee: godfrey he > > add ProjectMergeRule to merge projections. Some SQLs can not push projection > into scan without this rule. > e.g. > table1: id: int, name: string > table2: id: string, score: double, first: string, last: string > {code} > SELECT a.id, b.score FROM > (SELECT id FROM table1 WHERE id > 10) a > LEFT OUTER JOIN > (SELECT * FROM table2) b > ON CAST(a.id AS VARCHAR) = b.id > {code} > {code} > == Optimized Logical Plan without ProjectMergeRule == > DataSetCalc(select=[id, score]) > DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score, first, last], > joinType=[LeftOuterJoin]) > DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)]) > BatchTableSourceScan(table=[[table1]], fields=[id]) > BatchTableSourceScan(table=[[table2]], fields=[id, score, first, last]) > {code} > {code} > == Optimized Logical Plan with ProjectMergeRule == > DataSetCalc(select=[id, score]) > DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score], > joinType=[LeftOuterJoin]) > DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)]) > BatchTableSourceScan(table=[[table1]], fields=[id]) > BatchTableSourceScan(table=[[table2]], fields=[id, score]) > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-6326) add ProjectMergeRule at logical optimization stage
[ https://issues.apache.org/jira/browse/FLINK-6326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15974584#comment-15974584 ] ASF GitHub Bot commented on FLINK-6326: --- Github user fhueske commented on a diff in the pull request: https://github.com/apache/flink/pull/3739#discussion_r112192706 --- Diff: flink-libraries/flink-table/src/test/scala/org/apache/flink/table/api/scala/batch/ExplainTest.scala --- @@ -119,4 +120,25 @@ class ExplainTest "../../src/test/scala/resources/testUnion1.out").mkString.replaceAll("\\r\\n", "\n") assertEquals(result, source) } + + @Test + def testProjectMerge(): Unit = { --- End diff -- I don't think we need to add a test to `ExplainTest.scala`. The other plan tests are sufficient to validate that the desired plans are generated. > add ProjectMergeRule at logical optimization stage > -- > > Key: FLINK-6326 > URL: https://issues.apache.org/jira/browse/FLINK-6326 > Project: Flink > Issue Type: New Feature > Components: Table API & SQL >Reporter: godfrey he >Assignee: godfrey he > > add ProjectMergeRule to merge projections. Some SQLs can not push projection > into scan without this rule. > e.g. > table1: id: int, name: string > table2: id: string, score: double, first: string, last: string > {code} > SELECT a.id, b.score FROM > (SELECT id FROM table1 WHERE id > 10) a > LEFT OUTER JOIN > (SELECT * FROM table2) b > ON CAST(a.id AS VARCHAR) = b.id > {code} > {code} > == Optimized Logical Plan without ProjectMergeRule == > DataSetCalc(select=[id, score]) > DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score, first, last], > joinType=[LeftOuterJoin]) > DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)]) > BatchTableSourceScan(table=[[table1]], fields=[id]) > BatchTableSourceScan(table=[[table2]], fields=[id, score, first, last]) > {code} > {code} > == Optimized Logical Plan with ProjectMergeRule == > DataSetCalc(select=[id, score]) > DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score], > joinType=[LeftOuterJoin]) > DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)]) > BatchTableSourceScan(table=[[table1]], fields=[id]) > BatchTableSourceScan(table=[[table2]], fields=[id, score]) > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-6326) add ProjectMergeRule at logical optimization stage
[ https://issues.apache.org/jira/browse/FLINK-6326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15979262#comment-15979262 ] ASF GitHub Bot commented on FLINK-6326: --- Github user fhueske commented on the issue: https://github.com/apache/flink/pull/3739 Merging > add ProjectMergeRule at logical optimization stage > -- > > Key: FLINK-6326 > URL: https://issues.apache.org/jira/browse/FLINK-6326 > Project: Flink > Issue Type: New Feature > Components: Table API & SQL >Reporter: godfrey he >Assignee: godfrey he > > add ProjectMergeRule to merge projections. Some SQLs can not push projection > into scan without this rule. > e.g. > table1: id: int, name: string > table2: id: string, score: double, first: string, last: string > {code} > SELECT a.id, b.score FROM > (SELECT id FROM table1 WHERE id > 10) a > LEFT OUTER JOIN > (SELECT * FROM table2) b > ON CAST(a.id AS VARCHAR) = b.id > {code} > {code} > == Optimized Logical Plan without ProjectMergeRule == > DataSetCalc(select=[id, score]) > DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score, first, last], > joinType=[LeftOuterJoin]) > DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)]) > BatchTableSourceScan(table=[[table1]], fields=[id]) > BatchTableSourceScan(table=[[table2]], fields=[id, score, first, last]) > {code} > {code} > == Optimized Logical Plan with ProjectMergeRule == > DataSetCalc(select=[id, score]) > DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score], > joinType=[LeftOuterJoin]) > DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)]) > BatchTableSourceScan(table=[[table1]], fields=[id]) > BatchTableSourceScan(table=[[table2]], fields=[id, score]) > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-6326) add ProjectMergeRule at logical optimization stage
[ https://issues.apache.org/jira/browse/FLINK-6326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15979373#comment-15979373 ] ASF GitHub Bot commented on FLINK-6326: --- Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/3739 > add ProjectMergeRule at logical optimization stage > -- > > Key: FLINK-6326 > URL: https://issues.apache.org/jira/browse/FLINK-6326 > Project: Flink > Issue Type: New Feature > Components: Table API & SQL >Reporter: godfrey he >Assignee: godfrey he > > add ProjectMergeRule to merge projections. Some SQLs can not push projection > into scan without this rule. > e.g. > table1: id: int, name: string > table2: id: string, score: double, first: string, last: string > {code} > SELECT a.id, b.score FROM > (SELECT id FROM table1 WHERE id > 10) a > LEFT OUTER JOIN > (SELECT * FROM table2) b > ON CAST(a.id AS VARCHAR) = b.id > {code} > {code} > == Optimized Logical Plan without ProjectMergeRule == > DataSetCalc(select=[id, score]) > DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score, first, last], > joinType=[LeftOuterJoin]) > DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)]) > BatchTableSourceScan(table=[[table1]], fields=[id]) > BatchTableSourceScan(table=[[table2]], fields=[id, score, first, last]) > {code} > {code} > == Optimized Logical Plan with ProjectMergeRule == > DataSetCalc(select=[id, score]) > DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score], > joinType=[LeftOuterJoin]) > DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)]) > BatchTableSourceScan(table=[[table1]], fields=[id]) > BatchTableSourceScan(table=[[table2]], fields=[id, score]) > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)