[
https://issues.apache.org/jira/browse/FLINK-8897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16675124#comment-16675124
]
ASF GitHub Bot commented on FLINK-8897:
---------------------------------------
fhueske commented on issue #6987: [FLINK-8897] [table] Fix rowtime
materialization issues for filters and joins
URL: https://github.com/apache/flink/pull/6987#issuecomment-435869177
Hi @hequn8128, the problem with throwing exceptions in the rules is that
they result in canceling the optimizer completely, i.e., as soon as a rule is
applied on a plan that does not meet the condition, the query fails even if the
query could be transformed into a valid execution plan. I know that Calcite's
"Cannot optimize" exceptions are hard to digest, but they are only thrown when
the optimizer cannot generate a valid plan.
Aborting the optimization is especially tricky for specialized operators
like window join that require very specific conditions. If these conditions are
not met, the join can often be executed with a regular (fully materializing)
join. I don't think we should speculate whether this is something that the user
intended to do or not.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Rowtime materialization causes "mismatched type" AssertionError
> ---------------------------------------------------------------
>
> Key: FLINK-8897
> URL: https://issues.apache.org/jira/browse/FLINK-8897
> Project: Flink
> Issue Type: Bug
> Components: Table API & SQL
> Reporter: Xingcan Cui
> Assignee: Timo Walther
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.5.6
>
>
> As raised in [this
> thread|https://lists.apache.org/thread.html/e2ea38aa7ae224d7481145334955d84243690e9aad10d58310bdb8e7@%3Cuser.flink.apache.org%3E],
> the query created by the following code will throw a calcite "mismatch type"
> ({{Timestamp(3)}} and {{TimeIndicator}}) exception.
> {code:java}
> String sql1 = "select id, eventTs as t1, count(*) over (partition by id order
> by eventTs rows between 100 preceding and current row) as cnt1 from myTable1";
> String sql2 = "select distinct id as r_id, eventTs as t2, count(*) over
> (partition by id order by eventTs rows between 50 preceding and current row)
> as cnt2 from myTable2";
> Table left = tableEnv.sqlQuery(sql1);
> Table right = tableEnv.sqlQuery(sql2);
> left.join(right).where("id === r_id && t1 === t2").select("id,
> t1").writeToSink(...)
> {code}
> The logical plan is as follows.
> {code}
> LogicalProject(id=[$0], t1=[$1])
> LogicalFilter(condition=[AND(=($0, $3), =($1, $4))])
> LogicalJoin(condition=[true], joinType=[inner])
> LogicalAggregate(group=[{0, 1, 2}])
> LogicalWindow(window#0=[window(partition {0} order by [1] rows
> between $2 PRECEDING and CURRENT ROW aggs [COUNT()])])
> LogicalProject(id=[$0], eventTs=[$3])
> LogicalTableScan(table=[[_DataStreamTable_0]])
> LogicalAggregate(group=[{0, 1, 2}])
> LogicalWindow(window#0=[window(partition {0} order by [1] rows
> between $2 PRECEDING and CURRENT ROW aggs [COUNT()])])
> LogicalProject(id=[$0], eventTs=[$3])
> LogicalTableScan(table=[[_DataStreamTable_0]])
> {code}
> That is because the the rowtime field after an aggregation will be
> materialized while the {{RexInputRef}} type for the filter's operands ({{t1
> === t2}}) is still {{TimeIndicator}}. We should make them unified.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
