Weiqing Yang created FLINK-41446:
------------------------------------
Summary: EARLY_FIRE hint makes batch queries fail with "Unknown
join strategy : EARLY_FIRE"
Key: FLINK-41446
URL: https://issues.apache.org/jira/browse/FLINK-41446
Project: Flink
Issue Type: Bug
Components: Table SQL / Planner
Affects Versions: 2.4.0
Reporter: Weiqing Yang
Assignee: Weiqing Yang
Fix For: 2.4.0
*What happens*
Any query with an {{EARLY_FIRE}} hint fails to plan in batch mode:
{noformat}
org.apache.flink.table.api.ValidationException: Unknown join strategy :
EARLY_FIRE
{noformat}
This also happens on inner joins, where the docs say the hint is ignored. In
streaming mode the same queries plan and run.
*How to reproduce* (SQL Client)
{code:sql}
SET 'execution.runtime-mode' = 'batch';
CREATE TEMPORARY TABLE orders (id BIGINT, ts TIMESTAMP(3), WATERMARK FOR ts AS
ts) WITH ('connector' = 'datagen', 'number-of-rows' = '10');
CREATE TEMPORARY TABLE shipments (id BIGINT, ts TIMESTAMP(3), WATERMARK FOR ts
AS ts) WITH ('connector' = 'datagen', 'number-of-rows' = '10');
EXPLAIN SELECT /*+ EARLY_FIRE('delay'='5s') */ o.id, s.id
FROM orders o LEFT JOIN shipments s
ON o.id = s.id AND o.ts BETWEEN s.ts - INTERVAL '1' MINUTE AND s.ts;
{code}
* With the hint, LEFT or INNER join, batch: fails with the error above.
* Without the hint, batch: plans as a {{HashJoin}}.
* With the hint, streaming: plans as an {{IntervalJoin}} with an early-fire
delay of 5000 ms.
*Cause*
{{EARLY_FIRE}} is a {{JoinStrategy}} value, so
{{BatchPhysicalJoinRuleBase#getFirstValidJoinHint}} passes it to
{{checkJoinStrategyValid}}. That method returns {{(false, false)}} for
{{LOOKUP}}, which makes the batch rule skip the hint, but it has no case for
{{EARLY_FIRE}} and falls through to the default branch, which throws. The batch
join hint tests ({{JoinHintTestBase}}) exclude {{EARLY_FIRE}}, so no test
reaches this path.
*Proposed fix*
Handle {{EARLY_FIRE}} like {{LOOKUP}} in {{checkJoinStrategyValid}}, so the
batch planner ignores it and picks the join strategy as if no hint were given.
Early fire only changes when a streaming join emits, so a batch join has
nothing to apply it to. Add batch planner tests for the hint on LEFT and INNER
joins, and state in the docs that the hint is ignored in batch mode.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
