wud-if commented on PR #13455:
URL: https://github.com/apache/gluten/pull/13455#issuecomment-6496346742
> Curious whether Flink has optimization like Spark's constant folding.
Then, it can be evaluated during query optimization.
>
> Could you confirm this?
Confirmed — Flink does have constant folding (Calcite's expression
reduction rules, e.g. SimplifyFilterConditionRule), but it only applies to
deterministic expressions, and CURRENT_DATE / CURRENT_TIME are dynamic
(non-deterministic) functions, so they are never folded at optimization time.
Evidence from Flink 1.19.2:
1. FunctionGenerator.scala:547 routes CURRENT_DATE / CURRENT_TIME /
CURRENT_TIMESTAMP / NOW into the dedicated dynamic-function codegen
(CurrentTimePointCallGen) — they are classified as dynamic functions and
excluded from the deterministic reductions.
2. EXPLAIN on a streaming query shows they survive the full optimization
pipeline as runtime calls, not literals:
‘’‘
Calc(select=[CURRENT_DATE() AS CURRENT_DATE, CURRENT_TIME() AS CURRENT_TIME])
+- TableSourceScan(...)
’‘’
3. CurrentTimePointCallGen distinguishes two runtime granularities —
neither of which is an optimization-time constant:
- Streaming (our case): record-level — the generated code refreshes the
value per record, so a long-running job returns a different CURRENT_DATE after
midnight, per row.
- Batch: query-level — evaluated once at operator open, still at runtime.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
