L. C. Hsieh created SPARK-61563:
-----------------------------------
Summary: Write the direction strategy of SortMergeAsOfJoinExec once
Key: SPARK-61563
URL: https://issues.apache.org/jira/browse/SPARK-61563
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 5.0.0
Reporter: L. C. Hsieh
SortMergeAsOfJoinExec chooses what a backward or a forward scan does in three
places: the interpreted scans, and the two generated scans (cursor and
buffered). The generated scans match on the direction with a default case, so a
nearest join would silently get the backward scan. It cannot get there today,
because supportCodegen is false for a nearest join, but nothing in the
generated scans says so, and the interpreted and generated scans can drift
apart.
This is to hold the choices that depend on the direction in one place that both
the interpreted and the generated scans read, to make every match on the
direction exhaustive, and to fail with an internal error for a direction that a
scan does not support.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]