L. C. Hsieh created SPARK-61563:
-----------------------------------

             Summary: Write the direction strategy of SortMergeAsOfJoinExec once
                 Key: SPARK-61563
                 URL: https://issues.apache.org/jira/browse/SPARK-61563
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 5.0.0
            Reporter: L. C. Hsieh


SortMergeAsOfJoinExec chooses what a backward or a forward scan does in three 
places: the interpreted scans, and the two generated scans (cursor and 
buffered). The generated scans match on the direction with a default case, so a 
nearest join would silently get the backward scan. It cannot get there today, 
because supportCodegen is false for a nearest join, but nothing in the 
generated scans says so, and the interpreted and generated scans can drift 
apart.

This is to hold the choices that depend on the direction in one place that both 
the interpreted and the generated scans read, to make every match on the 
direction exhaustive, and to fail with an internal error for a direction that a 
scan does not support.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to