SEPURI-SAI-KRISHNA commented on PR #58047:
URL: https://github.com/apache/spark/pull/58047#issuecomment-5584378663
@uros-b good catch, you are right that this is older than 4.3. I traced it.
The faulty path was introduced by SPARK-43393 ("Sequence expression can
overflow", #41072), which replaced the old `getSequenceLength` helper with the
current `sequenceLength` and its `try`/`catch`. That change was backported at
the time, so its fix versions are 3.3.4, 3.4.2, 3.5.1 and 4.0.0. Checking the
release tags matches exactly: v3.3.3 does not contain the unreachable throw and
v3.3.4 does, v3.4.1 does not and v3.4.2 does, v3.5.0 does not and v3.5.1 does,
and every 4.x release has it. Before that change the length came from
`getSequenceLength`, which used `require` and had no `catch` block, so the path
did not exist.
From then to master the body of `sequenceLength` is unchanged apart from
cosmetics: the 3.x line raises `QueryExecutionErrors.unreachableError(...)`
where 4.x raises `internalError("Unreachable code reached.")`, and
`createArrayWithElementsExceedLimitError` later gained a `prettyName`
parameter. The overflow path and the inputs that reach it are the same
throughout, so every release since 3.3.4 returns the internal error for these
endpoints.
I have updated `Affects Version/s` on SPARK-58821 to 3.5.8, 4.0.4, 4.1.3,
4.2.0 and 4.3.0, the latest released version of each maintained line. I left
the 3.3 and 3.4 lines off as they are out of maintenance, but happy to add them
if you would rather record the full range.
Happy to prepare backports for whichever branches you would like to take
this to.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
