Yang Jie created SPARK-58440:
--------------------------------
Summary: Sequence codegen puts non-String values into the error
message parameter map
Key: SPARK-58440
URL: https://issues.apache.org/jira/browse/SPARK-58440
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 5.0.0
Reporter: Yang Jie
Under whole-stage codegen, {{sequence()}} raising the "Illegal sequence
boundaries" error
produces a {{SparkIllegalArgumentException}} whose message parameters are not
Strings, even
though {{SparkThrowable.getMessageParameters()}} declares
{{java.util.Map<String, String>}}.
{{Sequence.genSequenceLengthCode}} generates:
{code:java}
java.util.Map<String, String> params = new java.util.HashMap<String, String>();
params.put("start", $start);
params.put("stop", $stop);
params.put("step", $step);
throw new
org.apache.spark.SparkIllegalArgumentException("_LEGACY_ERROR_TEMP_3243",
params);
{code}
{{$start}} and {{$stop}} are numeric expressions, and {{$step}} is either
numeric or a
{{CalendarInterval}} reference depending on the sequence implementation.
Janino, which
compiles generated code, erases the type arguments and binds these calls to
{{put(Object, Object)}}, so the values are inserted unconverted. The map then
flows into
{{SparkIllegalArgumentException}} and is returned by {{getMessageParameters()}}
holding
{{java.lang.Long}} / {{java.lang.Integer}} / {{CalendarInterval}} values. Any
consumer that
reads a value as a {{String}} - the declared type - fails with a
{{ClassCastException}}.
The rendered message text is unaffected, because parameter substitution calls
{{toString}}. That is why this has gone unnoticed: only the structured
parameter map is
wrong, and this error path has no test coverage.
Reproducing with the parameter map (codegen path):
{code:scala}
val e = intercept[SparkIllegalArgumentException] {
spark.sql("SELECT sequence(1, 2, 0)").collect()
}
e.getMessageParameters // {start=1, stop=2, step=0} with Integer values, not
String
{code}
The fix is to convert in the generated source, i.e. {{String.valueOf($start)}}.
Affected versions: the map-based parameters were introduced by SPARK-46991
({{f5b0de07eff}}, replacing {{IllegalArgumentException}} with
{{SparkIllegalArgumentException}} in catalyst), so this is present from 4.0.0
onward.
Verified present on master, branch-4.x, branch-4.2, branch-4.1 and branch-4.0.
branch-3.5
is not affected - it still builds the message by string concatenation.
The interpreted path ({{Sequence.getSequenceLength}}) is a separate matter: it
throws a
plain {{IllegalArgumentException}} via {{require}} with no error class and no
parameters,
so codegen and interpreted execution are not symmetric here. Reconciling that
would change
a user-visible exception type and is out of scope.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
