[
https://issues.apache.org/jira/browse/SPARK-58506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-58506:
----------------------------------
Affects Version/s: 4.3.0
(was: 5.0.0)
> Assert Spark's own error contract in codegen compile failure tests
> ------------------------------------------------------------------
>
> Key: SPARK-58506
> URL: https://issues.apache.org/jira/browse/SPARK-58506
> Project: Spark
> Issue Type: Test
> Components: SQL
> Affects Versions: 4.3.0
> Reporter: Yang Jie
> Assignee: Yang Jie
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.3.0
>
>
> Two tests around codegen compile failures assert on the compiler's own
> diagnostic
> wording instead of what Spark itself produces.
> *{{CodeGeneratorWithInterpretedFallbackSuite}}*, test "codegen failures in
> the CODEGEN_ONLY
> mode":
> {code:scala}
> assert(errMsg.contains("Failed to compile:
> org.codehaus.commons.compiler.CompileException:"))
> {code}
> The class name here is Janino's. What Spark owns on this path is the wrapping
> - a
> source-level failure goes through {{QueryExecutionErrors.compilerError}},
> whose checked
> {{CompileException}} the compile cache wraps in an {{ExecutionException}}
> (SPARK-23711 /
> SPARK-25140 established exactly this) - and the {{"Failed to compile: "}}
> prefix that
> {{failedToCompileMsg}} prepends. Asserting the cause's type states that
> contract directly;
> asserting the class name inside a message states it by accident.
> *{{ObjectExpressionsSuite}}*, test "SPARK-23593: InitializeJavaBean should
> support
> interpreted execution":
> {code:scala}
> checkExceptionInExpression[Exception](initializeWithNonexistingMethod,
> """A method named "nonexistent" is not declared in any enclosing class """ +
> "nor any supertype")
> {code}
> {{checkExceptionInExpression}} applies one substring to both the interpreted
> and the codegen
> path, so this string has to match both. It does, but only because the two
> happen to overlap:
> interpreted execution raises Spark's own {{INTERNAL_ERROR}} from
> {{QueryExecutionErrors.methodNotDeclaredError}}, whose text is
> {noformat}
> [INTERNAL_ERROR] A method named "nonexistent" is not declared in any
> enclosing class nor any supertype SQLSTATE: XX000
> {noformat}
> while the codegen path fails in the Java compiler, which reports
> {noformat}
> org.codehaus.commons.compiler.CompileException: File 'generated.java', Line
> 41, Column 12: Failed to compile:
> org.codehaus.commons.compiler.CompileException: File 'generated.java', Line
> 41, Column 12: A method named "nonexistent" is not declared in any enclosing
> class nor any supertype, nor through a static import
> {noformat}
> Janino's diagnostic is a superstring of Spark's sentence, so one assertion
> covers both legs.
> Asserting each path against what raises it is more direct: the interpreted
> leg has an error
> class, parameters and a sqlState that {{checkError}} can pin exactly, which
> is stronger than
> the substring match it replaces.
> The same test also has an assertion that never ran - the {{contains}} result
> is discarded:
> {code:scala}
> intercept[Exception] {
> evaluateWithoutCodegen(initializeWithWrongParamType,
> InternalRow.fromSeq(Seq()))
> }.getMessage.contains(
> """A method named "setX" is not declared in any enclosing class """ +
> "nor any supertype")
> {code}
> It has been a bare expression since SPARK-23593 introduced it in 2018, so
> only "an exception
> was thrown" was ever verified. That leg raises the same
> {{methodNotDeclaredError}}, so it can
> use {{checkError}} as well.
> Not in scope, found while auditing for similar sites:
> {{DataFrameSuite.scala}}'s ignored test
> "SPARK-19372: Filter can be executed w/o generated code due to JVM code size
> limit" asserts
> {{e.contains("grows beyond 64 KiB")}}, but Janino's string is {{Code grows
> beyond 64 KB}} - no
> released version spells it {{KiB}}. The assertion is already stale and only
> survives because
> the test is {{ignore}}d.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
