andygrove opened a new issue, #193:
URL: https://github.com/apache/datafusion-spark/issues/193

   Part of #173 and apache/datafusion-comet#7300.
   
   ### What the function does
   
   `assert_true(expr [, message])` returns NULL when `expr` is true and raises 
an error otherwise, including
   when `expr` is NULL. It is a `RuntimeReplaceable` that Spark rewrites to
   `If(expr, NULL, raise_error(message))`, so `expr` must be `BOOLEAN` (the 
`If` predicate), `message` is
   implicitly cast to `STRING` by `raise_error`, and the result type is `VOID` 
(`NullType`). Available since
   Spark 2.0.0. No aliases.
   
   When `message` is omitted, Spark builds the default literal `'<expr>' is not 
true!` at analysis time,
   where `<expr>` is Spark's rendering of the expression (see version 
differences). The raised error is the
   same as `raise_error`: `USER_RAISED_EXCEPTION` (SQLSTATE `P0001`) with 
`errorMessage` set to the message
   in 4.0+, a bare `java.lang.RuntimeException(message)` in 3.4/3.5.
   
   ### Examples
   
   ```sql
   SELECT assert_true(0 < 1);
   -- NULL
   
   SELECT assert_true(true), assert_true(boolean(1));
   -- NULL      NULL
   
   SELECT assert_true(false);
   -- [USER_RAISED_EXCEPTION] 'false' is not true!
   
   SELECT assert_true(boolean(0));
   -- USER_RAISED_EXCEPTION, errorMessage "'0' is not true!"            (4.1+)
   -- USER_RAISED_EXCEPTION, errorMessage "'cast(0 as boolean)' is not true!" 
(4.0)
   
   SELECT assert_true(null);
   -- USER_RAISED_EXCEPTION, errorMessage "'NULL' is not true!"         (4.1+)
   -- USER_RAISED_EXCEPTION, errorMessage "'null' is not true!"         (4.0)
   
   SELECT assert_true(false, 'custom error message');
   -- USER_RAISED_EXCEPTION, errorMessage "custom error message"
   
   SELECT assert_true(col1 <= col2) FROM VALUES ('2025-03-01', '2025-03-10');
   -- schema struct<assert_true((col1 <= col2), '(col1 <= col2)' is not 
true!):void>
   -- NULL
   
   -- spark.sql.legacy.raiseErrorWithoutErrorClass=true
   SELECT assert_true(false);
   -- java.lang.RuntimeException: 'false' is not true!
   ```
   
   ### Edge cases to test
   
   - `expr` true -> NULL; false -> error; NULL (literal or `boolean(null)` or a 
NULL column value) -> error.
   - Column input where only some rows fail: the error must be raised for the 
first failing row, and no
     error when every row is true (or the input is empty).
   - Custom message from a literal, from a column, NULL (gives message `null` 
in 4.x), or non-string
     (cast to `STRING`).
   - Default message: the text is Spark's SQL rendering of `expr`, which 
depends on the Spark version and
     on analyzer rewrites (implicit casts). A native kernel cannot reproduce it 
in general, so the
     two-argument form with the message supplied by the caller should be the 
primary API.
   - Non-boolean `expr` (e.g. `assert_true(1)`) is an analysis error in Spark, 
not a runtime error.
   - Laziness and folding: `assert_true(false)` must not fail at plan time, and 
must not fire for rows not
     evaluated by an enclosing conditional.
   - `spark.sql.legacy.raiseErrorWithoutErrorClass` (internal, since 4.0, 
default `false`) removes the
     `[USER_RAISED_EXCEPTION]` class and throws a bare `RuntimeException`.
   
   ### Differences across Spark versions
   
   - 3.4, 3.5: raises `java.lang.RuntimeException(message)`. Default message 
uses
     `left.simpleString(...)`, e.g. `'cast(0 as boolean)' is not true!`, 
`'null' is not true!`.
   - 4.0: raises `USER_RAISED_EXCEPTION` (`P0001`); default message still uses 
`simpleString`
     (`'cast(0 as boolean)' is not true!`). Adds 
`spark.sql.legacy.raiseErrorWithoutErrorClass`.
   - 4.1, 4.2: default message uses `toPrettySQL(left)`, e.g. `'0' is not 
true!`, `'NULL' is not true!`.
     The message is part of the output column name, so it changes too: 
`assert_true(boolean(1))` is named
     `assert_true(1, '1' is not true!)` in 4.1+ and `assert_true(1, 'cast(1 as 
boolean)' is not true!)` in 3.4-4.0.
   
   ### Implementation notes
   
   - Spark: 
`sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala`,
 class
     `AssertTrue` (and `RaiseError`, which it lowers to).
   - DataFusion core has no `assert_true` or `raise_error`.
   - Implement on top of the `raise_error` kernel (separate issue): evaluate 
the predicate, and raise
     `USER_RAISED_EXCEPTION` with the message of the first row whose predicate 
is false or NULL. Mark the UDF
     `Volatility::Volatile` so the simplifier does not fold it at plan time. 
Belongs in the `misc` module.
   - Tests: `crates/sqllogictest/test_files/misc/assert_true.slt` already 
exists with commented-out queries
     ported from Sail; enable and extend it. Document with `#[user_doc]`.
   
   ---
   _This issue was generated by an LLM (Claude) from the Apache Spark source 
code and tests. Examples come from Spark's `ExpressionDescription` examples, 
test suites and `.sql.out` golden files, but please verify them against Spark 
before relying on them._


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to