andygrove opened a new issue, #193:
URL: https://github.com/apache/datafusion-spark/issues/193
Part of #173 and apache/datafusion-comet#7300.
### What the function does
`assert_true(expr [, message])` returns NULL when `expr` is true and raises
an error otherwise, including
when `expr` is NULL. It is a `RuntimeReplaceable` that Spark rewrites to
`If(expr, NULL, raise_error(message))`, so `expr` must be `BOOLEAN` (the
`If` predicate), `message` is
implicitly cast to `STRING` by `raise_error`, and the result type is `VOID`
(`NullType`). Available since
Spark 2.0.0. No aliases.
When `message` is omitted, Spark builds the default literal `'<expr>' is not
true!` at analysis time,
where `<expr>` is Spark's rendering of the expression (see version
differences). The raised error is the
same as `raise_error`: `USER_RAISED_EXCEPTION` (SQLSTATE `P0001`) with
`errorMessage` set to the message
in 4.0+, a bare `java.lang.RuntimeException(message)` in 3.4/3.5.
### Examples
```sql
SELECT assert_true(0 < 1);
-- NULL
SELECT assert_true(true), assert_true(boolean(1));
-- NULL NULL
SELECT assert_true(false);
-- [USER_RAISED_EXCEPTION] 'false' is not true!
SELECT assert_true(boolean(0));
-- USER_RAISED_EXCEPTION, errorMessage "'0' is not true!" (4.1+)
-- USER_RAISED_EXCEPTION, errorMessage "'cast(0 as boolean)' is not true!"
(4.0)
SELECT assert_true(null);
-- USER_RAISED_EXCEPTION, errorMessage "'NULL' is not true!" (4.1+)
-- USER_RAISED_EXCEPTION, errorMessage "'null' is not true!" (4.0)
SELECT assert_true(false, 'custom error message');
-- USER_RAISED_EXCEPTION, errorMessage "custom error message"
SELECT assert_true(col1 <= col2) FROM VALUES ('2025-03-01', '2025-03-10');
-- schema struct<assert_true((col1 <= col2), '(col1 <= col2)' is not
true!):void>
-- NULL
-- spark.sql.legacy.raiseErrorWithoutErrorClass=true
SELECT assert_true(false);
-- java.lang.RuntimeException: 'false' is not true!
```
### Edge cases to test
- `expr` true -> NULL; false -> error; NULL (literal or `boolean(null)` or a
NULL column value) -> error.
- Column input where only some rows fail: the error must be raised for the
first failing row, and no
error when every row is true (or the input is empty).
- Custom message from a literal, from a column, NULL (gives message `null`
in 4.x), or non-string
(cast to `STRING`).
- Default message: the text is Spark's SQL rendering of `expr`, which
depends on the Spark version and
on analyzer rewrites (implicit casts). A native kernel cannot reproduce it
in general, so the
two-argument form with the message supplied by the caller should be the
primary API.
- Non-boolean `expr` (e.g. `assert_true(1)`) is an analysis error in Spark,
not a runtime error.
- Laziness and folding: `assert_true(false)` must not fail at plan time, and
must not fire for rows not
evaluated by an enclosing conditional.
- `spark.sql.legacy.raiseErrorWithoutErrorClass` (internal, since 4.0,
default `false`) removes the
`[USER_RAISED_EXCEPTION]` class and throws a bare `RuntimeException`.
### Differences across Spark versions
- 3.4, 3.5: raises `java.lang.RuntimeException(message)`. Default message
uses
`left.simpleString(...)`, e.g. `'cast(0 as boolean)' is not true!`,
`'null' is not true!`.
- 4.0: raises `USER_RAISED_EXCEPTION` (`P0001`); default message still uses
`simpleString`
(`'cast(0 as boolean)' is not true!`). Adds
`spark.sql.legacy.raiseErrorWithoutErrorClass`.
- 4.1, 4.2: default message uses `toPrettySQL(left)`, e.g. `'0' is not
true!`, `'NULL' is not true!`.
The message is part of the output column name, so it changes too:
`assert_true(boolean(1))` is named
`assert_true(1, '1' is not true!)` in 4.1+ and `assert_true(1, 'cast(1 as
boolean)' is not true!)` in 3.4-4.0.
### Implementation notes
- Spark:
`sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala`,
class
`AssertTrue` (and `RaiseError`, which it lowers to).
- DataFusion core has no `assert_true` or `raise_error`.
- Implement on top of the `raise_error` kernel (separate issue): evaluate
the predicate, and raise
`USER_RAISED_EXCEPTION` with the message of the first row whose predicate
is false or NULL. Mark the UDF
`Volatility::Volatile` so the simplifier does not fold it at plan time.
Belongs in the `misc` module.
- Tests: `crates/sqllogictest/test_files/misc/assert_true.slt` already
exists with commented-out queries
ported from Sail; enable and extend it. Document with `#[user_doc]`.
---
_This issue was generated by an LLM (Claude) from the Apache Spark source
code and tests. Examples come from Spark's `ExpressionDescription` examples,
test suites and `.sql.out` golden files, but please verify them against Spark
before relying on them._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]