rdtr opened a new issue, #13571:
URL: https://github.com/apache/gluten/issues/13571
### Backend
VL (Velox)
### Bug description
### Bug description
**[Expected behavior]** Spark 4.0 added `try_url_decode` (SPARK-48865). It
returns NULL when the input has an invalid `%` escape. On a column:
```scala
Seq("http%3A%2F%2spark.apache.org",
"abc%2").toDF("a").write.parquet("/tmp/url_tbl")
spark.read.parquet("/tmp/url_tbl").selectExpr("try_url_decode(a)").show()
// +-----------------+
// |try_url_decode(a)|
// +-----------------+
// | NULL|
// | NULL|
// +-----------------+
```
(A literal or `VALUES` input is evaluated by Spark before Gluten sees it, so
the repro reads from a file.)
**[Actual behavior]** With Gluten the query fails:
```
org.apache.gluten.exception.GlutenException: Exception: VeloxUserError
Error Source: USER
Error Code: INVALID_ARGUMENT
Reason: Illegal hex characters in escape (%) pattern: 2s
Retriable: False
Expression: endptr == buf + 2 && !std::isspace(buf[0]) &&
!std::isspace(buf[1])
Context: Top-level Expression: url_decode(n0_0)
Function: decodeByte
```
**Cause.** `TryUrlDecode` is replaced by `UrlDecode(child, failOnError =
false)`, which Spark turns into `StaticInvoke(UrlCodec, "decode", Seq(child,
Literal(false)))` (`urlExpressions.scala`, v4.0.0). `UrlCodec.decode` returns
NULL on `IllegalArgumentException` when `failOnError` is false.
`ExpressionConverter.replaceStaticInvokeWithExpressionTransformer` maps every
`UrlCodec.decode` call to Velox `url_decode` with only the first argument, so
the flag is dropped. Velox's `url_decode` (registered from the Presto
`UrlDecodeFunction`) always throws on a bad escape.
Spark 3.5 has no `try_url_decode`, and its `UrlCodec.decode` takes the
charset as the second argument, so `url_decode` on 3.5 is not affected.
Velox has no `try_url_decode` in sparksql yet (listed in
facebookincubator/velox#20057). Until it does, Gluten should fall back when
`failOnError` is false.
### Gluten version
main branch
### Spark version
Spark-4.0.x
### Spark configurations
`spark.sql.ansi.enabled=false`
### System information
_No response_
### Relevant logs
```bash
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]