rdtr opened a new issue, #13571:
URL: https://github.com/apache/gluten/issues/13571

   ### Backend
   
   VL (Velox)
   
   ### Bug description
   
   ### Bug description
   
   **[Expected behavior]** Spark 4.0 added `try_url_decode` (SPARK-48865). It 
returns NULL when the input has an invalid `%` escape. On a column:
   
   ```scala
   Seq("http%3A%2F%2spark.apache.org", 
"abc%2").toDF("a").write.parquet("/tmp/url_tbl")
   spark.read.parquet("/tmp/url_tbl").selectExpr("try_url_decode(a)").show()
   // +-----------------+
   // |try_url_decode(a)|
   // +-----------------+
   // |             NULL|
   // |             NULL|
   // +-----------------+
   ```
   
   (A literal or `VALUES` input is evaluated by Spark before Gluten sees it, so 
the repro reads from a file.)
   
   **[Actual behavior]** With Gluten the query fails:
   
   ```
   org.apache.gluten.exception.GlutenException: Exception: VeloxUserError
   Error Source: USER
   Error Code: INVALID_ARGUMENT
   Reason: Illegal hex characters in escape (%) pattern: 2s
   Retriable: False
   Expression: endptr == buf + 2 && !std::isspace(buf[0]) && 
!std::isspace(buf[1])
   Context: Top-level Expression: url_decode(n0_0)
   Function: decodeByte
   ```
   
   **Cause.** `TryUrlDecode` is replaced by `UrlDecode(child, failOnError = 
false)`, which Spark turns into `StaticInvoke(UrlCodec, "decode", Seq(child, 
Literal(false)))` (`urlExpressions.scala`, v4.0.0). `UrlCodec.decode` returns 
NULL on `IllegalArgumentException` when `failOnError` is false. 
`ExpressionConverter.replaceStaticInvokeWithExpressionTransformer` maps every 
`UrlCodec.decode` call to Velox `url_decode` with only the first argument, so 
the flag is dropped. Velox's `url_decode` (registered from the Presto 
`UrlDecodeFunction`) always throws on a bad escape.
   
   Spark 3.5 has no `try_url_decode`, and its `UrlCodec.decode` takes the 
charset as the second argument, so `url_decode` on 3.5 is not affected.
   
   Velox has no `try_url_decode` in sparksql yet (listed in 
facebookincubator/velox#20057). Until it does, Gluten should fall back when 
`failOnError` is false.
   
   ### Gluten version
   
   main branch
   
   ### Spark version
   
   Spark-4.0.x
   
   ### Spark configurations
   
   `spark.sql.ansi.enabled=false`
   
   ### System information
   
   _No response_
   
   ### Relevant logs
   
   ```bash
   
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to