srielau commented on code in PR #59868:
URL: https://github.com/apache/spark/pull/59868#discussion_r4405826677
##########
sql/core/src/main/scala/org/apache/spark/sql/catalyst/parser/ParseSqlResult.scala:
##########
@@ -123,8 +131,66 @@ object ParseSqlResult {
JObject(fields.toList)
}
+ /** 1-based Unicode code-point offset of `segment` in `batch`. */
+ private def charStart(batch: String, segment: PositionedSqlStatement): Int =
{
+ val utf16 = math.max(0, math.min(segment.start, batch.length))
+ batch.codePointCount(0, utf16) + 1
Review Comment:
Addressed in fd544ef: `fromSql` now computes 1-based code-point starts in
one left-to-right pass (`charStarts`). Each count covers only the UTF-16 gap
since the previous statement start.
##########
sql/core/src/test/scala/org/apache/spark/sql/catalyst/parser/ParseSqlResultSuite.scala:
##########
@@ -246,4 +253,32 @@ class ParseSqlResultSuite extends SparkFunSuite {
assert(j \ "parse_success" === JBool(false))
assert(j \ "error" \ "errorClass" === JString("PARSE_SYNTAX_ERROR"))
}
+
+ test("error queryContext indexes are Unicode code points") {
Review Comment:
Addressed in fd544ef. The same test now also pins:
- a failing statement that follows and contains a supplementary character
(`start` 13, `length` 17)
- `LABELS_MISMATCH` after a supplementary character (`queryContext` 19..28,
fragment `lbl_begin:`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
