srielau commented on code in PR #58530:
URL: https://github.com/apache/spark/pull/58530#discussion_r3938965157
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/SqlStatementSplitter.scala:
##########
@@ -58,6 +58,26 @@ case class SqlStatementSplitResult(
def isEmpty: Boolean = completeStatements.isEmpty && partialStatement.isEmpty
}
+/** A split SQL statement together with its 0-based start in the original
input. */
+private[sql] case class PositionedSqlStatement(
+ statement: String,
+ terminator: String,
+ start: Int) {
+ def length: Int = statement.length
Review Comment:
Fixed in 003156797de. ANTLR code-point offsets are now converted with
`offsetByCodePoints` before both slicing candidate regions and storing
statement starts, so `start` and `length` consistently use UTF-16 code units.
Added splitter and `parse_sql` regressions for `SELECT '😀😀'; SELECT 2;` that
verify both statements parse and each reported span reconstructs the original
statement. `SqlStatementSplitterSuite`, `ParseSqlResultSuite`, `ParseSqlSuite`,
and both Catalyst/SQL Scalastyle checks pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
