srielau commented on PR #59868: URL: https://github.com/apache/spark/pull/59868#issuecomment-6475516722
Self-review notes: - `start`/`length` come from the splitter's UTF-16 span converted with `codePointCount`, so the batch offset and the trimmed statement length use the same unit. Unpaired surrogates count as one character each, the same as `String.codePointCount`. - STANDARD `queryContext` indexes are 1-based and inclusive; `fragment` is re-sliced as code points `[startIndex - 1, stopIndex)`, the same range. - Only contexts with an empty `objectType` (the statement itself) are rewritten; a view-body context keeps its own indexes. - A context outside the statement bounds is widened to the whole statement rather than producing a clipped or garbled fragment. I did not find a parser path that produces one; it is a guard. - Gap found and fixed in af12a07: the new test only covered a context spanning the whole statement. It now also checks a narrow `INVALID_TYPED_LITERAL` context that starts after an emoji, where UTF-16 slicing would shift the fragment. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
