[
https://issues.apache.org/jira/browse/GROOVY-12085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18088662#comment-18088662
]
ASF GitHub Bot commented on GROOVY-12085:
-----------------------------------------
paulk-asert merged PR #2606:
URL: https://github.com/apache/groovy/pull/2606
> SourceText slices UTF-16 with code-point AST columns, truncating source for
> supplementary characters
> ----------------------------------------------------------------------------------------------------
>
> Key: GROOVY-12085
> URL: https://issues.apache.org/jira/browse/GROOVY-12085
> Project: Groovy
> Issue Type: Bug
> Reporter: Paul King
> Assignee: Paul King
> Priority: Major
> Labels: breaking
>
> Power-assert renders truncated source text for any assertion containing
> supplementary (astral-plane) characters such as emoji. Anything reusing
> {{SourceText}} (e.g. groovy-contracts/groovy-verify source capture) is
> affected, and the same column mismatch mis-positions syntax-error carets.
> h4. Steps to reproduce
> {code:groovy}
> assert (true ? '🥤🐝' : 'z').length() == 999
> {code}
> The rendered expression is cut short by one character per emoji preceding the
> end (e.g. {{== 999}} renders as {{== 9}}).
> h4. Root cause
> AST column numbers are *code-point*-based ({{GroovyLangLexer}} uses
> {{CharStreams.fromReader}} -> ANTLR {{CodePointCharStream}};
> {{PositionConfigureUtils}} sets {{columnNumber = getCharPositionInLine() +
> 1}}), but {{SourceText}} slices a *UTF-16* {{String}} from
> {{SourceUnit.getSample()}} with {{substring()}} using those columns. Each
> astral char before the slice boundary under-counts the UTF-16 index by one,
> cutting the slice short.
> Compounding this, {{PositionConfigureUtils}} computes {{lastColumnNumber =
> getCharPositionInLine() + 1 + token.getText().length()}} -- a code-point
> start plus a UTF-16 length. The UTF-16 term accidentally compensates for
> astral chars *inside* the final token (so "emoji as last token" works) but
> not for those *before* it, which is why the bug looks intermittent.
> h4. Suggested fix
> Make column numbers uniformly code-point-based
> ({{token.getText().codePointCount(0, len)}} in {{PositionConfigureUtils}}),
> then convert code-point columns to UTF-16 indices at the slice sites via
> {{String.offsetByCodePoints}} ({{SourceText}}, and the syntax-error message
> renderers). Note: converting the hybrid {{lastColumnNumber}} without the
> {{codePointCount}} fix first will overshoot/throw, so both changes are needed
> together.
> h4. Affects
> All current versions (present on master). Affects power-assert rendering and
> any consumer of {{SourceText}}/AST column positions for supplementary
> characters.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
