0z5a opened a new pull request, #402:
URL: https://github.com/apache/dubbo-go-hessian2/pull/402

   <!--  Thanks for sending a pull request! 
   -->
   
   **What this PR does**:
   
   When `decode2utf8` sees a high surrogate (U+D800 to U+DBFF), it always reads 
3 more bytes to look for a low surrogate. It does this without checking how 
many chars are left in the current string chunk. #393 added a check on the 
value of those 3 bytes, but not on whether they belong to the string at all.
   
   A Java `String` can end with a lone high surrogate, for example when 
`substring` cuts an emoji in half while truncating a name or a title. As the 
comment on the existing `TestDecodeStringLoneHighSurrogateRealWorld` test 
notes, Java writes a lone surrogate as 3-byte CESU-8 (U+D83D becomes `0xED 0xA0 
0xBD`). When such a char is the last char of the chunk, or is followed by only 
one or two ASCII chars, the decoder reads past the end of the string.
   
   There are two symptoms on master:
   
   1. The lone high surrogate is the last char: the read goes past the 
`charTotal*3` buffer and the decoder panics.
   
   ```
   []byte{0x02, 'a', 0xED, 0xA0, 0xBD}  // "a\ud83d"
   panics with: runtime error: slice bounds out of range [:7] with capacity 6
   ```
   
   2. The lone high surrogate is followed by one or two ASCII chars and then 
another value: the string takes the first byte(s) of the next value, and 
decoding the rest of the stream goes wrong.
   
   ```
   []byte{0x7a, 0x03, 0xED, 0xA0, 0xBD, 'a', 'b', 0x02, 'c', 'd'}  // 
["\ud83dab", "cd"]
   error: illegal class index @idx 3
   
   []byte{0x7a, 0x02, 0xED, 0xA0, 0xBD, 'a', 0x91}  // ["\ud83da", 1]
   error: unexpected EOF
   ```
   
   The fix passes the number of chars left in the chunk to `decode2utf8`. A low 
surrogate is only looked for when at least one more char is left. The decoder 
then reads one byte first, and reads the other two only if that byte starts a 
3-byte sequence. Otherwise the high surrogate is kept as a lone 3-byte char, 
the same as #393 already does when the next char is not a low surrogate. Valid 
surrogate pairs are decoded as before.
   
   The new `TestDecodeStringLoneHighSurrogateAtChunkEnd` test covers both 
symptoms, plus a valid surrogate pair as a control. On master, 4 of its 5 cases 
fail: two panic, and two return `illegal class index @idx 3` and `unexpected 
EOF`. The valid pair case passes. With the fix, all 5 cases pass.
   
   I also ran every Go test in the module one by one. With the fix, 88 pass and 
none fail. Another 81 tests need the `test_hessian` jar, which I did not build 
locally. That set is the same on master and with the fix, and only the new test 
differs between the two runs. I also round-tripped random CJK and emoji strings 
through the encoder and decoder, including lengths at the 4096-char chunk 
boundary, with `-race`.
   
   **Which issue(s) this PR fixes**:
   <!--
   *Automatically closes linked issue when PR is merged.
   Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
   _If PR is about `failing-tests or flakes`, please post the related 
issues/tests in a comment and do not use `Fixes`_*
   -->
   None filed.
   
   **Special notes for your reviewer**:
   
   The change is limited to `decode2utf8` and its only caller, 
`readStringChunkData`. The surrogate-pair branch keeps its existing code; it 
only gets the two new guards (a char left in the chunk, and a 3-byte lead byte).
   
   **Does this PR introduce a user-facing change?**:
   <!--
   If no, just write "NONE" in the release-note block below.
   If yes, a release note is required:
   Enter your extended release note in the block below. If the PR requires 
additional action from users switching to the new release, include the string 
"action required".
   -->
   ```release-note
   Fix a panic, or wrong decoding of the following values, when a string ends 
with a lone high surrogate.
   ```
   
   This change was drafted with Claude Code (claude-opus-5-5); the test results 
above are from the runs described.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to