mrhhsg opened a new pull request, #68244:
URL: https://github.com/apache/doris/pull/68244
### What problem does this PR solve?
Issue Number: None
Problem Summary: `JsonbToJson::string_to_json()` walks the string with `ptr
!= str + len && *ptr`, so it stops at the first NUL byte even though the length
says the string is longer. U+0000 is a legal character inside a JSON string and
the parser stores it by length, so every path that renders a JSONB value back
to text loses everything after it:
```
SELECT k, v FROM (SELECT 1) d
LATERAL VIEW json_each_text(concat('{"x":["a', unhex('5C'), 'u0000b"]}')) t
AS k, v;
-- before: x ["a"]
-- after: x ["a\u0000b"]
SELECT c FROM (SELECT 1) d
LATERAL VIEW explode_json_array_string(concat('[["a', unhex('5C'),
'u0000b"]]')) t AS c;
-- before: ["a"]
-- after: ["a\u0000b"]
```
The same string as a direct array element is copied by length and already
keeps the NUL, which is why only nested complex values are truncated.
Bound the loop by the length only. The NUL then reaches the escaping branch
and is written as `\u0000`, like any other control character.
### Release note
None
### Check List (For Author)
- Test:
- Unit Test: Yes, `JsonbParserTest.ParseJsonWithEscapedNulInString`,
`ParseJsonWithEscapedNulInNestedArray` and `ParseJsonWithEscapedNulInKey` round
trip a U+0000 through the JSON text output.
- Regression test: Yes, `test_json_text_embedded_nul` covers
`json_each_text` and `explode_json_array_string` over nested values that
contain U+0000.
- Behavior changed: Yes, textualized JSON values keep everything after a
U+0000 and escape it as `\u0000` instead of being silently truncated.
- Does this need documentation: No
https://claude.ai/code/session_01RZ36Pij3o8fGnYYg33PKnq
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
