manner opened a new pull request, #29884: URL: https://github.com/apache/flink/pull/29884
## What is the purpose of the change A variant stores a key once, but `XML_STRING` writes it for each item of its array. So a variant within the 16 MiB size limit can give an XML string of billions of characters. For example, a 1,000-character key with 3,000,000 nulls is a 12 MB variant. `XML_STRING` then fails with an `OutOfMemoryError`, which takes down the job instead of failing the row with a clear error. This PR limits the XML string to 16,777,216 characters, the same as the size limit of a variant. A longer result fails with a `TableRuntimeException`. The PR also has a separate docs commit for FLINK-40807, which fixes two statements in the docs of `PARSE_XML` and `XML_STRING`. ## Brief change log - `VariantToXmlWriter` checks the length of the output after each item of content and once at the end, and fails if it is longer than `BinaryVariantUtil.SIZE_LIMIT`. - The docs of `XML_STRING` mention the limit. - Docs fix (FLINK-40807): a `dateTime` whose offset moves it past the year 9999 in UTC stays a `STRING`, but the docs said it maps to a `TIMESTAMP_LTZ`. The docs of `XML_STRING` also said that every value other than a string or a UUID gets an `xsi:type`. Attribute values and text next to child elements are written without one and read back as strings. ## Verifying this change This change added tests and can be verified as follows: - `VariantToXmlWriterTest#testLongerThanMaxLength` writes the variant from the ticket and expects the error. Without the check in the loop, the test fails with `OutOfMemoryError: Java heap space`. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no - The serializers: no - The runtime per-record code paths (performance sensitive): yes. `XML_STRING` compares the output length with the limit once per item. A benchmark with and without the check showed no difference beyond the noise of about ±3%, for documents with 10 and with 10,000 elements. - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? no - If yes, how is the feature documented? not applicable --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Claude Opus 5.5 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
