Stephen0421 opened a new pull request, #10514:
URL: https://github.com/apache/paimon/pull/10514
### Purpose
Flink `sys.data_evolution_merge_into` previously rejected updates to
managed raw BLOB
columns. This PR supports updating `BLOB`, `ARRAY<BLOB>`, and `MAP<K,
BLOB>` columns,
including updates to `NULL`.
- Unchanged rows are written as placeholders, so their payloads stay in
the previous
blob file instead of being copied.
- Descriptor bytes keep a catalog-aware reader across the sort buffer, so
an external
blob descriptor is copied instead of failing with a null `FileIO`.
- Blob files produced by one partial write receive contiguous
`first_row_id`s.
- Video-frame fields are rejected: they require a `VideoFrameDescriptor`,
and BYTES
input is not a video frame.
- A single row larger than `write-buffer-size` (256 MB by default) fails
the job,
because the managed BLOB value is inlined into the merge sort buffer.
Documented.
Also in this PR:
- Removes `CoreOptions#updatableBlobFields()`. It was an alias of
`blobInlineField()`
with a single caller, and its contract ("only descriptor fields are
updatable") no
longer holds.
- Fixes the engine name in the BLOB `MERGE INTO` table: the procedure is
`sys.data_evolution_merge_into`, not `sys.merge_into`.
### Tests
`DataEvolutionMergeIntoActionITCase` covers:
- Scalar, `NULL`, array, and map updates, plus existing non-blob updates
on raw and descriptor blob tables.
- A blob-column update with `blob.target-file-size=1b`: new blob files
have contiguous `first_row_id`s, and an unchanged 64 KB payload is not copied
into the new blob file.
- Two successive updates, a local `sys.path_to_descriptor` source, and
rejection of a `video-frame-field` column.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
