Jyothsna Konisa created CASSANALYTICS-217: ---------------------------------------------
Summary: Non-frozen UDTs with unset fields or partial updates fail
to deserialize in bulk reader and CDC
Key: CASSANALYTICS-217
URL: https://issues.apache.org/jira/browse/CASSANALYTICS-217
Project: Apache Cassandra Analytics
Issue Type: Bug
Reporter: Jyothsna Konisa
A non-frozen UDT stores one cell per field, so a field that wasn't written has
no cell. This happens after a partial update (UPDATE t SET udt.field = value)
or a full write that leaves out a field (INSERT ... {status: 'X'}). UdtBuffer
packs the cells in order without their field positions,so CqlUdt.deserializeUdt
fails:
IllegalArgumentException: Unexpected number of fields deserializing UDT
'single_result', expected 2 fields but 1 found
Impact:
- Bulk reader: the Spark job aborts when it reads such a row.
- CDC: the event is retried forever and blocks the partition.
- CDC: SET udt.field = null publishes the whole column as null, so consumers
can't tell which field was cleared.
Fix:
- UdtBuffer places each cell at the field position from its cell path and
packs missing fields as null. This fixes both the bulk reader and CDC.
- CDC updateFields lists the fields written by a partial update as udt.field
(e.g. result.code); a full write still lists the column.
- UDT fields in CDC Avro payload schemas are now nullable, and the envelope
version is bumped to "3".
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
