[
https://issues.apache.org/jira/browse/CASSANALYTICS-217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jyothsna Konisa reassigned CASSANALYTICS-217:
---------------------------------------------
Assignee: Jyothsna Konisa
> Non-frozen UDTs with unset fields or partial updates fail to deserialize in
> bulk reader and CDC
> -----------------------------------------------------------------------------------------------
>
> Key: CASSANALYTICS-217
> URL: https://issues.apache.org/jira/browse/CASSANALYTICS-217
> Project: Apache Cassandra Analytics
> Issue Type: Bug
> Reporter: Jyothsna Konisa
> Assignee: Jyothsna Konisa
> Priority: Normal
>
> A non-frozen UDT stores one cell per field, so a field that wasn't written
> has no cell. This happens after a partial update (UPDATE t SET udt.field =
> value) or a full write that leaves out a field (INSERT ... {status: 'X'}).
> UdtBuffer packs the cells in order without their field positions,so
> CqlUdt.deserializeUdt fails:
>
>
>
>
>
>
>
> IllegalArgumentException: Unexpected number of fields deserializing UDT
> 'single_result', expected 2 fields but 1 found
>
>
>
>
>
>
> Impact:
>
>
>
> - Bulk reader: the Spark job aborts when it reads such a row.
>
>
>
> - CDC: the event is retried forever and blocks the partition.
>
>
>
> - CDC: SET udt.field = null publishes the whole column as null, so
> consumers can't tell which field was cleared.
>
>
>
>
>
>
> Fix:
>
>
>
> - UdtBuffer places each cell at the field position from its cell path and
> packs missing fields as null. This fixes both the bulk reader and CDC.
>
>
> - CDC updateFields lists the fields written by a partial update as
> udt.field (e.g. result.code); a full write still lists the column.
>
>
> - UDT fields in CDC Avro payload schemas are now nullable, and the envelope
> version is bumped to "3".
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
