[
https://issues.apache.org/jira/browse/CASSSIDECAR-507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laxmikant Upadhyay updated CASSSIDECAR-507:
-------------------------------------------
Description:
While testing Cassandra CDC using the {{*docker/cdc-demo*}} \{*}setup from the
Apache Cassandra Sidecar trunk branch{*}, with the locally built
{{*cassandra-sidecar:dev*}} \{*}Docker image{*}, I encountered a
{{ClassCastException}} when processing CDC mutations containing nested UDT
columns.
h3. Steps to Reproduce using docker compose cdc-demo
{code:java}
1. Create a simple UDT
{{CREATE TYPE IF NOT EXISTS cdc_demo.task_result (
status text,
message text
);}}
2. Create a CDC-enabled table containing a list of frozen UDTs
{{CREATE TABLE IF NOT EXISTS cdc_demo.task (
id uuid PRIMARY KEY,
name text,
results list<frozen<task_result>>
) WITH cdc = true;}}
3. Insert a record containing the UDT collection
{{INSERT INTO cdc_demo.task (
id,
name,
results
) VALUES (
11111111-1111-1111-1111-111111111111,
'test-task',
[
{ status: 'SUCCESS', message: 'completed' }
]
);}}
{code}
*4. Observe Sidecar CDC logs*
The CDC consumer detects the mutation and resolves the correct Kafka topic, but
fails while deserializing the {{results}} column.
{code:java}
{{ERROR CdcLogMode.java:166 - Failed to deserialize column value.
columnName=results
java.lang.ClassCastException: null
INFO CdcLogMode.java:104 - Processing CDC event.
kind=INSERT keyspace=cdc_demo table=task
partitionKeys=[id : 11111111-1111-1111-1111-111111111111]
clusteringKeys=null
valueColumns=[name : test-task], [results : unknown]
topic=cdc-mutations-cdc_demo-task
WARN CdcLogMode.java:113 - Skip publishing the event because it cannot be
serialized.}}
{code}
*In a separate reproduction using a below schema, the exception included
additional classloader details:*
{code:java}
– 1. Create transient_event UDT
CREATE TYPE IF NOT EXISTS cdc_demo.transient_event (
id uuid,
event_class text
);
– 2. Create execution_result UDT
CREATE TYPE IF NOT EXISTS cdc_demo.execution_result (
domain_event frozen<transient_event>,
status text
);
– 3. Create command table with CDC enabled
CREATE TABLE IF NOT EXISTS cdc_demo.command (
id uuid PRIMARY KEY,
command_class text,
execution_results list<frozen<execution_result>>
) WITH cdc = true;
{{INSERT INTO cdc_demo.task (
id,
name,
results
) VALUES (
11111111-1111-1111-1111-111111111111,
'test-task',
[
{
status: 'SUCCESS',
message: 'completed'
}
]
);}}
{{}}
{code}
{{you will observe:}}
{code:java}
{{java.lang.ClassCastException:
class org.apache.cassandra.spark.data.complex.CqlUdt
cannot be cast to
class org.apache.cassandra.spark.data.CqlType
(org.apache.cassandra.spark.data.complex.CqlUdt
is in unnamed module of loader
org.apache.cassandra.bridge.PostDelegationClassLoader @7b1655c3;
org.apache.cassandra.spark.data.CqlType
is in unnamed module of loader
org.apache.cassandra.bridge.PostDelegationClassLoader @38afe24f)}}
{code}
h3. Expected Behavior
CDC should successfully deserialize UDT column values and publish the mutation
to the configured Kafka topic.
was:
Spotted during review of CASSSIDECAR-501.
Basically, *RestoreJobConfigurationImpl* is validated only through its builder,
while Jackson takes another path to instantiate and parse values into config
object which results in bypassing validation.
Across the codebase, validation of each config setting is delegated to its
setters. It's enough for single setting validation, but it's not possible for
settings that depend on other settings like _jobDiscoveryMinimumRecencyDays_ of
{*}RestoreJobConfigurationImpl{*}. Value _jobDiscoveryMinimumRecencyDays_
cannot be greater than
_restoreJobTablesTtl_ value which is configurable. The order of deseriazliation
is unknown, so we have to validate object right after it is deserialized
(including calls of setters).
We can achieve such by hooking into Jackson deserialize chain (using
{*}BeanDeserializerModifier{*}) and validate object there or just by creating
an all arg constructor with @JsonCreator annotation so removing all setters
entirely.
I tried to implement the first approach with help of AI
[here|https://github.com/apache/cassandra-sidecar/compare/trunk...worryg0d:cassandra-sidecar:jackson-validating-deserializer?expand=1]
> CDC fails to deserialize nested UDT columns
> -------------------------------------------
>
> Key: CASSSIDECAR-507
> URL: https://issues.apache.org/jira/browse/CASSSIDECAR-507
> Project: Sidecar for Apache Cassandra
> Issue Type: Bug
> Components: Configuration
> Reporter: Laxmikant Upadhyay
> Priority: Major
>
> While testing Cassandra CDC using the {{*docker/cdc-demo*}} \{*}setup from
> the Apache Cassandra Sidecar trunk branch{*}, with the locally built
> {{*cassandra-sidecar:dev*}} \{*}Docker image{*}, I encountered a
> {{ClassCastException}} when processing CDC mutations containing nested UDT
> columns.
> h3. Steps to Reproduce using docker compose cdc-demo
>
> {code:java}
> 1. Create a simple UDT
>
> {{CREATE TYPE IF NOT EXISTS cdc_demo.task_result (
> status text,
> message text
> );}}
> 2. Create a CDC-enabled table containing a list of frozen UDTs
>
> {{CREATE TABLE IF NOT EXISTS cdc_demo.task (
> id uuid PRIMARY KEY,
> name text,
> results list<frozen<task_result>>
> ) WITH cdc = true;}}
> 3. Insert a record containing the UDT collection
>
> {{INSERT INTO cdc_demo.task (
> id,
> name,
> results
> ) VALUES (
> 11111111-1111-1111-1111-111111111111,
> 'test-task',
> [
> { status: 'SUCCESS', message: 'completed' }
> ]
> );}}
> {code}
>
> *4. Observe Sidecar CDC logs*
> The CDC consumer detects the mutation and resolves the correct Kafka topic,
> but fails while deserializing the {{results}} column.
>
>
> {code:java}
> {{ERROR CdcLogMode.java:166 - Failed to deserialize column value.
> columnName=results
> java.lang.ClassCastException: null
> INFO CdcLogMode.java:104 - Processing CDC event.
> kind=INSERT keyspace=cdc_demo table=task
> partitionKeys=[id : 11111111-1111-1111-1111-111111111111]
> clusteringKeys=null
> valueColumns=[name : test-task], [results : unknown]
> topic=cdc-mutations-cdc_demo-task
> WARN CdcLogMode.java:113 - Skip publishing the event because it cannot be
> serialized.}}
> {code}
>
> *In a separate reproduction using a below schema, the exception included
> additional classloader details:*
>
> {code:java}
> – 1. Create transient_event UDT
> CREATE TYPE IF NOT EXISTS cdc_demo.transient_event (
> id uuid,
> event_class text
> );
> – 2. Create execution_result UDT
> CREATE TYPE IF NOT EXISTS cdc_demo.execution_result (
> domain_event frozen<transient_event>,
> status text
> );
> – 3. Create command table with CDC enabled
> CREATE TABLE IF NOT EXISTS cdc_demo.command (
> id uuid PRIMARY KEY,
> command_class text,
> execution_results list<frozen<execution_result>>
> ) WITH cdc = true;
>
> {{INSERT INTO cdc_demo.task (
> id,
> name,
> results
> ) VALUES (
> 11111111-1111-1111-1111-111111111111,
> 'test-task',
> [
> {
> status: 'SUCCESS',
> message: 'completed'
> }
> ]
> );}}
> {{}}
> {code}
>
> {{you will observe:}}
>
> {code:java}
> {{java.lang.ClassCastException:
> class org.apache.cassandra.spark.data.complex.CqlUdt
> cannot be cast to
> class org.apache.cassandra.spark.data.CqlType
> (org.apache.cassandra.spark.data.complex.CqlUdt
> is in unnamed module of loader
> org.apache.cassandra.bridge.PostDelegationClassLoader @7b1655c3;
> org.apache.cassandra.spark.data.CqlType
> is in unnamed module of loader
> org.apache.cassandra.bridge.PostDelegationClassLoader @38afe24f)}}
> {code}
>
> h3. Expected Behavior
> CDC should successfully deserialize UDT column values and publish the
> mutation to the configured Kafka topic.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
