[
https://issues.apache.org/jira/browse/KAFKA-8713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cheng Pan updated KAFKA-8713:
-----------------------------
Description:
Class JsonConverter line: 582
{code:java}
private static JsonNode convertToJson(Schema schema, Object logicalValue) {
if (logicalValue == null) {
if (schema == null) // Any schema is valid and we don't have a
default, so treat this as an optional schema
return null;
if (schema.defaultValue() != null)
return convertToJson(schema, schema.defaultValue());
if (schema.isOptional())
return JsonNodeFactory.instance.nullNode();
throw new DataException("Conversion error: null value for field
that is required and has no default value");
}
....
}
{code}
h1.The Excepted:
Value `null` is valid for an optional filed, even though the filed has a
default value.
Only when field is required, the converter return default value fallback when
value is `null`.
h1.The Current:
Always return default value if `null` was given.
h1. Addition:
I'm not sure if the current behavior is the exactly expected, but at least on
MySQL, a table define as
{code:sql}
create table t1 {
name varchar(40) not null,
create_time datetime default '1999-01-01 11:11:11' null,
update_time datetime default '1999-01-01 11:11:11' null
}
{code}
Just insert a record:
{code:sql}
INSERT INTO `t1` (`name`, `update_time`) VALUES ('kafka', null);
{code}
The result is:
{code:json}
{
"name": "kafka",
"create_time": "1999-01-01 11:11:11",
"update_time": null
}
{code}
But when I use debezium pull binlog and send the record to Kafka with
JsonConverter, the result changed to:
{code:json}
{
"name": "kafka",
"create_time": "1999-01-01 11:11:11",
"update_time": "1999-01-01 11:11:11"
}
{code}
For details, see: https://issues.jboss.org/browse/DBZ-1064
was:
Class JsonConverter line: 582
{code:java}
private static JsonNode convertToJson(Schema schema, Object logicalValue) {
if (logicalValue == null) {
if (schema == null) // Any schema is valid and we don't have a
default, so treat this as an optional schema
return null;
if (schema.defaultValue() != null)
return convertToJson(schema, schema.defaultValue());
if (schema.isOptional())
return JsonNodeFactory.instance.nullNode();
throw new DataException("Conversion error: null value for field
that is required and has no default value");
}
....
}
{code}
h1.The Excepted:
Value `null` is valid for an optional filed, even though the filed has a
default value.
Only when field is required, the converter return default value fallback when
value is `null`.
h1.The Current:
Always return default value if `null` was given.
h1. Addition:
I'm not sure if the current behavior is the exactly expected, but at least on
MySQL, a table define as
{code:sql}
create table t1 {
name varchar(40) not null,
create_time datetime default '1999-01-01 11:11:11' null,
update_time datetime default '1999-01-01 11:11:11' null
}
{code}
Just insert a record:
{code:sql}
INSERT INTO `t1` (`name`, `update_time`) VALUES ('kafka', null);
{code}
The result is:
{code:json}
{
"name": "kafka",
"create_time": "1999-01-01 11:11:11",
"update_time": null
}
{code}
But when I use debezium pull binlog and send the record to Kafka with
JsonConverter, the result changed to:
{code:json}
{
"name": "kafka",
"create_time": "1999-01-01 11:11:11",
"update_time": "1999-01-01 11:11:11"
}
{code}
> Connect JsonConverter swallow null value on optional filed which has default
> value
> ----------------------------------------------------------------------------------
>
> Key: KAFKA-8713
> URL: https://issues.apache.org/jira/browse/KAFKA-8713
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Affects Versions: 2.0.0, 2.0.1, 2.1.0, 2.2.0, 2.3.0, 2.2.1
> Reporter: Cheng Pan
> Priority: Major
>
> Class JsonConverter line: 582
> {code:java}
> private static JsonNode convertToJson(Schema schema, Object logicalValue)
> {
> if (logicalValue == null) {
> if (schema == null) // Any schema is valid and we don't have a
> default, so treat this as an optional schema
> return null;
> if (schema.defaultValue() != null)
> return convertToJson(schema, schema.defaultValue());
> if (schema.isOptional())
> return JsonNodeFactory.instance.nullNode();
> throw new DataException("Conversion error: null value for field
> that is required and has no default value");
> }
> ....
> }
> {code}
> h1.The Excepted:
> Value `null` is valid for an optional filed, even though the filed has a
> default value.
> Only when field is required, the converter return default value fallback
> when value is `null`.
> h1.The Current:
> Always return default value if `null` was given.
> h1. Addition:
> I'm not sure if the current behavior is the exactly expected, but at least on
> MySQL, a table define as
> {code:sql}
> create table t1 {
> name varchar(40) not null,
> create_time datetime default '1999-01-01 11:11:11' null,
> update_time datetime default '1999-01-01 11:11:11' null
> }
> {code}
> Just insert a record:
> {code:sql}
> INSERT INTO `t1` (`name`, `update_time`) VALUES ('kafka', null);
> {code}
> The result is:
> {code:json}
> {
> "name": "kafka",
> "create_time": "1999-01-01 11:11:11",
> "update_time": null
> }
> {code}
> But when I use debezium pull binlog and send the record to Kafka with
> JsonConverter, the result changed to:
> {code:json}
> {
> "name": "kafka",
> "create_time": "1999-01-01 11:11:11",
> "update_time": "1999-01-01 11:11:11"
> }
> {code}
> For details, see: https://issues.jboss.org/browse/DBZ-1064
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
