[
https://issues.apache.org/jira/browse/KAFKA-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15319515#comment-15319515
]
Randall Hauch edited comment on KAFKA-3803 at 6/7/16 10:25 PM:
---------------------------------------------------------------
The {{equals(...)}} method on {{org.apache.kafka.connect.data.Struct}}
currently uses {{Arrays.equals(value,o.value)}} to compare the array of field
values to that of another. However, this only works when the elements in those
arrays (e.g., the field values) are primitives or objects, but fails to work
when they are arrays such as {{byte[]}}.
Interestingly, the {{StructTest}} unit test populates all fields of type
{{Schema.BYTES_SCHEMA}} using {{ByteBuffer}} object, which means the current
logic works fine. However, the JSON converter rehydrates the {{Struct}} objects
using {{byte[]}}, whereas the Avro converter rehydrates using {{ByteBuffer}}.
This means that when a {{Struct}} containing a {{Schema.BYTES_SCHEMA}} or
{{Schema.OPTIONAL_BYTES_SCHEMA}} field is serialized and then deserialized with
the JSON converter, the rehydrated object will not be deemed "equal" to the
original.
Apart from doing nothing and closing this issue with WONTFIX (which is a valid
option), the changes that can be made include any of the following:
# Change the {{Struct}} JavaDoc to better explain that {{ByteBuffer}} should be
used for values in all fields of type {{Schema.BYTES_SCHEMA}} or
{{Schema.OPTIONAL_BYTES_SCHEMA}}.
# Change only the JSON converter to wrap all byte arrays in {{ByteBuffer}}, to
be more consistent with current expectations of using {{ByteBuffer}} values.
# Change {{Struct.equals}} to use {{Arrays.deepEquals(...)}} so that it works
with object values including byte arrays or {{ByteBuffer}}, although
{{Struct.equals}} will only work for maps and arrays when {{ByteBuffer}} is
used and {{byte[]}} is _not_ used.
It all depends on how important it is to support {{Struct.equals}}.
was (Author: rhauch):
The {{equals(...)}} method on {{org.apache.kafka.connect.data.Struct}}
currently uses {{Arrays.equals(value,o.value)}} to compare the array of field
values to that of another. However, this only works when the elements in those
arrays (e.g., the field values) are primitives or objects, but fails to work
when they are arrays such as {{byte[]}}.
Interestingly, the {{StructTest}} unit test populates all fields of type
{{Schema.BYTES_SCHEMA}} using {{ByteBuffer}} object, which means the current
logic works fine. However, the JSON converter rehydrates the {{Struct}} objects
using {{byte[]}}, whereas the Avro converter rehydrates using {{ByteBuffer}}.
This means that when a {{Struct}} containing a {{Schema.BYTES_SCHEMA}} or
{{Schema.OPTIONAL_BYTES_SCHEMA}} field is serialized and then deserialized with
the JSON converter, the rehydrated object will not be deemed "equal" to the
original.
There are three ways to fix this:
# Change only {{Struct.equals}} to use {{Arrays.deepEquals(...)}} so that it
works with object values including byte arrays or {{ByteBuffer}}; or
# Change only the JSON converter to wrap all byte arrays in {{ByteBuffer}}, in
which case {{Struct.equals}} will properly compare values only when
{{ByteBuffer}} values are used in place of all {{byte[]}}; or
# Change both the {{Struct.equals}} to use {{Arrays.deepEquals(...)}} and JSON
converter to wrap all byte arrays in {{ByteBuffer}}. This will allow both byte
arrays and {{ByteBuffer}} values to be used while making rehydrated objects
equal to the original, and will make the JSON converter more consistent with
what appears to be the expectations.
Personally, I think either option 1 or 3 is best. I have a local fix for 1
including a change to the unit tests. Option 3 is just a bit more work.
> JsonConverter deserialized Struct containing bytes field does not return true
> for equals()
> ------------------------------------------------------------------------------------------
>
> Key: KAFKA-3803
> URL: https://issues.apache.org/jira/browse/KAFKA-3803
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Affects Versions: 0.10.0.0
> Reporter: Ewen Cheslack-Postava
>
> The problem is that byte[] comparisons will return false for equality, so
> even if the two are effectively equal, Struct.equals will not return true.
> It's unclear if this should be fixed or not. Equality wouldn't work for map
> or array types containing bytes either. However, on possibility is making
> ByteBuffer the default instead to alleviate this, although then we may end up
> with asymmetry in equality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
