[
https://issues.apache.org/jira/browse/KAFKA-5972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192178#comment-16192178
]
ASF GitHub Bot commented on KAFKA-5972:
---------------------------------------
GitHub user shivsantham opened a pull request:
https://github.com/apache/kafka/pull/4021
KAFKA-5972 Flatten SMT does not work with null values
A bug in Flatten SMT while doing tests with different SMTs that are
provided out-of-box. Flatten SMT does not work as expected with schemaless JSON
that has properties with null values.
Example json:
{A={D=dValue, B=null, C=cValue}}
The issue is in if statement that checks for null value.
CURRENT VERSION:
for (Map.Entry<String, Object> entry : originalRecord.entrySet()) {
final String fieldName = fieldName(fieldNamePrefix,
entry.getKey());
Object value = entry.getValue();
if (value == null) {
newRecord.put(fieldName(fieldNamePrefix, entry.getKey()),
null);
return;
}
PROPOSED VERSION:
for (Map.Entry<String, Object> entry : originalRecord.entrySet()) {
final String fieldName = fieldName(fieldNamePrefix,
entry.getKey());
Object value = entry.getValue();
if (value == null) {
newRecord.put(fieldName(fieldNamePrefix, entry.getKey()),
null);
continue;
}
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/shivsantham/kafka trunk
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/4021.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4021
----
commit ff377759a943c7bfb89a56ad721e7ba1b3b0b24c
Author: siva santhalingam <ssanthalin...@netskope.com>
Date: 2017-09-28T23:37:47Z
KAFKA-5967 Ineffective check of negative value in
CompositeReadOnlyKeyValueStore#approximateNumEntries()
long total = 0;
for (ReadOnlyKeyValueStore<K, V> store : stores) {
total += store.approximateNumEntries();
}
return total < 0 ? Long.MAX_VALUE : total;
The check for negative value seems to account for wrapping. However,
wrapping can happen within the for loop. So the check should be performed
inside the loop.
commit 3ea736ac17a4a8ce799b1214f6c0b167b44ee977
Author: siva santhalingam <ssanthalin...@netskope.com>
Date: 2017-09-29T03:16:29Z
KAFKA-5967 Ineffective check of negative value in
CompositeReadOnlyKeyValueStore
Adding a test for #KAFKA-5967
commit 921664384a7d6f53e2cc76cf5699021cdca73893
Author: siva santhalingam <ssanthalin...@netskope.com>
Date: 2017-09-30T08:22:41Z
KAFKA-5967 Ineffective check of negative value in
CompositeReadOnlyKeyValueStore
-Fixing test
commit 48e50992139c03099b3249195efc316a84d6bba1
Author: siva santhalingam <ssanthalin...@netskope.com>
Date: 2017-10-03T17:41:18Z
Flatten SMT does not work with null values
A bug in Flatten SMT while doing tests with different SMTs that are
provided out-of-box. Flatten SMT does not work as expected with schemaless JSON
that has properties with null values.
Example json:
{A={D=dValue, B=null, C=cValue}}
The issue is in if statement that checks for null value.
CURRENT VERSION:
for (Map.Entry<String, Object> entry : originalRecord.entrySet()) {
final String fieldName = fieldName(fieldNamePrefix,
entry.getKey());
Object value = entry.getValue();
if (value == null) {
newRecord.put(fieldName(fieldNamePrefix, entry.getKey()),
null);
return;
}
PROPOSED VERSION:
for (Map.Entry<String, Object> entry : originalRecord.entrySet()) {
final String fieldName = fieldName(fieldNamePrefix,
entry.getKey());
Object value = entry.getValue();
if (value == null) {
newRecord.put(fieldName(fieldNamePrefix, entry.getKey()),
null);
continue;
}
commit 7abd3d5c76febc3e3f90b140aaf4c8bec7e08e8a
Author: siva santhalingam <ssanthalin...@netskope.com>
Date: 2017-10-03T18:07:07Z
Revert "Flatten SMT does not work with null values"
This reverts commit 48e50992139c03099b3249195efc316a84d6bba1.
commit 6f9c11726d1baf9eb2446867899218a4f46a77a4
Author: shivsantham <siva.santhalin...@gmail.com>
Date: 2017-10-04T22:53:49Z
Merge branch 'trunk' of https://github.com/shivsantham/kafka into trunk
commit 084466bfd1f8bb9c9a07ec4f8255a42dfc6b8768
Author: shivsantham <siva.santhalin...@gmail.com>
Date: 2017-10-04T22:59:27Z
KAFKA-5972 Flatten SMT does not work with null values
A bug in Flatten SMT while doing tests with different SMTs that are
provided out-of-box. Flatten SMT does not work as expected with schemaless JSON
that has properties with null values.
Example json:
{A={D=dValue, B=null, C=cValue}}
The issue is in if statement that checks for null value.
CURRENT VERSION:
for (Map.Entry<String, Object> entry : originalRecord.entrySet()) {
final String fieldName = fieldName(fieldNamePrefix,
entry.getKey());
Object value = entry.getValue();
if (value == null) {
newRecord.put(fieldName(fieldNamePrefix, entry.getKey()),
null);
return;
}
PROPOSED VERSION:
for (Map.Entry<String, Object> entry : originalRecord.entrySet()) {
final String fieldName = fieldName(fieldNamePrefix,
entry.getKey());
Object value = entry.getValue();
if (value == null) {
newRecord.put(fieldName(fieldNamePrefix, entry.getKey()),
null);
continue;
}
----
> Flatten SMT does not work with null values
> ------------------------------------------
>
> Key: KAFKA-5972
> URL: https://issues.apache.org/jira/browse/KAFKA-5972
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Affects Versions: 0.11.0.0, 0.11.0.1
> Reporter: Tomas Zuklys
> Assignee: siva santhalingam
> Priority: Minor
> Labels: easyfix, patch
> Attachments: kafka-transforms.patch
>
>
> Hi,
> I noticed a bug in Flatten SMT while doing tests with different SMTs that are
> provided out-of-box.
> Flatten SMT does not work as expected with schemaless JSON that has
> properties with null values.
> Example json:
> {code}
> {A={D=dValue, B=null, C=cValue}}
> {code}
> The issue is in if statement that checks for null value.
> Current version:
> {code}
> for (Map.Entry<String, Object> entry : originalRecord.entrySet()) {
> final String fieldName = fieldName(fieldNamePrefix,
> entry.getKey());
> Object value = entry.getValue();
> if (value == null) {
> newRecord.put(fieldName(fieldNamePrefix, entry.getKey()),
> null);
> return;
> }
> ...
> {code}
> should be
> {code}
> for (Map.Entry<String, Object> entry : originalRecord.entrySet()) {
> final String fieldName = fieldName(fieldNamePrefix,
> entry.getKey());
> Object value = entry.getValue();
> if (value == null) {
> newRecord.put(fieldName(fieldNamePrefix, entry.getKey()),
> null);
> continue;
> }
> {code}
> I have attached a patch containing the fix for this issue.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
