GitHub user shivsantham opened a pull request:
https://github.com/apache/kafka/pull/4021
KAFKA-5972 Flatten SMT does not work with null values
A bug in Flatten SMT while doing tests with different SMTs that are
provided out-of-box. Flatten SMT does not work as expected with schemaless JSON
that has properties with null values.
Example json:
{A={D=dValue, B=null, C=cValue}}
The issue is in if statement that checks for null value.
CURRENT VERSION:
for (Map.Entry<String, Object> entry : originalRecord.entrySet()) {
final String fieldName = fieldName(fieldNamePrefix,
entry.getKey());
Object value = entry.getValue();
if (value == null) {
newRecord.put(fieldName(fieldNamePrefix, entry.getKey()),
null);
return;
}
PROPOSED VERSION:
for (Map.Entry<String, Object> entry : originalRecord.entrySet()) {
final String fieldName = fieldName(fieldNamePrefix,
entry.getKey());
Object value = entry.getValue();
if (value == null) {
newRecord.put(fieldName(fieldNamePrefix, entry.getKey()),
null);
continue;
}
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/shivsantham/kafka trunk
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/4021.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4021
----
commit ff377759a943c7bfb89a56ad721e7ba1b3b0b24c
Author: siva santhalingam <[email protected]>
Date: 2017-09-28T23:37:47Z
KAFKA-5967 Ineffective check of negative value in
CompositeReadOnlyKeyValueStore#approximateNumEntries()
long total = 0;
for (ReadOnlyKeyValueStore<K, V> store : stores) {
total += store.approximateNumEntries();
}
return total < 0 ? Long.MAX_VALUE : total;
The check for negative value seems to account for wrapping. However,
wrapping can happen within the for loop. So the check should be performed
inside the loop.
commit 3ea736ac17a4a8ce799b1214f6c0b167b44ee977
Author: siva santhalingam <[email protected]>
Date: 2017-09-29T03:16:29Z
KAFKA-5967 Ineffective check of negative value in
CompositeReadOnlyKeyValueStore
Adding a test for #KAFKA-5967
commit 921664384a7d6f53e2cc76cf5699021cdca73893
Author: siva santhalingam <[email protected]>
Date: 2017-09-30T08:22:41Z
KAFKA-5967 Ineffective check of negative value in
CompositeReadOnlyKeyValueStore
-Fixing test
commit 48e50992139c03099b3249195efc316a84d6bba1
Author: siva santhalingam <[email protected]>
Date: 2017-10-03T17:41:18Z
Flatten SMT does not work with null values
A bug in Flatten SMT while doing tests with different SMTs that are
provided out-of-box. Flatten SMT does not work as expected with schemaless JSON
that has properties with null values.
Example json:
{A={D=dValue, B=null, C=cValue}}
The issue is in if statement that checks for null value.
CURRENT VERSION:
for (Map.Entry<String, Object> entry : originalRecord.entrySet()) {
final String fieldName = fieldName(fieldNamePrefix,
entry.getKey());
Object value = entry.getValue();
if (value == null) {
newRecord.put(fieldName(fieldNamePrefix, entry.getKey()),
null);
return;
}
PROPOSED VERSION:
for (Map.Entry<String, Object> entry : originalRecord.entrySet()) {
final String fieldName = fieldName(fieldNamePrefix,
entry.getKey());
Object value = entry.getValue();
if (value == null) {
newRecord.put(fieldName(fieldNamePrefix, entry.getKey()),
null);
continue;
}
commit 7abd3d5c76febc3e3f90b140aaf4c8bec7e08e8a
Author: siva santhalingam <[email protected]>
Date: 2017-10-03T18:07:07Z
Revert "Flatten SMT does not work with null values"
This reverts commit 48e50992139c03099b3249195efc316a84d6bba1.
commit 6f9c11726d1baf9eb2446867899218a4f46a77a4
Author: shivsantham <[email protected]>
Date: 2017-10-04T22:53:49Z
Merge branch 'trunk' of https://github.com/shivsantham/kafka into trunk
commit 084466bfd1f8bb9c9a07ec4f8255a42dfc6b8768
Author: shivsantham <[email protected]>
Date: 2017-10-04T22:59:27Z
KAFKA-5972 Flatten SMT does not work with null values
A bug in Flatten SMT while doing tests with different SMTs that are
provided out-of-box. Flatten SMT does not work as expected with schemaless JSON
that has properties with null values.
Example json:
{A={D=dValue, B=null, C=cValue}}
The issue is in if statement that checks for null value.
CURRENT VERSION:
for (Map.Entry<String, Object> entry : originalRecord.entrySet()) {
final String fieldName = fieldName(fieldNamePrefix,
entry.getKey());
Object value = entry.getValue();
if (value == null) {
newRecord.put(fieldName(fieldNamePrefix, entry.getKey()),
null);
return;
}
PROPOSED VERSION:
for (Map.Entry<String, Object> entry : originalRecord.entrySet()) {
final String fieldName = fieldName(fieldNamePrefix,
entry.getKey());
Object value = entry.getValue();
if (value == null) {
newRecord.put(fieldName(fieldNamePrefix, entry.getKey()),
null);
continue;
}
----
---
