[jira] [Updated] (NIFI-4008) ConsumeKafkaRecord_0_10 assumes there is always one Record in a message
[
https://issues.apache.org/jira/browse/NIFI-4008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph Witt updated NIFI-4008:
--
Fix Version/s: 1.5.0
> ConsumeKafkaRecord_0_10 assumes there is always one Record in a message
> ---
>
> Key: NIFI-4008
> URL: https://issues.apache.org/jira/browse/NIFI-4008
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.2.0
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
> Fix For: 1.5.0
>
>
> ConsumeKafkaRecord_0_10 uses ConsumerLease underneath, and it [assumes there
> is one Record available in a consumed
> message|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-0-10-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/ConsumerLease.java#L434]
> retrieved from a Kafka topic.
> But in fact, a message can contain 0 or more records in it. For example, with
> a record schema shown below:
> {code}
> {
> "type": "record",
> "name": "temp",
> "fields" : [
> {"name": "value", "type": "string"}
> ]
> }
> {code}
> Multiple records can be sent within a single message, e.g. using JSON:
> {code}
> [{"value": "a"}, {"value": "b"}, {"value": "c"}]
> {code}
> But ConsumeKafkaRecord only outputs the first record:
> {code}
> [{"value": "a"}]
> {code}
> Also, if a message doesn't contain any record in it, the processor fails with
> NullPointerException.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (NIFI-4008) ConsumeKafkaRecord_0_10 assumes there is always one Record in a message
[
https://issues.apache.org/jira/browse/NIFI-4008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph Witt updated NIFI-4008:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
> ConsumeKafkaRecord_0_10 assumes there is always one Record in a message
> ---
>
> Key: NIFI-4008
> URL: https://issues.apache.org/jira/browse/NIFI-4008
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.2.0
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
>
> ConsumeKafkaRecord_0_10 uses ConsumerLease underneath, and it [assumes there
> is one Record available in a consumed
> message|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-0-10-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/ConsumerLease.java#L434]
> retrieved from a Kafka topic.
> But in fact, a message can contain 0 or more records in it. For example, with
> a record schema shown below:
> {code}
> {
> "type": "record",
> "name": "temp",
> "fields" : [
> {"name": "value", "type": "string"}
> ]
> }
> {code}
> Multiple records can be sent within a single message, e.g. using JSON:
> {code}
> [{"value": "a"}, {"value": "b"}, {"value": "c"}]
> {code}
> But ConsumeKafkaRecord only outputs the first record:
> {code}
> [{"value": "a"}]
> {code}
> Also, if a message doesn't contain any record in it, the processor fails with
> NullPointerException.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (NIFI-4008) ConsumeKafkaRecord_0_10 assumes there is always one Record in a message
[
https://issues.apache.org/jira/browse/NIFI-4008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-4008:
-
Status: Patch Available (was: Reopened)
> ConsumeKafkaRecord_0_10 assumes there is always one Record in a message
> ---
>
> Key: NIFI-4008
> URL: https://issues.apache.org/jira/browse/NIFI-4008
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.2.0
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
>
> ConsumeKafkaRecord_0_10 uses ConsumerLease underneath, and it [assumes there
> is one Record available in a consumed
> message|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-0-10-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/ConsumerLease.java#L434]
> retrieved from a Kafka topic.
> But in fact, a message can contain 0 or more records in it. For example, with
> a record schema shown below:
> {code}
> {
> "type": "record",
> "name": "temp",
> "fields" : [
> {"name": "value", "type": "string"}
> ]
> }
> {code}
> Multiple records can be sent within a single message, e.g. using JSON:
> {code}
> [{"value": "a"}, {"value": "b"}, {"value": "c"}]
> {code}
> But ConsumeKafkaRecord only outputs the first record:
> {code}
> [{"value": "a"}]
> {code}
> Also, if a message doesn't contain any record in it, the processor fails with
> NullPointerException.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (NIFI-4008) ConsumeKafkaRecord_0_10 assumes there is always one Record in a message
[
https://issues.apache.org/jira/browse/NIFI-4008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-4008:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
> ConsumeKafkaRecord_0_10 assumes there is always one Record in a message
> ---
>
> Key: NIFI-4008
> URL: https://issues.apache.org/jira/browse/NIFI-4008
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.2.0
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
>
> ConsumeKafkaRecord_0_10 uses ConsumerLease underneath, and it [assumes there
> is one Record available in a consumed
> message|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-0-10-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/ConsumerLease.java#L434]
> retrieved from a Kafka topic.
> But in fact, a message can contain 0 or more records in it. For example, with
> a record schema shown below:
> {code}
> {
> "type": "record",
> "name": "temp",
> "fields" : [
> {"name": "value", "type": "string"}
> ]
> }
> {code}
> Multiple records can be sent within a single message, e.g. using JSON:
> {code}
> [{"value": "a"}, {"value": "b"}, {"value": "c"}]
> {code}
> But ConsumeKafkaRecord only outputs the first record:
> {code}
> [{"value": "a"}]
> {code}
> Also, if a message doesn't contain any record in it, the processor fails with
> NullPointerException.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (NIFI-4008) ConsumeKafkaRecord_0_10 assumes there is always one Record in a message
[
https://issues.apache.org/jira/browse/NIFI-4008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Koji Kawamura updated NIFI-4008:
Status: Patch Available (was: Open)
Thanks for the comments [~joewitt] [~markap14], I have submitted a PR for this
JIRA. I agree with you as we expect a message contains a record, but I also
think if ConsumeKafkaRecord can extract records from a message, it also would
be useful. So, I tried to support both ways in my PR. Let's keep discussing on
Github PR.
> ConsumeKafkaRecord_0_10 assumes there is always one Record in a message
> ---
>
> Key: NIFI-4008
> URL: https://issues.apache.org/jira/browse/NIFI-4008
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.2.0
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
>
> ConsumeKafkaRecord_0_10 uses ConsumerLease underneath, and it [assumes there
> is one Record available in a consumed
> message|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-0-10-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/ConsumerLease.java#L434]
> retrieved from a Kafka topic.
> But in fact, a message can contain 0 or more records in it. For example, with
> a record schema shown below:
> {code}
> {
> "type": "record",
> "name": "temp",
> "fields" : [
> {"name": "value", "type": "string"}
> ]
> }
> {code}
> Multiple records can be sent within a single message, e.g. using JSON:
> {code}
> [{"value": "a"}, {"value": "b"}, {"value": "c"}]
> {code}
> But ConsumeKafkaRecord only outputs the first record:
> {code}
> [{"value": "a"}]
> {code}
> Also, if a message doesn't contain any record in it, the processor fails with
> NullPointerException.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (NIFI-4008) ConsumeKafkaRecord_0_10 assumes there is always one Record in a message
[
https://issues.apache.org/jira/browse/NIFI-4008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Koji Kawamura updated NIFI-4008:
Description:
ConsumeKafkaRecord_0_10 uses ConsumerLease underneath, and it [assumes there is
one Record available in a consumed
message|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-0-10-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/ConsumerLease.java#L434]
retrieved from a Kafka topic.
But in fact, a message can contain 0 or more records in it. For example, with a
record schema shown below:
{code}
{
"type": "record",
"name": "temp",
"fields" : [
{"name": "value", "type": "string"}
]
}
{code}
Multiple records can be sent within a single message, e.g. using JSON:
{code}
[{"value": "a"}, {"value": "b"}, {"value": "c"}]
{code}
But ConsumeKafkaRecord only outputs the first record:
{code}
[{"value": "a"}]
{code}
Also, if a message doesn't contain any record in it, the processor fails with
NullPointerException.
was:
ConsumeKafkaRecord_0_10 uses ConsumerLease underneath, and it [assumes there is
one Record available in a consumed
message|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-0-10-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/ConsumerLease.java#L434]
retrieved from a Kafka topic.
But in fact, a message can contain 0 or more records in it. For example, with a
record schema shown below:
{code}
{
"type": "record",
"name": "temp",
"fields" : [
{"name": "value", "type": "string"}
]
}
{code}
Multiple records can be sent within a single message, e.g. using JSON:
{code}
[{"value": "a"}, {"value": "b"}, {"value": "c"}]
{code}
But ConsumeKafkaRecord only outputs the first record '[{"value": "a"}]'.
Also, if a message doesn't contain any record in it, the processor fails with
NullPointerException.
> ConsumeKafkaRecord_0_10 assumes there is always one Record in a message
> ---
>
> Key: NIFI-4008
> URL: https://issues.apache.org/jira/browse/NIFI-4008
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.2.0
>Reporter: Koji Kawamura
>
> ConsumeKafkaRecord_0_10 uses ConsumerLease underneath, and it [assumes there
> is one Record available in a consumed
> message|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-0-10-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/ConsumerLease.java#L434]
> retrieved from a Kafka topic.
> But in fact, a message can contain 0 or more records in it. For example, with
> a record schema shown below:
> {code}
> {
> "type": "record",
> "name": "temp",
> "fields" : [
> {"name": "value", "type": "string"}
> ]
> }
> {code}
> Multiple records can be sent within a single message, e.g. using JSON:
> {code}
> [{"value": "a"}, {"value": "b"}, {"value": "c"}]
> {code}
> But ConsumeKafkaRecord only outputs the first record:
> {code}
> [{"value": "a"}]
> {code}
> Also, if a message doesn't contain any record in it, the processor fails with
> NullPointerException.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
