Corey Shaw created NIFI-10077:
---------------------------------
Summary: ConsumeKafkaRecord_2_6 not consuming data when using
patterns for topic names
Key: NIFI-10077
URL: https://issues.apache.org/jira/browse/NIFI-10077
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Affects Versions: 1.16.2
Environment: CentOS 7.9
OpenJDK 11.0.10
Reporter: Corey Shaw
I've been seeing some odd behavior with the ConsumeKafkaRecord_2_6 processor
(on NiFi 1.16.2). I have been testing out using "Topic Name Format" set to
"pattern". I've found that no matter what I set for the "Topic Name(s)" value,
the processor never actually starts consuming data. It will properly talk to
Kafka and join the necessary consumer group, but data is never read.
I have confirmed in both the NiFi logs and Kafka itself (using the
"kafka-consumer-groups" CLI tool) that the processor has in fact connected and
joined the consumer group. If I instead use a "Topic Name Format" set to
"names" (and "Topic Name(s)" changed accordingly to a comma-delimited list),
then everything functions perfectly (data is consumed as it should be).
h2. Example One (Works)
"Topic Name Format" set to "names" and "Topic Name(s)" set to "topic1,topic2" :
* Processor connects to Kafka fine
* Processor joins the consumer group fine and shows as consuming from the
expected topics
* Processor consumes data from both topics as expected
h2. Example Two (Works)
"Topic Name Format" set to "names" and "Topic Name(s)" set to "topic1":
* Processor connects to Kafka fine
* Processor joins the consumer group fine and shows as consuming from the
expected topic
* Processor consumes data from the topic as expected
h2. Example Three (Fails)
"Topic Name Format" set to "pattern" and "Topic Name(s)" set to "topic\d":
* Processor connects to Kafka fine
* Processor joins the consumer group fine and shows as consuming from the
expected topics
* No data is consumed by the processor
h2. Example Four (Fails)
"Topic Name Format" set to "pattern" and "Topic Name(s)" set to "topic1":
* Processor connects to Kafka fine
* Processor joins the consumer group fine and shows as consuming from the
expected topics
* No data is consumed by the processor
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
