Hi All,
We've got a NiFi 3 Node Cluster running on 3 x 40 CPU, 256GB RAM (32G Java
Heap) servers. However, we have only been able to achieve a consumption of
~9.48GB Consumption Compressed (38.53GB Uncompressed) over 5 minutes, with a
production rate of ~16.84GB out of the cluster over 5 mins.
Nathan
You have plenty powerful machines to hit super high speeds but what I
cannot tell is how the disks are setup/capability and layout wise and
relative to our three repos of importance. You'll need to share those
details.
That said, the design of the flow matters. The Kafka processors that
kafka.topic attribute doesn’t exist
at this stage? We use the Avro Schema Registry Controller as we don’t have a
schema registry in place yet.
Kind Regards,
Nathan
From: Joe Witt [mailto:joe.w...@gmail.com]
Sent: 23 September 2020 17:33
To: users@nifi.apache.org
Subject: Re: NiFi V1.9.2 Performance
hemas to topics. At the moment the ConsumeKafka processor is reading
> from all topics in one consumer. I’m assuming the attribute kafka.topic
> attribute doesn’t exist at this stage? We use the Avro Schema Registry
> Controller as we don’t have a schema registry in place yet.
>
>
>
> find information (Apart from your great videos) to assist in getting to the
> bottom of it.
>
>
>
> Kind Regards,
>
>
>
> Nathan
>
>
>
> *From:* Mark Payne [mailto:marka...@hotmail.com]
> *Sent:* 24 September 2020 15:12
> *To:* users@nifi.apache.org
"Field3":
> "[&(1)].OutputField2",
>
> "Sub": {
>
> "0": {
>
> "SubField1": "[&(3)].OutputField4",
>
>
his would be to have a separate processor and
> AvroReader controller service for each topic?
>
>
>
> Kind Regards,
>
>
>
> Nathan
>
>
>
>
>
> *From:* Matt Burgess [mailto:mattyb...@apache.org]
> *Sent:* 24 September 2020 17:08
> *To:* users@nifi.apache.org
&
or steps taken
> to improve the throughput.
>
>
>
> Thanks for all the assistance!
>
>
>
> Nathan
>
>
>
> [1] -
> https://community.cloudera.com/t5/Support-Questions/JoltTransformJSON-performance-issue-when-using-default/td-p/226827
>
>
>
> *Fr
