[
https://issues.apache.org/jira/browse/NIFI-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17509905#comment-17509905
]
Josef Zahner commented on NIFI-9820:
------------------------------------
It's really a big advantage that we can change the value with a property since
1.14.0, however I could imagine that the internal default value from the Kudu
Client library doesn't expect to have multiple clients (aka processors in nifi)
on one node. So it makes sense to use the number of CPUs there. In NiFi however
it's a different case, it's very likely that you don't have just one client.
The question is, why should we use a "dynamic" calculated value as any other
property (eg. FlowFiles per Batch) is as well just on a fixed value. The use
has to test/find anyway a good value in his setup.
> Change PutKudu Property "Kudu Client Worker Count" Default Value
> ----------------------------------------------------------------
>
> Key: NIFI-9820
> URL: https://issues.apache.org/jira/browse/NIFI-9820
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Affects Versions: 1.15.3
> Reporter: Josef Zahner
> Assignee: David Handermann
> Priority: Minor
>
> The PutKudu processor property "Kudu Client Worker Count" has a suboptimal
> value. Please don't use the current "number of CPUs multiplied by 2"
> behaviour as it leads to a massive amount of workers in our case with
> physical servers. We have a 8-node cluster where each server has 64 CPUs. We
> have about 30 PutKudu processors configured -> a lot of worker threads per
> default just for kudu.
> We have changed the number of worker threads in our case to the number of
> concurrent tasks. I don't know, maybe it would be great to set it a bit
> higher than that, but to be honest, I don't exactly understand the impact. It
> looks still fast with the current config.
> *To sum it up, please set a low default value (eg. 4 or 8) for the property
> "Kudu Client Worker Count" and not a pseudo dynamic one for the PutKudu
> processor.*
> Btw. are there any suggestions how big the number should be?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
