[
https://issues.apache.org/jira/browse/NIFI-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17376449#comment-17376449
]
Jul Tomten edited comment on NIFI-3431 at 7/7/21, 9:52 AM:
-----------------------------------------------------------
I'm on NiFi 1.13.2
I have a WAIT processor and 2000 files queud up in the upstream connection
The problem is that the WAIT processor uses more than 100% CPU all the time as
long as the messages are on queue.
Any idea about the problem?
Release Signal Identifier = ${uuid}
Target Signal Count = 1
Signal Counter Name = No value set
Wait Buffer Count = 1500
Releasable FlowFile Count = 1
Expiration Duration = 10 min
Attribute Copy Mode = Keep original
Wait Mode = Keep in the upstream connection
Wait Penalty Duration = 5 min
Number of Threads 10
I'd planned to have more than 10 000 000 messages on queue and then let them
expire after 4 days. Some or all messages may be notified to proceed. Is the
WAIT intended to handel such a scenario?
Are you supposed to have multiple threads on the WAIT/NOTIFY processors? Seems
like the MapCache hits problems ConcurrentModificationException.
was (Author: tomten1970):
I'm on NiFi 1.13.2
I have a WAIT processor and 2000 files queud up in the upstream connection
The problem is that the WAIT processor uses more than 100% CPU all the time as
long as the messages are on queue.
Any idea about the problem?
Release Signal Identifier = ${uuid}
Target Signal Count = 1
Signal Counter Name = No value set
Wait Buffer Count = 1500
Releasable FlowFile Count = 1
Expiration Duration = 10 min
Attribute Copy Mode = Keep original
Wait Mode = Keep in the upstream connection
Wait Penalty Duration = 5 min
I'd planned to have more than 10 000 000 messages on queue and then let them
expire after 4 days. Some or all messages may be notified to proceed. Is the
WAIT intended to handel such a scenario?
> Support batch update in Notify processor
> ----------------------------------------
>
> Key: NIFI-3431
> URL: https://issues.apache.org/jira/browse/NIFI-3431
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Affects Versions: 1.2.0
> Reporter: Koji Kawamura
> Assignee: Koji Kawamura
> Priority: Major
> Fix For: 1.2.0
>
>
> NIFI-3216 added ability to wait for N signals. It supports waiting for N
> fragments split by SplitXXXX processors. However, since Notify processor has
> to increase count one by one by calling expensive replace cache operation
> over network, it doesn't provide a practical performance when user configured
> a flow looks like below as N glow:
> {code}
> Split
> --> (original) -> Wait for N
> --> (split) -> Do something -> Notify 1
> {code}
> This JIRA improves Notify processor by:
> - Add "Signal Buffer Count" to specify max number of flow files that can be
> buffered and update cache at once
> - Add "Signal Counter Delta" to specify delta grater than 1, EL supported
> "Signal Buffer Count" would be useful in a flow like this:
> {code}
> Split
> --> (original) -> Wait for N
> --> (split) -> Filter or something -> Notify M
> {code}
> Buffer incoming M flow files and perform cache replace operation once.
> So does "Signal Counter Delta":
> {code}
> Split
> --> (original) -> Wait for N
> --> (split) -> Filter or something -> Merge M -> PutXXX -> Notify M
> {code}
> Specify 'M' via Attribute Expression Language.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
