[
https://issues.apache.org/jira/browse/NIFI-1669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15207655#comment-15207655
]
ASF subversion and git services commented on NIFI-1669:
-------------------------------------------------------
Commit f120952ab79d844b92c9d7e1014068065250635f in nifi's branch
refs/heads/master from [~markap14]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f120952 ]
NIFI-1669: Ensure that we take into account the number of 'ready bins' when
determining whether or not we have reached out Max Bin threshold for BinFiles
Signed-off-by: Aldrin Piri <ald...@apache.org>
> MergeContent does not always honor the 'Maximum number of Bins' property
> ------------------------------------------------------------------------
>
> Key: NIFI-1669
> URL: https://issues.apache.org/jira/browse/NIFI-1669
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Reporter: Mark Payne
> Assignee: Mark Payne
> Priority: Critical
> Fix For: 0.6.0
>
> Attachments:
> 0001-NIFI-1669-Ensure-that-we-take-into-account-the-numbe.patch,
> 0001-NIFI-1669-Ensure-that-we-take-into-account-the-numbe.patch
>
>
> MergeContent exposes a "Maximum number of Bins" property. However, the way in
> which this property is used limits only the maximum number of 'in progress'
> bins. Once a bin has been declared 'ready' (i.e., the FlowFiles have been
> grouped together and are just waiting to be written out), that 'ready bin' is
> not counted toward the maximum number of allowable bins.
> As a result, if we have a very high volume of FlowFiles coming in, we could
> end up with MergeContent creating many bins and buffering potentially
> millions of FlowFiles, exhausting Java Heap and preventing backpressure and
> age-off from kicking in.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
