[ https://issues.apache.org/jira/browse/APEXMALHAR-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15303124#comment-15303124 ]
ASF GitHub Bot commented on APEXMALHAR-2096: -------------------------------------------- Github user chandnisingh commented on a diff in the pull request: https://github.com/apache/incubator-apex-malhar/pull/287#discussion_r64835548 --- Diff: library/src/main/java/com/datatorrent/lib/io/fs/AbstractFileSplitter.java --- @@ -53,6 +53,7 @@ * This is a threshold on the no. of blocks emitted per window. A lot of blocks emitted * per window can overwhelm the downstream operators. This setting helps to control that. */ + @NotNull --- End diff -- @DT-Priyanka It is of type primitive int. So default value will be 0. Since the min is 1, it will be necessary for the user to set it. @NotNull is not relevant here. > Add blockThreshold parameter to FSInputModule > --------------------------------------------- > > Key: APEXMALHAR-2096 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2096 > Project: Apache Apex Malhar > Issue Type: Improvement > Reporter: Priyanka Gugale > Assignee: Priyanka Gugale > > FileSplitter is very fast, the downstream operators can't work at that speed, > so we need to limit the speed with which fileSplitter works. > One way to do is limit number of blocks emitted per window. -- This message was sent by Atlassian JIRA (v6.3.4#6332)