[
https://issues.apache.org/jira/browse/APEXMALHAR-2081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15276988#comment-15276988
]
ASF GitHub Bot commented on APEXMALHAR-2081:
--------------------------------------------
Github user DT-Priyanka commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/267#discussion_r62568807
--- Diff:
library/src/main/java/com/datatorrent/lib/io/fs/FileSplitterInput.java ---
@@ -266,6 +266,7 @@ public TimeBasedDirectoryScanner getScanner()
public static class TimeBasedDirectoryScanner implements Runnable,
Component<Context.OperatorContext>
{
private static long DEF_SCAN_INTERVAL_MILLIS = 5000;
+ private static String FILE_BEING_COPIED = "_COPYING_";
--- End diff --
Couple of questions:
1. Why do we have it as independent field and then check it as extension?
We can put it as default in ignore regex part right?
2. This is generic class applicable to file systems other than HDFS, it is
safe to ignore these files on other file systems? This file may not mean same
on other file systems.
> Remove FSFileSplitter, BlockReader, HDFSFileSplitter, HDFSInputModule
> ---------------------------------------------------------------------
>
> Key: APEXMALHAR-2081
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2081
> Project: Apache Apex Malhar
> Issue Type: Bug
> Components: adapters other
> Reporter: Chandni Singh
> Assignee: Chandni Singh
> Priority: Critical
> Fix For: 3.4.0
>
>
> Recently there was an FSFileSplitter added to the library.
> This operator adds 2 additional properties which can be added to the base
> operator.
> A problem with this approach of adding new extension which just add small
> features is that it confuses the users of which Operator to use and
> essentially we have multiple operators with the same basic functionality.
> This should be done before the new version of Malhar is released.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
