[
https://issues.apache.org/jira/browse/APEXMALHAR-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Weise moved APEXCORE-343 to APEXMALHAR-2021:
---------------------------------------------------
Affects Version/s: (was: 3.3.0)
3.3.0
Workflow: Default workflow, editable Closed status (was: jira)
Key: APEXMALHAR-2021 (was: APEXCORE-343)
Project: Apache Apex Malhar (was: Apache Apex Core)
> Add property to AbstractFileInputOperator to trim processedFiles and
> ignoredFiles
> ---------------------------------------------------------------------------------
>
> Key: APEXMALHAR-2021
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2021
> Project: Apache Apex Malhar
> Issue Type: Improvement
> Affects Versions: 3.3.0
> Reporter: Munagala V. Ramanath
> Assignee: Munagala V. Ramanath
>
> In AbstractFileInputOperator, the processedFiles and
> DirectoryScanner.ignoredFiles sets can continue to grow without bound as new
> files are added to the monitored directory. There are scenarios where an
> input file is deleted by the application once it is processed; here, it is
> useful to provide a property that, if true, will cause the scanner to remove
> deleted files from these sets to prevent unbounded growth.
> The only way to do this currently is to extend DirectoryScanner and provide
> implementations of the scan() and createPartition() methods which forces the
> operator writer to unnecessarily grapple with the internals of the base class.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
