[ https://issues.apache.org/jira/browse/NIFI-6762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16948046#comment-16948046 ]
Aram S Openden commented on NIFI-6762: -------------------------------------- Please note: In the course of my work for my project team at IBM Watson Health, _I've done almost all of the work for this Processor already_ in a segregated proprietary project for our team. My plan is to port that code over to the NiFi main source project in the {{nifi-nar-bundles}} sub-project, making a small set of changes/refactorings (mostly around common code factored out) on a feature branch and then submit my PR for Review to the NiFi team. --Aram > New Processor - Watch Multiple S3 Buckets for content changes > ------------------------------------------------------------- > > Key: NIFI-6762 > URL: https://issues.apache.org/jira/browse/NIFI-6762 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions > Affects Versions: 1.9.2 > Reporter: Aram S Openden > Priority: Major > Labels: Processor, S3, watch > Original Estimate: 384h > Remaining Estimate: 384h > > Proposing New S3 processor that is a "variation" on the existing ListS3 > Processor. For lack of better term, I will call this "Watch Multiple S3 > Buckets Processor" (naming here is open to feedback). This new Processor > takes an input attribute "Bucket" from the FlowFile, which could use NiFi > ExpressionLanguage to compute that value. Hence, the Processor could "watch" > different buckets in an S3 Object Store. > This processor must handle the State Management logic for multiple S3 > buckets. The simplest solution for this problem add to modify the State > Management logic to prepend the bucket name to this key/objectName that is > being used to track state. > Finally, the Dev should document the maximum number of buckets it will > monitor at once, maximum amount of state/storage it can have to track the > various listings in various buckets. -- This message was sent by Atlassian Jira (v8.3.4#803005)