[ https://issues.apache.org/jira/browse/NIFI-6762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16949572#comment-16949572 ]
Steve Loughran commented on NIFI-6762: -------------------------------------- How do you plan to watch? Scan? Because Amazon AWS also has that event queue, though it's only eventually consistent and can miss changes... > New Processor - Watch Multiple S3 Buckets for content changes > ------------------------------------------------------------- > > Key: NIFI-6762 > URL: https://issues.apache.org/jira/browse/NIFI-6762 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions > Affects Versions: 1.9.2 > Reporter: Aram S Openden > Priority: Major > Labels: Processor, S3, watch > Original Estimate: 384h > Remaining Estimate: 384h > > Proposing New S3 processor that is a "variation" on the existing ListS3 > Processor. For lack of better term, I will call this "Watch Multiple S3 > Buckets Processor" (naming here is open to feedback). This new Processor > takes an input attribute "Bucket" from the FlowFile, which could use NiFi > ExpressionLanguage to compute that value. Hence, the Processor could "watch" > different buckets in an S3 Object Store. > This processor must handle the State Management logic for multiple S3 > buckets. The simplest solution for this problem add to modify the State > Management logic to prepend the bucket name to this key/objectName that is > being used to track state. > Finally, the Dev should document the maximum number of buckets it will > monitor at once, maximum amount of state/storage it can have to track the > various listings in various buckets. -- This message was sent by Atlassian Jira (v8.3.4#803005)