[ https://issues.apache.org/jira/browse/NIFI-3644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bryan Bende resolved NIFI-3644. ------------------------------- Resolution: Fixed Assignee: Bryan Bende Merged to master. > Add DetectDuplicateUsingHBase processor > --------------------------------------- > > Key: NIFI-3644 > URL: https://issues.apache.org/jira/browse/NIFI-3644 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions > Reporter: Bjorn Olsen > Assignee: Bryan Bende > Priority: Minor > Fix For: 1.3.0 > > > The DetectDuplicate processor makes use of a distributed map cache for > maintaining a list of unique file identifiers (such as hashes). > The distributed map cache functionality could be provided by an HBase table, > which then allows for reliably storing a huge volume of file identifiers and > auditing information. The downside of this approach is of course that HBase > is required. > Storing the unique file identifiers in a reliable, query-able manner along > with some audit information is of benefit to several use cases. -- This message was sent by Atlassian JIRA (v6.3.15#6346)