Re: Detect duplicate record reader

2021-02-15 Thread Jeremy Pemberton-Pigott
Thanks for the replies guys. Yes, NIFI-6047 is pretty much exactly what I'm looking for. Jeremy On Mon, Feb 15, 2021 at 2:37 PM Chris Sampson wrote: > NIFI-6047 [1] is possibly what you're after, but that won't help you just > now because it appears to remain unfinished. > > > [1]

Re: Detect duplicate record reader

2021-02-14 Thread Chris Sampson
NIFI-6047 [1] is possibly what you're after, but that won't help you just now because it appears to remain unfinished. [1] https://issues.apache.org/jira/browse/NIFI-6047 Cheers, Chris Sampson On Mon, 15 Feb 2021, 06:27 Jorge Machado, wrote: > Hey Jeremy, > > Something linke this >

Re: Detect duplicate record reader

2021-02-14 Thread Jorge Machado
Hey Jeremy, Something linke this https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.DetectDuplicate/index.html

Detect duplicate record reader

2021-02-14 Thread Jeremy Pemberton-Pigott
Hi everyone, I'm wondering if there is a Detect Duplicate processor that can read records from a flow file and as output gives just the non-duplicates (can be single records or a group of non-duplicates would be better). I want to use a record reader to avoid splitting the json content into