PartitionRecord might be what you're after. This will allow you to analyse fields and separate records from flowfiles into chunks containing the same field values and those values will be added as flowfile attributes.
https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.13.2/org.apache.nifi.processors.standard.PartitionRecord/index.html Cheers, Chris Sampson On Tue, 11 May 2021, 23:06 Richard Beare, <[email protected]> wrote: > Hi, > Warning about a likely newbie question. > > I'm extracting records from an SQL DB that include a blob that I need > to feed through some custom groovy/java. I have the basic version > working using avro records throughout. However there is a complexity > in that a small proportion of blobs span multiple rows and require > concatenation before processing. Thus I need to ensure that all the > blobs that belong together get assembled into the same flowfile before > performing the concatenation (probably using a custom groovy script > because there are some suffixes to remove before concatenation). > > I've added a "PARTS" field via my initial SQL query and there is also > a sequence number column and an ID column. My plan was to use > RecordPartition based on the ID, modify fragment ID and count and then > reassemble with mergerecord using the defragment strategy. > > My problem is that I can't figure out how to get record fields into > attributes. I'm hoping there is a recordpath/expression combination > allowing this. Any suggestions? > > My fallback is to separate the initial sql queries into two parts, one > for batches of single row blobs and a second that collects the > multi-row ones one at a time. >
