Issue with empty files in content repository

2019-09-13 Thread Malthe
Trying to figure out why a `MergeContent` processor was producing a linearly rising amount of content which wasn't reaped correctly (the retention policies would not be upheld and disk space would fall to zero), we realized that some flow files in the queue pointed to content which didn't exist on

Re: Using ConvertRecord on compressed input

2019-07-29 Thread Malthe
l > Adding compressed input support at RecordReaderFactory is my suggestion. > > Thanks, > Koji > > On Sun, Jul 28, 2019 at 6:04 PM Malthe wrote: > > > > CSV, JSON, and XML files all compress well and thus you often see them > > stored in a compressed forma

Using ConvertRecord on compressed input

2019-07-28 Thread Malthe
CSV, JSON, and XML files all compress well and thus you often see them stored in a compressed format. Actually, it's not feasible or simply practical to first decompress the files and then process them into a binary format. How would I go about extending or modifying the `AbstractRecordProcessor`