Hello I think the comments around CompressContent are valid and fixable and your ideas there make a lot of sense.
As you noted in the second part various components have their own built in compression. These are likely a bridge too far as their config/options/etc are well embedded into those libraries. Thanks On Thu, Nov 24, 2022 at 6:36 PM Matthew Hawkins <hawko2...@gmail.com> wrote: > Hi team, > > Whilst giving a first go at adding brotli and zstd to CompressContent I > noticed a few things that I think we should address. > > 1. compression levels are hard-coded as 0-9 in the UI, this prevents > accessing every level possible under certain compressors, e.g. zstd has 22 > levels, brotli has 15. I hacked it, it's ugly, we should do the "right" > thing instead. > - I think this should be addressed by having a compressor register itself > along with metadata like how many levels it supports to a factory & the UI > utilise that property in a more dynamic fashion > - we could also use metadata to let a compressor flag known high cpu / > high mem situations so the flow Admin can avoid silly mistakes in config or > the JVM can be hinted at. > > 2. Some other processors such as PublishKafka, InvokeHTTP, > JsonRecordSetWriter, and others implement some compression algorithms > directly. > - This made me realise it would IMO be better offered as a Service and > each of these use cases can have the simple API of using the controller > service rather than duplicating compression algorithm specific code each > time the functionality is required. > > Am I on the right track? Java isn't my forte and I'm new to contributing to > NiFi so keen to hear what experienced devs think. > > -- > Kind regards, > Matthew Hawkins >
