No, thanks. I got past the issue with the Mophline stuff for this release. We will have to discuss what to do about it long term.
I have been fighting Avro/Netty for the last week upgrading those dependencies. I just have one more test to figure out and then that too will be done. Then I have to look at the hadoop family dependencies and determine what kinds of upgrades they may need. Waiting years to do releases really sucks. Ralph > On Jan 23, 2022, at 10:57 AM, Tristan Stevens <tris...@apache.org> wrote: > > Hi Ralph, > Here is the data. > > MorphlineSolrSink and MorphlineInterceptor appear 13-14% of the time in this > sample set. > > org.apache.flume.channel.kafka.KafkaChannel 64% > hdfs 38% > org.apache.flume.source.kafka.KafkaSource 33% > memory 29% > file 28% > spooldir 26% > null 25% > org.apache.flume.sink.kafka.KafkaSink 25% > Custom JMSSource 18% > jms 17% > static 14% > org.apache.flume.sink.solr.morphline.MorphlineInterceptor 14% > ElasticSearchSink 13% > org.apache.flume.sink.solr.morphline.MorphlineSolrSink 13% > host 13% > timestamp 13% > avro 11% > hbase 11% > > Let me know if you’d like to drill down further. > > Tristan > > From: Ralph Goers <ralph.go...@dslextreme.com> > Reply: dev@flume.apache.org <dev@flume.apache.org> > Date: 15 January 2022 at 06:48:43 > To: dev@flume.apache.org <dev@flume.apache.org> > Subject: Re: Morphlines-solr-sink > > I would like to see the data on the usage. I’m not sure how you would know > since Cloudera doesn’t seem to include Flume in its products any more from > what I can tell. > > The kite-morphines project consists of 18 sub-modules plug 4 aggregation > modules. That is a heck of a lot of stuff to try to drag in. I would prefer > to fork the parts of kite we would need to a new flume-kite repo. > > It seems that the CVE the reporter mentioned does have a fix. It is available > in parquet-avro 1.11.2 and 1.12.2. I was able to swap the new version for the > old one even though the groupId has changed. That said, the kite-sdk > dependency that includes it is marked as optional, so parquet-avro would be > optional as well. So I have no idea if it is even used. > > In any case, the unit tests all pass with the updated dependency. > > Ralph > > > >> On Jan 14, 2022, at 3:33 PM, Tristan Stevens <tris...@apache.org> wrote: >> >> -1 from me. >> >> First wee can’t do that in a patch release, but that’s semantics. >> >> Both the Morphlines interceptor and the Morphlines-Solr-Sink are components >> that are widely used amongst the community. I did some analysis last year >> that I’ll dig out and share, but they are two of the most used components >> after HDFS sink, Kafka and JMS. >> >> Whilst I agree it’s sucky that Cloudera aren’t supporting Kite anymore, I >> wonder whether we can find a way to bring Morphlines into here, or otherwise >> get upstream and fix the bits that need fixing. >> >> Tristan >> >> >> From: Ralph Goers <ralph.go...@dslextreme.com> >> <mailto:ralph.go...@dslextreme.com> >> Reply: dev@flume.apache.org <mailto:dev@flume.apache.org> >> <dev@flume.apache.org> <mailto:dev@flume.apache.org> >> Date: 13 January 2022 at 15:26:12 >> To: dev@flume.apache.org <dev@flume.apache.org> >> <mailto:dev@flume.apache.org> >> Subject: Morphlines-solr-sink >> >>> While I am not having any trouble building the morphline-solr-sink >>> component, it is dependent on the abandoned kite-sdk, which makes its life >>> very limited. >>> >>> In addition, the kite-sdk has a dependency on parquet-avro which, according >>> to https://issues.apache.org/jira/browse/FLUME-3403, has vulnerabilities in >>> every available release. >>> >>> Due to these factors I am going to remove the morphline-solr-sink module >>> from Flume for the 1.10.0 release. >>> >>> Ralph >
