I’ve not gotten any replies to this so I am assuming everyone is ok with my going with option 2.
Ralph > On Mar 29, 2022, at 1:05 AM, Ralph Goers <ralph.go...@dslextreme.com> wrote: > > I think addressing Elasticsearch may be the last major hurdle before 1.10.0 > can be released. > > Flume currently uses Elasticsearch 0.90.1. > https://www.elastic.co/support/eol#maintenance-tables shows that version > 1.0.x reached end of life in 2015 so obviously the version Flume is using has > been unsupported from even before that. > > The major goal of the 1.10.0 release has been to upgrade all the dependencies > so that we aren’t using any that have known security vulnerabilities. From > looking at https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=elasticsearch I > am quite sure there is at least one CVE that applies to 0.90.1. > > Furthermore, this release is so old that I find it hard to imagine that > anyone could still be using it. > > In addition, flume-ng-elasticsearch-sink is including > org.elasticsearch:elasticsearch as an optional dependency. I suspect at the > time that there was no Java client. We really should have a required > dependency on the Java client and the Elasticsearch dependency should be a > test dependency, not optional. > > I see 4 choices here for the 1.10.0 release: > > 1. Do nothing. This is not ideal since the component is practically useless > as it exists and has security vulnerabilities. > 2. Drop the module in 1.10.0. This is obviously not backward compatible but > any upgrade is going to break compatibility. We can defer re-implementing it > until 2.0. > 3. Upgrade to a supported version of the Java client (which I believe has a > license that is compatible with the ASF). Again, this is not backward > compatible. It would need to use ElasticSearch for testing. The latest > versions of ElasticSearch use a license which is Category X so we will need > to include something in the NOTICE file and in the user’s guide warning about > the ElasticSearch license if the component is used. > 4. Upgrade to the latest version of > https://opensearch.org/docs/latest/clients/java/. This would use OpenSearch > instead of Elasticsearch and AFAIK would be incompatible with Elasticsearch. > > Of these options, my recommendation is to go with option 2. Once we > modularize things in 2.0 we can implement support for both Elasticsearch and > OpenSearch. We could also support Solr if desired. > > Thoughts? > > Ralph
