I think addressing Elasticsearch may be the last major hurdle before 1.10.0 can be released.
Flume currently uses Elasticsearch 0.90.1. https://www.elastic.co/support/eol#maintenance-tables shows that version 1.0.x reached end of life in 2015 so obviously the version Flume is using has been unsupported from even before that. The major goal of the 1.10.0 release has been to upgrade all the dependencies so that we aren’t using any that have known security vulnerabilities. From looking at https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=elasticsearch I am quite sure there is at least one CVE that applies to 0.90.1. Furthermore, this release is so old that I find it hard to imagine that anyone could still be using it. In addition, flume-ng-elasticsearch-sink is including org.elasticsearch:elasticsearch as an optional dependency. I suspect at the time that there was no Java client. We really should have a required dependency on the Java client and the Elasticsearch dependency should be a test dependency, not optional. I see 4 choices here for the 1.10.0 release: 1. Do nothing. This is not ideal since the component is practically useless as it exists and has security vulnerabilities. 2. Drop the module in 1.10.0. This is obviously not backward compatible but any upgrade is going to break compatibility. We can defer re-implementing it until 2.0. 3. Upgrade to a supported version of the Java client (which I believe has a license that is compatible with the ASF). Again, this is not backward compatible. It would need to use ElasticSearch for testing. The latest versions of ElasticSearch use a license which is Category X so we will need to include something in the NOTICE file and in the user’s guide warning about the ElasticSearch license if the component is used. 4. Upgrade to the latest version of https://opensearch.org/docs/latest/clients/java/. This would use OpenSearch instead of Elasticsearch and AFAIK would be incompatible with Elasticsearch. Of these options, my recommendation is to go with option 2. Once we modularize things in 2.0 we can implement support for both Elasticsearch and OpenSearch. We could also support Solr if desired. Thoughts? Ralph
