Sorry for the late reply. Can we make our next release a major if we're making incompatible changes?
Donat On Wed, Mar 30, 2022 at 11:06 PM Ralph Goers <ralph.go...@dslextreme.com> wrote: > > I’ve not gotten any replies to this so I am assuming everyone is ok with my > going with option 2. > > Ralph > > > On Mar 29, 2022, at 1:05 AM, Ralph Goers <ralph.go...@dslextreme.com> wrote: > > > > I think addressing Elasticsearch may be the last major hurdle before 1.10.0 > > can be released. > > > > Flume currently uses Elasticsearch 0.90.1. > > https://www.elastic.co/support/eol#maintenance-tables shows that version > > 1.0.x reached end of life in 2015 so obviously the version Flume is using > > has been unsupported from even before that. > > > > The major goal of the 1.10.0 release has been to upgrade all the > > dependencies so that we aren’t using any that have known security > > vulnerabilities. From looking at > > https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=elasticsearch I am quite > > sure there is at least one CVE that applies to 0.90.1. > > > > Furthermore, this release is so old that I find it hard to imagine that > > anyone could still be using it. > > > > In addition, flume-ng-elasticsearch-sink is including > > org.elasticsearch:elasticsearch as an optional dependency. I suspect at the > > time that there was no Java client. We really should have a required > > dependency on the Java client and the Elasticsearch dependency should be a > > test dependency, not optional. > > > > I see 4 choices here for the 1.10.0 release: > > > > 1. Do nothing. This is not ideal since the component is practically useless > > as it exists and has security vulnerabilities. > > 2. Drop the module in 1.10.0. This is obviously not backward compatible but > > any upgrade is going to break compatibility. We can defer re-implementing > > it until 2.0. > > 3. Upgrade to a supported version of the Java client (which I believe has a > > license that is compatible with the ASF). Again, this is not backward > > compatible. It would need to use ElasticSearch for testing. The latest > > versions of ElasticSearch use a license which is Category X so we will need > > to include something in the NOTICE file and in the user’s guide warning > > about the ElasticSearch license if the component is used. > > 4. Upgrade to the latest version of > > https://opensearch.org/docs/latest/clients/java/. This would use OpenSearch > > instead of Elasticsearch and AFAIK would be incompatible with Elasticsearch. > > > > Of these options, my recommendation is to go with option 2. Once we > > modularize things in 2.0 we can implement support for both Elasticsearch > > and OpenSearch. We could also support Solr if desired. > > > > Thoughts? > > > > Ralph >
