I’d really like to get 1.10.0 released. Doing a 2.0 is going to be a much larger effort. I would argue that no matter what we do the elastic search support is incompatible, even if we just release what is currently there. But if the choice is between releasing 1.10.0 with the current elastic search support (knowing full well it is unusable) or having to do a 2.0.0 release I will leave it in. It has been over 3 years since the last release and it is quite apparent that a bunch of stuff that should have been updated then wasn’t.
So please let me know whether 1.10.0 should contain the existing elastic search support or not. Ralph > On Mar 31, 2022, at 12:13 AM, Bessenyei Balázs Donát <bes...@apache.org> > wrote: > > Sorry for the late reply. > Can we make our next release a major if we're making incompatible changes? > > > Donat > > > On Wed, Mar 30, 2022 at 11:06 PM Ralph Goers <ralph.go...@dslextreme.com> > wrote: >> >> I’ve not gotten any replies to this so I am assuming everyone is ok with my >> going with option 2. >> >> Ralph >> >>> On Mar 29, 2022, at 1:05 AM, Ralph Goers <ralph.go...@dslextreme.com> wrote: >>> >>> I think addressing Elasticsearch may be the last major hurdle before 1.10.0 >>> can be released. >>> >>> Flume currently uses Elasticsearch 0.90.1. >>> https://www.elastic.co/support/eol#maintenance-tables shows that version >>> 1.0.x reached end of life in 2015 so obviously the version Flume is using >>> has been unsupported from even before that. >>> >>> The major goal of the 1.10.0 release has been to upgrade all the >>> dependencies so that we aren’t using any that have known security >>> vulnerabilities. From looking at >>> https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=elasticsearch I am quite >>> sure there is at least one CVE that applies to 0.90.1. >>> >>> Furthermore, this release is so old that I find it hard to imagine that >>> anyone could still be using it. >>> >>> In addition, flume-ng-elasticsearch-sink is including >>> org.elasticsearch:elasticsearch as an optional dependency. I suspect at the >>> time that there was no Java client. We really should have a required >>> dependency on the Java client and the Elasticsearch dependency should be a >>> test dependency, not optional. >>> >>> I see 4 choices here for the 1.10.0 release: >>> >>> 1. Do nothing. This is not ideal since the component is practically useless >>> as it exists and has security vulnerabilities. >>> 2. Drop the module in 1.10.0. This is obviously not backward compatible but >>> any upgrade is going to break compatibility. We can defer re-implementing >>> it until 2.0. >>> 3. Upgrade to a supported version of the Java client (which I believe has a >>> license that is compatible with the ASF). Again, this is not backward >>> compatible. It would need to use ElasticSearch for testing. The latest >>> versions of ElasticSearch use a license which is Category X so we will need >>> to include something in the NOTICE file and in the user’s guide warning >>> about the ElasticSearch license if the component is used. >>> 4. Upgrade to the latest version of >>> https://opensearch.org/docs/latest/clients/java/. This would use OpenSearch >>> instead of Elasticsearch and AFAIK would be incompatible with Elasticsearch. >>> >>> Of these options, my recommendation is to go with option 2. Once we >>> modularize things in 2.0 we can implement support for both Elasticsearch >>> and OpenSearch. We could also support Solr if desired. >>> >>> Thoughts? >>> >>> Ralph >>
