I should also note that this is exactly why Flume needs to be broken up. With the policy you suggest virtually every release would need to be a new major release since some component or another seems to become incompatible and/ unsupported with a prior version regularly.
Ralph > On Mar 31, 2022, at 1:25 AM, Ralph Goers <ralph.go...@dslextreme.com> wrote: > > I’d really like to get 1.10.0 released. Doing a 2.0 is going to be a much > larger effort. I would argue that no matter what we do the elastic search > support is incompatible, even if we just release what is currently there. > But if the choice is between releasing 1.10.0 with the current elastic search > support (knowing full well it is unusable) or having to do a 2.0.0 release I > will leave it in. It has been over 3 years since the last release and it is > quite apparent that a bunch of stuff that should have been updated then > wasn’t. > > So please let me know whether 1.10.0 should contain the existing elastic > search support or not. > > Ralph > >> On Mar 31, 2022, at 12:13 AM, Bessenyei Balázs Donát <bes...@apache.org> >> wrote: >> >> Sorry for the late reply. >> Can we make our next release a major if we're making incompatible changes? >> >> >> Donat >> >> >> On Wed, Mar 30, 2022 at 11:06 PM Ralph Goers <ralph.go...@dslextreme.com> >> wrote: >>> >>> I’ve not gotten any replies to this so I am assuming everyone is ok with my >>> going with option 2. >>> >>> Ralph >>> >>>> On Mar 29, 2022, at 1:05 AM, Ralph Goers <ralph.go...@dslextreme.com> >>>> wrote: >>>> >>>> I think addressing Elasticsearch may be the last major hurdle before >>>> 1.10.0 can be released. >>>> >>>> Flume currently uses Elasticsearch 0.90.1. >>>> https://www.elastic.co/support/eol#maintenance-tables shows that version >>>> 1.0.x reached end of life in 2015 so obviously the version Flume is using >>>> has been unsupported from even before that. >>>> >>>> The major goal of the 1.10.0 release has been to upgrade all the >>>> dependencies so that we aren’t using any that have known security >>>> vulnerabilities. From looking at >>>> https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=elasticsearch I am quite >>>> sure there is at least one CVE that applies to 0.90.1. >>>> >>>> Furthermore, this release is so old that I find it hard to imagine that >>>> anyone could still be using it. >>>> >>>> In addition, flume-ng-elasticsearch-sink is including >>>> org.elasticsearch:elasticsearch as an optional dependency. I suspect at >>>> the time that there was no Java client. We really should have a required >>>> dependency on the Java client and the Elasticsearch dependency should be a >>>> test dependency, not optional. >>>> >>>> I see 4 choices here for the 1.10.0 release: >>>> >>>> 1. Do nothing. This is not ideal since the component is practically >>>> useless as it exists and has security vulnerabilities. >>>> 2. Drop the module in 1.10.0. This is obviously not backward compatible >>>> but any upgrade is going to break compatibility. We can defer >>>> re-implementing it until 2.0. >>>> 3. Upgrade to a supported version of the Java client (which I believe has >>>> a license that is compatible with the ASF). Again, this is not backward >>>> compatible. It would need to use ElasticSearch for testing. The latest >>>> versions of ElasticSearch use a license which is Category X so we will >>>> need to include something in the NOTICE file and in the user’s guide >>>> warning about the ElasticSearch license if the component is used. >>>> 4. Upgrade to the latest version of >>>> https://opensearch.org/docs/latest/clients/java/. This would use >>>> OpenSearch instead of Elasticsearch and AFAIK would be incompatible with >>>> Elasticsearch. >>>> >>>> Of these options, my recommendation is to go with option 2. Once we >>>> modularize things in 2.0 we can implement support for both Elasticsearch >>>> and OpenSearch. We could also support Solr if desired. >>>> >>>> Thoughts? >>>> >>>> Ralph >>> >
