The question comes back to the DISCUSS I opened the other day about upgrading ES. I believe we could theoretically maintain backwards compatibility, but we'd have to keep the existing TransportClient. It's not deprecated yet, but it will be. Keeping the ability to manage ES 2.x and 5.x+ via Ambari might be some work, but it looks like it's just calling shell commands from python, e.g. https://github.com/apache/metron/blob/master/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/ELASTICSEARCH/2.3.3/package/scripts/elastic_master.py#L43-L45
On Wed, Oct 4, 2017 at 11:08 AM, Casey Stella <[email protected]> wrote: > Ok, so, whoever does this ES work, we should ensure the upgrade path is at > least spelled out in the Upgrade doc. This would also probably, IMO, > necessitate a major version change in metron. > > On Wed, Oct 4, 2017 at 1:07 PM, Justin Leet <[email protected]> wrote: > > > Forgot the link > > https://www.elastic.co/guide/en/elasticsearch/reference/ > > current/setup-upgrade.html > > > > On Wed, Oct 4, 2017 at 1:07 PM, Simon Elliston Ball < > > [email protected]> wrote: > > > > > The simplest option would probably be to upgrade the ES and then > reindex > > > from the HDFS store. Alternatively there are means to do inplace > upgrades > > > from 2.x to 5.x I believe. > > > > > > Simon > > > > > > > On 4 Oct 2017, at 18:05, Casey Stella <[email protected]> wrote: > > > > > > > > So, how would this work in an upgrade scenario that does not involve > > > losing > > > > the existing indexed data? > > > > > > > > On Wed, Oct 4, 2017 at 12:55 PM, Michael Miklavcic < > > > > [email protected]> wrote: > > > > > > > >> The client I'm currently working on moving towards would *not* be > > > backwards > > > >> compatible. > > > >> https://www.elastic.co/guide/en/elasticsearch/client/java- > > > >> rest/current/java-rest-high-compatibility.html > > > >> > > > >> " > > > >> The High Level Client is guaranteed to be able to communicate with > any > > > >> Elasticsearch node running on the same major version and greater or > > > equal > > > >> minor version. It doesn’t need to be in the same minor version as > the > > > >> Elasticsearch nodes it communicates with, as it is forward > compatible > > > >> meaning that it supports communicating with later versions of > > > Elasticsearch > > > >> than the one it was developed for. > > > >> > > > >> The 5.6 client can communicate with any 5.6.x Elasticsearch node. > > > Previous > > > >> 5.x minor versions like 5.5.x, 5.4.x etc. are not (fully) supported. > > > >> " > > > >> > > > >> Best, > > > >> Mike > > > >> > > > >> > > > >> On Wed, Oct 4, 2017 at 10:45 AM, Simon Elliston Ball < > > > >> [email protected]> wrote: > > > >> > > > >>> A number of people are currently working on upgrading the ES > support > > in > > > >>> Metron to 5.x (including the clients, and the mpack managed > install). > > > >>> > > > >>> Would anyone have any objections to dropping formal support for 2.x > > as > > > a > > > >>> result of this work? In theory the clients should be backward > > > compatible > > > >>> against older data stores, so metron could be upgraded without > > needing > > > an > > > >>> elastic upgrade. > > > >>> > > > >>> In practice, we would need to do pretty extensive testing and I > > > wouldn’t > > > >>> want us to have to code around long term support on older clients > if > > > >> no-one > > > >>> in the community cares enough about the older ES. Do we think there > > is > > > a > > > >>> case to be made for maintaining long term support for older > clients? > > > >>> > > > >>> Simon > > > >> > > > > > > > > >
