Something worth noting here is that HDP 2.6.5 is quite old and approaching EoL rapidly, so the issue of upgrade is urgent. I am aware of a large number of users who require this upgrade ASAP, and in fact an aware of zero users who wish to remain on HDP 2.
Perhaps those users who want to stay on the old platform can stick their hands up and raise concerns, but this move will likely have to happen very soon. Simon On Tue, 27 Aug 2019 at 15:04, Otto Fowler <ottobackwa...@gmail.com> wrote: > Although we had the discussion, and some great ideas where passed around, > I do not believe we came to some kind of consensus on what 1.0 should look > like. So that discussion would have to be picked up again so that we could > know where we are at, and make it an actual thing if we were going to make > this a 1.0 release. > > I believe that the issues raised in that discussion gating 1.0 are still > largely applicable, including upgrade. > > Right now we have *ZERO* HDP 3.1 users. We will go from that to *only* > supporting 3.1 work and releases. So every user and deployment we currently > have will feel real pain, have to slay real dragons to move forward with > metron. > > With regards to support for older versions, the “backward capability” that > has been mentioned, I would not say that I have any specific plan for that > in mind. What I would say rather, is that I believe that we must be > explicit, setting expectations correctly and clearly with regards to our > intent while demonstrating that we have thought through the situation. That > discussion has not happened, at least I do not believe that the prior dev > thread really handles it in context. > > Depending on the upgrade situation for going to 3.1, it may be that a dual > stream of releases or fixes or new features to the extent that we can do it > will greatly reduce the pain for folks, or make it viable to stick with > metron until they can upgrade. > > The issue of what metron *is* features wise may be another one we want to > take up at some point. The idea being can we separate the metron > _integration parts from the metron core functionality such that we can work > on them separately and thus support multiple platforms through > integrations/applications. Of course definition of metron’s value beyond > integration, and what those features and application boundaries are would > be necessary. > > > > > On August 26, 2019 at 18:52:57, Michael Miklavcic ( > michael.miklav...@gmail.com) wrote: > > Hi devs and users, > > Some questions were asked in the Slack channel about our ongoing > HDP/Hadoop upgrade and I'd like to get a discussion rolling. The original > Hadoop upgrade discuss thread can be found here > https://lists.apache.org/thread.html/37cc29648f0592cc39d3c78a0d07fce38521bdbbc4cf40e022a7a8ea@%3Cdev.metron.apache.org%3E > > The major issue and impact with upgrading the Hadoop platform is that > there are breaking changes. Code that runs on HDP 3.1 will not run on 2.x. > Here is a sampling of core components we depend on that we know of so > far that are not backwards compatible: > > 1. The Core OS - we currently base our builds and test deployment off > of artifacts pulled from HDP. The latest rev of HDP no longer ships RPMs > for Centos 6, which means we need to upgrade to Centos 7 > 2. Ambari > 3. HBase > > This differs from individual components we've upgraded in the past in that > our code could still be deployed on the old as well as new version of the > component in a backwards compatible way. Based on semantic versioning, I > don't know if we can introduce this level of change in a point release, > which is the reason for kicking off this discussion. In the past, users and > developers in the community have suggested that they are -1 on a 1.x > release that does not provide an upgrade > https://lists.apache.org/thread.html/eb1a8df2d0a6a79c5d50540d1fdbf215ec83d831ff15d3117c2592cc@%3Cdev.metron.apache.org%3E > . > > Is there a way we can avoid a 1.x release? If we do need 1.x, do we still > see upgrades as a gating function? The main issue is that this has the > potential to drag out the upgrade and further couple it with other > features. And with Storm 1.x being eol'ed, I'm not sure this is something > we can wait much longer for. I'll think on this and send out my own > thoughts once folks have had a chance to review. > > Best, > Mike Miklavcic > Apache Metron, PMC, committer > > > -- -- simon elliston ball @sireb
