Adding the dev list back into the thread (a reply-all was missed). On Tue, Aug 27, 2019 at 10:49 AM James Sirota <jsir...@apache.org> wrote:
> I agree with Simon. HDP 2.x platform is rapidly approaching EOL and > everyone will likely need to migrate by end of year. Doing this platform > upgrade sooner will give everyone visibility into what Metron on HDP 3.x > looks like so they have time to plan and upgrade at their own pace. > Feature-wise, the Metron application itself will be unchanged. It is > merely the platform underneath that is changing. HDP itself can be > upgraded per instructions here: > https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.1.0/release-notes/content/upgrading_parent.html > > Once you back up your metron configs, the same configs that worked on the > previous version will continue to work on the version running on HDP 3.x. > If there is any discrepancy between the two or additional settings will be > required, those will be documented in the release notes. From the Metron > perspective, this upgrade would be no different than simply upgrading to > the new Metron version. > > James > > > 27.08.2019, 07:09, "Simon Elliston Ball" <si...@simonellistonball.com>: > > Something worth noting here is that HDP 2.6.5 is quite old and approaching > EoL rapidly, so the issue of upgrade is urgent. I am aware of a large > number of users who require this upgrade ASAP, and in fact an aware of zero > users who wish to remain on HDP 2. > > Perhaps those users who want to stay on the old platform can stick their > hands up and raise concerns, but this move will likely have to happen very > soon. > > Simon > > On Tue, 27 Aug 2019 at 15:04, Otto Fowler <ottobackwa...@gmail.com> wrote: > > Although we had the discussion, and some great ideas where passed around, > I do not believe we came to some kind of consensus on what 1.0 should look > like. So that discussion would have to be picked up again so that we could > know where we are at, and make it an actual thing if we were going to make > this a 1.0 release. > > I believe that the issues raised in that discussion gating 1.0 are still > largely applicable, including upgrade. > > Right now we have *ZERO* HDP 3.1 users. We will go from that to *only* > supporting 3.1 work and releases. So every user and deployment we currently > have will feel real pain, have to slay real dragons to move forward with > metron. > > With regards to support for older versions, the “backward capability” that > has been mentioned, I would not say that I have any specific plan for that > in mind. What I would say rather, is that I believe that we must be > explicit, setting expectations correctly and clearly with regards to our > intent while demonstrating that we have thought through the situation. That > discussion has not happened, at least I do not believe that the prior dev > thread really handles it in context. > > Depending on the upgrade situation for going to 3.1, it may be that a dual > stream of releases or fixes or new features to the extent that we can do it > will greatly reduce the pain for folks, or make it viable to stick with > metron until they can upgrade. > > The issue of what metron *is* features wise may be another one we want to > take up at some point. The idea being can we separate the metron > _integration parts from the metron core functionality such that we can work > on them separately and thus support multiple platforms through > integrations/applications. Of course definition of metron’s value beyond > integration, and what those features and application boundaries are would > be necessary. > > > > > On August 26, 2019 at 18:52:57, Michael Miklavcic ( > michael.miklav...@gmail.com) wrote: > > Hi devs and users, > > Some questions were asked in the Slack channel about our ongoing > HDP/Hadoop upgrade and I'd like to get a discussion rolling. The original > Hadoop upgrade discuss thread can be found here > https://lists.apache.org/thread.html/37cc29648f0592cc39d3c78a0d07fce38521bdbbc4cf40e022a7a8ea@%3Cdev.metron.apache.org%3E > > The major issue and impact with upgrading the Hadoop platform is that > there are breaking changes. Code that runs on HDP 3.1 will not run on 2.x. > Here is a sampling of core components we depend on that we know of so > far that are not backwards compatible: > > 1. The Core OS - we currently base our builds and test deployment off > of artifacts pulled from HDP. The latest rev of HDP no longer ships RPMs > for Centos 6, which means we need to upgrade to Centos 7 > 2. Ambari > 3. HBase > > This differs from individual components we've upgraded in the past in that > our code could still be deployed on the old as well as new version of the > component in a backwards compatible way. Based on semantic versioning, I > don't know if we can introduce this level of change in a point release, > which is the reason for kicking off this discussion. In the past, users and > developers in the community have suggested that they are -1 on a 1.x > release that does not provide an upgrade > https://lists.apache.org/thread.html/eb1a8df2d0a6a79c5d50540d1fdbf215ec83d831ff15d3117c2592cc@%3Cdev.metron.apache.org%3E > . > > Is there a way we can avoid a 1.x release? If we do need 1.x, do we still > see upgrades as a gating function? The main issue is that this has the > potential to drag out the upgrade and further couple it with other > features. And with Storm 1.x being eol'ed, I'm not sure this is something > we can wait much longer for. I'll think on this and send out my own > thoughts once folks have had a chance to review. > > Best, > Mike Miklavcic > Apache Metron, PMC, committer > > > -- > -- > simon elliston ball > @sireb > > > > ------------------- > Thank you, > > James Sirota > PMC- Apache Metron > jsirota AT apache DOT org > >
