Hi folks, Many things happened during DataWorks Summit San Jose 2017. Some of the folks(Cos, Roman, Amir, Nate, Mike, etc) gathered together to discuss 1.2.1 and the future 1.3 release of Bigtop. I'd like to get back those discussions to the mailing list so that who can't make it there can still be with us for further discussions:
* 1.2.1 release a). Some of the folks expecting Docker on YARN to be back ported to 2.7.4 and included in the release b). Get rotted code out of our code base: packaging, deployment, testing, etc c). Get integration test to work in CI * 1.3.0 release a). More machine learning integrations b). K8S integration will be an interesting topic Please help me to complete the list if I miss something. :) OTOH, for me specifically, I visited Cloudera for doing a tech talk. I meet Sean Mackrory and there Hadoop and HBase lead. The pain point they're having for a long time is not having an integration test framework for there work on the bleeding edge. For example, whether a specific patch from Hadoop breaks HBase or Hive? My thinking towards this is this is what Bigtop tries to solve at the very beginning. We supposed to have folks from multiple projects to work with us to upgrade packages, and use our frameworks to properly integrate, test their code with other components. So, the future of Bigtop. I think tightly work with the other communities is a better way we move forward. But, that means something need to be changed. For example, our distribution is somehow, from developers perspective, old. Which can not support the integration and testing on the bleeding edge. If we still like to release something suggested for Production only, one of the solution is to have both dev and stable releases in Bigtop, so developers can work on the dev branch and test against newest components. In that case, people from other communities might be possible to help us upgrade the package to the newer version, which makes things easier. What do you guys think? Please join me for the discussion.
