On Tue, Jan 12, 2021 at 10:26 AM Keith Turner <ke...@deenlo.com> wrote:
> I have been looking around at what may need to be done before a Fluo > release. There were two issues w/ the 2.0.0 milestone, one was > already completed. There other could not be done until after Accumulo > 2.1.0 is released so I removed the 2.0.0 milestone from it. > > I have been running the full build and fixed one reliably flaky IT. > There are some other flaky ones, but they are not as reliably flaky. > > I successfully ran a small run of stresso on a single VM yesterday w/o > issue. I used Hadoop 3.1.3, ZK 3.4.14, and Accumulo 2.0.1. I tried > using a newer version of ZK and ran into problems w/ Curator, ZK, > Hadoop and Accumulo. These problems have nothing to do w/ Fluo, they > are just problems w/ the dependencies interacting w/ each other. Fluo > compiles fine against the old and new versions of ZK and Curator. I > am not quite sure where it occurs, but at some point curator stopped > supporting ZK 3.4. I suspect these issues could be resolved at runtime > w/o any changes to Fluo. The following are the issues I recall > encountering. > > * Seems Mini Accumulo 2.0.0 does not support newer than ZK 3.4? It > kept hanging w/ newer versions. > I believe the main problem with minicluster was fixed in https://github.com/apache/accumulo/pull/1531, because minicluster requires the use of some four-letter word commands. This fix did not make it into 2.0. > * Newer versions of curator were trying to use a non-existant ZK > method. Not exactly sure what was going on, some change between ZK > and Curator results in runtime exceptions. > * I think newer version of Hadoop (maybe 3.3) are pulling in new > version of curator onto the classpath, which caused issues w/ older ZK > versions. Not completely sure about this assumption, I may be wrong. > This problem was specific to stresso, which uses the yarn jar command > to run things. > * Unrelated to curator, the Hadoop 3.1.4 shaded jars in maven seemed > to be empty. > > I think the worst case of doing a 2.0.0 release now is that a 2.0.1 > release w/ no API changes could possibly be needed to address possible > transitive dependency issues. > > I think it would be nice to get the webindex example working against > 2.0 before release, but I don't think it's required. It's not using > any new APIs, so it would not test any new functionality. I started > working on getting it in shape in the past and this required getting > Fluo Recipes up to date w/ the latest Accumulo 2.0 changes which I > did. I also started getting Webindex up to date w/ Accumulo 2.0 > changes, but did not complete that task. > > There is always more work to do, but a lot has already been done for > 2.0.0 and I think what is there is in really good shape and long > overdue for a release. > I'm inclined to agree. I think it would be fine to release with what is there now, as long as we're clear about what versions we've tried. I think you're right that most issues with newer versions could be addressed downstream with class path decisions. And, if there is something that can't be easily fixed, then releasing a 2.0.1 later would be reasonable. If we're headed for a release, we should make sure the copyright dates are up-to-date, since it's now 2021.
