Since Accumulo doesn't bundle Hadoop into the release, the only
difference this makes is whether or not it breaks our builds during
testing, which could indicate a bug in Hadoop, or an incompatibility
with that version of Hadoop. The version of Accumulo built with 3.3.0
should work perfectly fine with 3.3.1. If you want to actually build
with 3.3.1 on the classpath during compilation time, though, it's
trivial to add -Dhadoop.version=3.3.1 to the Maven command line.
I'm looking at the commit you referenced, and I don't see how that has
anything to do with addressing the test that was flaky.
If you are correct in your assessment that the test was overwhelming
MiniDFS, then that would still be a bug... perhaps a test bug... but
either way, the failure is not expected and should be addressed. If we
bump the version, we'd have to make sure it doesn't break the build...
and that means fixing whatever test issue was causing it. If you're
willing to contribute a pull request to do that work, we can consider
including it. However, as I said initially, it doesn't really
matter... the version specified in the POM is only the version being
built with/tested with by default... it's not bundled into Accumulo in
any way. It's just a fixed version whose API we expect to be
compatible with.
If we do bump it, it should be to 3.3.3, which is the latest 3.3 version.
On Thu, May 19, 2022 at 5:26 PM Chris Bevard wrote:
>
> Hi,
>
> I was wondering if the dev team would reconsider using the Hadoop 3.3.1
> version for the next release version of Accumulo. I noticed that the hadoop
> dependency version was updated to 3.3.1 briefly by
> commit 3c3a91f7a4b6ea290a383a77844cabae34eaeb1f, but it was dropped back to
> 3.3.0 in commit 48679fef73e246de52fbeecad03f974f2116b97a shortly after.
> The explanation for undoing the change was that hadoop 3.3.1 was causing
> intermittent IT failures, most frequently in the CountNameNodeOpsBulkIT.
>
> I checked out that commit myself and also noticed that the
> CountNameNodeOpsBulkIT was failing often with an IOException "Unable to
> close file because the last block...does not have enough number of
> replicas", which I don't believe is indicative of a bug in hadoop or the
> accumulo code. I think what's more likely is that the multithreaded test
> was overwhelming the minidfs cluster with requests. I'm not sure which
> default value/behavior was updated in hadoop 3.3.1 that would cause the
> minicluster to blow up where it wasn't previously in 3.3.0, but I noticed
> in later commits the issue was resolved. It looks like the ClientContext
> changes in the very next code change (commit
> 4b66b96b8f6c65c390fc26c11acf8c51cb78d858) resolve the IT failures that were
> the reason for moving the hadoop version back to 3.3.0. If you check out
> that or any later commit, and update the hadoop dependency version in the
> parent pom to 3.3.1, then the IT failures are resolved.
>
> The reason I'd like the hadoop 3.3.1 version to make it into the next
> release version of Accumulo is because I've been experimenting with
> Accumulo using S3 as the underlying file system. This change (
> https://issues.apache.org/jira/browse/HADOOP-17597) that was added in
> hadoop 3.3.1 makes it possible to use the S3AFileSystem defined in
> hadoop-aws with Accumulo to replace HDFS with S3. The only change needed is
> to update the manager.walog.closer.implementation property and supply an
> S3LogCloser implementation on the classpath.
>
> Thanks,
> Chris
