Re: [DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?
On Thu, Sep 14, 2017 at 4:23 PM, Andrew Wangwrote: > > > > > I discussed this on yetus-dev a while back and Allen thought it'd be > non-trivial: > > https://lists.apache.org/thread.html/552ad614d1b3d5226a656b60c01084 > 57bcaa1219fb9ad985f8750ba1@%3Cdev.yetus.apache.org%3E > > I unfortunately don't have the test-patch.sh expertise to dig into this. > > > Hurm. getting something generic certainly sounds like a lot of work, but getting something that works specifically with Maven maybe not. lemme see if I can describe what I think the pieces look like in a jira.
Re: [DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?
I actually like this idea: > One approach: do a dependency:list of each module and for those that show a change with the patch we run tests there. Can 'jdeps' be used to prune the list of sub modules on which we do pre-commit ? Essentially, we figure out which classes actually use the modified classes from the patch and then run the pre-commit on those packages ? Cheers -Arun On Thu, Sep 14, 2017 at 2:23 PM, Andrew Wangwrote: > On Thu, Sep 14, 2017 at 1:59 PM, Sean Busbey wrote: > > > > > > > On 2017-09-14 15:36, Chris Douglas wrote: > > > This has gotten bad enough that people are dismissing legitimate test > > > failures among the noise. > > > > > > On Thu, Sep 14, 2017 at 1:20 PM, Allen Wittenauer > > > wrote: > > > > Someone should probably invest some time into integrating the > > HBase flaky test code a) into Yetus and then b) into Hadoop. > > > > > > What does the HBase flaky test code do? Another extension to > > > test-patch could run all new/modified tests multiple times, and report > > > to JIRA if any run fails. > > > > > > > The current HBase stuff segregates untrusted tests by looking through > > nightly test runs to find things that fail intermittently. We then don't > > include those tests in either nightly or precommit tests. We have a > > different job that just runs the untrusted tests and if they start > passing > > removes them from the list. > > > > There's also a project getting used by SOLR called "BeastIT" that goes > > through running parallel copies of a given test a large number of times > to > > reveal flaky tests. > > > > Getting either/both of those into Yetus and used here would be a huge > > improvement. > > > > I discussed this on yetus-dev a while back and Allen thought it'd be > non-trivial: > > https://lists.apache.org/thread.html/552ad614d1b3d5226a656b60c01084 > 57bcaa1219fb9ad985f8750ba1@%3Cdev.yetus.apache.org%3E > > I unfortunately don't have the test-patch.sh expertise to dig into this. > > - > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > > > > >
Re: [DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?
On Thu, Sep 14, 2017 at 1:59 PM, Sean Busbeywrote: > > > On 2017-09-14 15:36, Chris Douglas wrote: > > This has gotten bad enough that people are dismissing legitimate test > > failures among the noise. > > > > On Thu, Sep 14, 2017 at 1:20 PM, Allen Wittenauer > > wrote: > > > Someone should probably invest some time into integrating the > HBase flaky test code a) into Yetus and then b) into Hadoop. > > > > What does the HBase flaky test code do? Another extension to > > test-patch could run all new/modified tests multiple times, and report > > to JIRA if any run fails. > > > > The current HBase stuff segregates untrusted tests by looking through > nightly test runs to find things that fail intermittently. We then don't > include those tests in either nightly or precommit tests. We have a > different job that just runs the untrusted tests and if they start passing > removes them from the list. > > There's also a project getting used by SOLR called "BeastIT" that goes > through running parallel copies of a given test a large number of times to > reveal flaky tests. > > Getting either/both of those into Yetus and used here would be a huge > improvement. > > I discussed this on yetus-dev a while back and Allen thought it'd be non-trivial: https://lists.apache.org/thread.html/552ad614d1b3d5226a656b60c0108457bcaa1219fb9ad985f8750ba1@%3Cdev.yetus.apache.org%3E I unfortunately don't have the test-patch.sh expertise to dig into this. - > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > >
Re: [DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?
On 2017-09-14 15:36, Chris Douglaswrote: > This has gotten bad enough that people are dismissing legitimate test > failures among the noise. > > On Thu, Sep 14, 2017 at 1:20 PM, Allen Wittenauer > wrote: > > Someone should probably invest some time into integrating the HBase > > flaky test code a) into Yetus and then b) into Hadoop. > > What does the HBase flaky test code do? Another extension to > test-patch could run all new/modified tests multiple times, and report > to JIRA if any run fails. > The current HBase stuff segregates untrusted tests by looking through nightly test runs to find things that fail intermittently. We then don't include those tests in either nightly or precommit tests. We have a different job that just runs the untrusted tests and if they start passing removes them from the list. There's also a project getting used by SOLR called "BeastIT" that goes through running parallel copies of a given test a large number of times to reveal flaky tests. Getting either/both of those into Yetus and used here would be a huge improvement. - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?
On Thu, Sep 14, 2017 at 1:43 PM, Ray Chiangwrote: > The other solution I've seen (from Oozie?) is to re-run just the subset of > failing tests once more. That should help cut down the failures except for > the mostly flaky of flakies. Many of our unit tests generate random cases and report the seed to reproduce, and others are flaky because they collide with other tests' artifacts. Success on reexec risks missing some important cases. I'd rather err on the side of fixing/removing tests that are too unreliable to serve their purpose. I understand the counter-argument, but Hadoop has accumulated a ton of tests without a recent round of pruning. We could stand to lose a few cycles to highlight and trim the cases that cost us the most dev and CI time. -C - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?
On 9/14/17 1:36 PM, Chris Douglas wrote: This has gotten bad enough that people are dismissing legitimate test failures among the noise. On Thu, Sep 14, 2017 at 1:20 PM, Allen Wittenauerwrote: Someone should probably invest some time into integrating the HBase flaky test code a) into Yetus and then b) into Hadoop. What does the HBase flaky test code do? Another extension to test-patch could run all new/modified tests multiple times, and report to JIRA if any run fails. Test code is not as thoroughly reviewed, and we shouldn't expect that will change. We can at least identify the tests that are unreliable, assign responsibility for fixing them, and disable noisy tests so we can start trusting the CI output. I'd rather miss a regression by disabling a flaky test than lose confidence in the CI infrastructure. Would anyone object to disabling, even deleting failing tests in trunk until they're fixed? -C The other solution I've seen (from Oozie?) is to re-run just the subset of failing tests once more. That should help cut down the failures except for the mostly flaky of flakies. -Ray - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?
This has gotten bad enough that people are dismissing legitimate test failures among the noise. On Thu, Sep 14, 2017 at 1:20 PM, Allen Wittenauerwrote: > Someone should probably invest some time into integrating the HBase > flaky test code a) into Yetus and then b) into Hadoop. What does the HBase flaky test code do? Another extension to test-patch could run all new/modified tests multiple times, and report to JIRA if any run fails. Test code is not as thoroughly reviewed, and we shouldn't expect that will change. We can at least identify the tests that are unreliable, assign responsibility for fixing them, and disable noisy tests so we can start trusting the CI output. I'd rather miss a regression by disabling a flaky test than lose confidence in the CI infrastructure. Would anyone object to disabling, even deleting failing tests in trunk until they're fixed? -C - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?
> On Sep 14, 2017, at 11:01 AM, Sean Busbeywrote: > >> Committers MUST check the qbt output after a commit. They MUST make sure > their commit didn’t break something new. > > How do we make this easier / more likely to happen? > > For example, I don't see any notice on HADOOP-14654 that the qbt > post-commit failed. Is this a timing thing? Did Steve L just notice the > break before we could finish the 10 hours it takes to get qbt done? qbt doesn't update JIRA because... > > How solid would qbt have to be for us to do something drastic like > auto-revert changes after a failure? ... I have never seen the unit tests for Hadoop pass completely. So it would always fail every JIRA that it was testing. There's no point in enabling the JIRA issue update or anything like that until our unit tests actually get reliable. But that also means we're reliant upon the community to self police. That is also failing. Prior to Yetus getting involved, the only unit tests that would reliably pass was mapreduce. The rest would almost always fail. It had been that way for years. Someone should probably invest some time into integrating the HBase flaky test code a) into Yetus and then b) into Hadoop. It's also worth pointing out that the YARN findbugs error has been there for about six months. It would also fail the build. - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?
Thanks Sean busbey for starting the discussion.This was one of pain point even I notified to Allen couple of times.IIUC his point is committers and contributors should monitor qbt. IMHO,instead of post-commit try to fix in pre-commit only?? May be we can get dependcy module(dependency:list) and execute pre-commit. Even auto revert also good option. Following previous discussions which I came across ,just for reference. Suggestions: 1) let qbt give alarm on broken jira. http://mail-archives.apache.org/mod_mbox/hadoop-common-dev/201708.mbox/%3c3b4c977e-5920-4f55-8ebe-cc7e10bcf...@effectivemachines.com%3e 2) Run on parent project,as of now it's run on the current module ( where changes happened),this might not completely ignore? 3) Update dummy class,when contributors/committers knows that it can impact other modules such that pre-commit can run. --Brahma Reddy Battula On Thu, 14 Sep 2017 at 11:31 PM, Sean Busbeywrote: > > Committers MUST check the qbt output after a commit. They MUST make sure > their commit didn’t break something new. > > How do we make this easier / more likely to happen? > > For example, I don't see any notice on HADOOP-14654 that the qbt > post-commit failed. Is this a timing thing? Did Steve L just notice the > break before we could finish the 10 hours it takes to get qbt done? > > How solid would qbt have to be for us to do something drastic like > auto-revert changes after a failure? > > > On Thu, Sep 14, 2017 at 11:05 AM, Allen Wittenauer < > a...@effectivemachines.com > > wrote: > > > > > > On Sep 14, 2017, at 8:03 AM, Sean Busbey wrote: > > > > > > * HADOOP-14654 updated commons-httplient to a new patch release in > > > hadoop-project > > > * Precommit checked the modules that changed (i.e. not many) > > > * nightly had Azure support break due to a change in behavior. > > > > OK, so it worked as coded/designed. > > > > > Is this just the cost of our approach to precommit vs post commit > > testing? > > > > Yes. It’s a classic speed vs. size computing problem. > > > > test-patch: quick but only runs a subset of tests > > qbt: comprehensive but takes a very long time > > > > Committers MUST check the qbt output after a commit. They MUST > > make sure their commit didn’t break something new. > > > > > One approach: do a dependency:list of each module and for those that > > show a > > > change with the patch we run tests there. > > > > As soon as you change something like junit, you’re running over > > everything … > > > > Plus, let’s get real: there is a large contingent of committers > > that barely take the time to read or even comprehend the current Yetus > > output. Adding *more* output is the last thing we want to do. > > > > > This will cause a slew of tests to run when dependencies change. For > the > > > change in HADOOP-14654 probably we'd just have to run at the top level. > > > > … e.g., exactly what qbt does for 10+ hours every night. > > > > It’s important to also recognize that we need to be “good > > citizens” in the ASF. If we can do dependency checking in one 10 hour > > streak vs. several, that reduces the load on the ASF build > infrastructure. > > > > > > > > > -- > busbey > -- --Brahma Reddy Battula
Re: [DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?
> Committers MUST check the qbt output after a commit. They MUST make sure their commit didn’t break something new. How do we make this easier / more likely to happen? For example, I don't see any notice on HADOOP-14654 that the qbt post-commit failed. Is this a timing thing? Did Steve L just notice the break before we could finish the 10 hours it takes to get qbt done? How solid would qbt have to be for us to do something drastic like auto-revert changes after a failure? On Thu, Sep 14, 2017 at 11:05 AM, Allen Wittenauerwrote: > > > On Sep 14, 2017, at 8:03 AM, Sean Busbey wrote: > > > > * HADOOP-14654 updated commons-httplient to a new patch release in > > hadoop-project > > * Precommit checked the modules that changed (i.e. not many) > > * nightly had Azure support break due to a change in behavior. > > OK, so it worked as coded/designed. > > > Is this just the cost of our approach to precommit vs post commit > testing? > > Yes. It’s a classic speed vs. size computing problem. > > test-patch: quick but only runs a subset of tests > qbt: comprehensive but takes a very long time > > Committers MUST check the qbt output after a commit. They MUST > make sure their commit didn’t break something new. > > > One approach: do a dependency:list of each module and for those that > show a > > change with the patch we run tests there. > > As soon as you change something like junit, you’re running over > everything … > > Plus, let’s get real: there is a large contingent of committers > that barely take the time to read or even comprehend the current Yetus > output. Adding *more* output is the last thing we want to do. > > > This will cause a slew of tests to run when dependencies change. For the > > change in HADOOP-14654 probably we'd just have to run at the top level. > > … e.g., exactly what qbt does for 10+ hours every night. > > It’s important to also recognize that we need to be “good > citizens” in the ASF. If we can do dependency checking in one 10 hour > streak vs. several, that reduces the load on the ASF build infrastructure. > > > -- busbey
[jira] [Resolved] (HADOOP-14647) Update third-party libraries for Hadoop 3
[ https://issues.apache.org/jira/browse/HADOOP-14647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang resolved HADOOP-14647. - Resolution: Done Fix Version/s: 3.0.0-beta1 Moved out unfinished subtasks to a later version. > Update third-party libraries for Hadoop 3 > - > > Key: HADOOP-14647 > URL: https://issues.apache.org/jira/browse/HADOOP-14647 > Project: Hadoop Common > Issue Type: Task >Affects Versions: 3.0.0-beta1 >Reporter: Ray Chiang >Assignee: Ray Chiang > Fix For: 3.0.0-beta1 > > > There are a bunch of old third-party dependencies in trunk. Before we get to > the final release candidate, it would be good to move some of these > dependencies to the latest (or at least the latest compatible) > , since it could be a while before the next update. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-14654) Update httpclient version to 4.5.3
[ https://issues.apache.org/jira/browse/HADOOP-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang resolved HADOOP-14654. - Resolution: Won't Do Sorry about the breakage [~ste...@apache.org]. Going to close this w.r.t. beta1. We can redo this sometime in the future. > Update httpclient version to 4.5.3 > -- > > Key: HADOOP-14654 > URL: https://issues.apache.org/jira/browse/HADOOP-14654 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.0.0-beta1 >Reporter: Ray Chiang >Assignee: Ray Chiang > Fix For: 3.0.0-beta1 > > Attachments: HADOOP-14654.001.patch > > > Update the dependency > org.apache.httpcomponents:httpclient:4.5.2 > to the latest (4.5.3). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?
> On Sep 14, 2017, at 8:03 AM, Sean Busbeywrote: > > * HADOOP-14654 updated commons-httplient to a new patch release in > hadoop-project > * Precommit checked the modules that changed (i.e. not many) > * nightly had Azure support break due to a change in behavior. OK, so it worked as coded/designed. > Is this just the cost of our approach to precommit vs post commit testing? Yes. It’s a classic speed vs. size computing problem. test-patch: quick but only runs a subset of tests qbt: comprehensive but takes a very long time Committers MUST check the qbt output after a commit. They MUST make sure their commit didn’t break something new. > One approach: do a dependency:list of each module and for those that show a > change with the patch we run tests there. As soon as you change something like junit, you’re running over everything … Plus, let’s get real: there is a large contingent of committers that barely take the time to read or even comprehend the current Yetus output. Adding *more* output is the last thing we want to do. > This will cause a slew of tests to run when dependencies change. For the > change in HADOOP-14654 probably we'd just have to run at the top level. … e.g., exactly what qbt does for 10+ hours every night. It’s important to also recognize that we need to be “good citizens” in the ASF. If we can do dependency checking in one 10 hour streak vs. several, that reduces the load on the ASF build infrastructure. - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?
Moving discussion here from HADOOP-14654. Short synopsis: * HADOOP-14654 updated commons-httplient to a new patch release in hadoop-project * Precommit checked the modules that changed (i.e. not many) * nightly had Azure support break due to a change in behavior. Is this just the cost of our approach to precommit vs post commit testing? One approach: do a dependency:list of each module and for those that show a change with the patch we run tests there. This will cause a slew of tests to run when dependencies change. For the change in HADOOP-14654 probably we'd just have to run at the top level. Steve L and I had some more details about things we could do on the ticket if folks are interested. -- busbey
[jira] [Reopened] (HADOOP-14868) regression: wasb tests failing
[ https://issues.apache.org/jira/browse/HADOOP-14868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened HADOOP-14868: - > regression: wasb tests failing > -- > > Key: HADOOP-14868 > URL: https://issues.apache.org/jira/browse/HADOOP-14868 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 2.9.0, 3.0.0-beta1, 3.1.0 >Reporter: Steve Loughran > Fix For: 3.0.0-beta1, 3.1.0 > > Attachments: failures.txt > > > Looks like some of the azure tests are failing, including the mock test > TestNativeAzureFileSystemOperationsMocked > switching back to commit 13eda5000304099d and only rebuilding the > hadoop-azure module is enough for them to work again > so, possible cause HADOOP-14520, that being the only other commit to have > gone in. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-14868) regression: wasb tests failing
[ https://issues.apache.org/jira/browse/HADOOP-14868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-14868. - Resolution: Fixed Fix Version/s: 3.1.0 3.0.0-beta1 reverted the relevant commit in trunk & branch-3; tests should be good again > regression: wasb tests failing > -- > > Key: HADOOP-14868 > URL: https://issues.apache.org/jira/browse/HADOOP-14868 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 2.9.0, 3.0.0-beta1, 3.1.0 >Reporter: Steve Loughran > Fix For: 3.0.0-beta1, 3.1.0 > > > Looks like some of the azure tests are failing, including the mock test > TestNativeAzureFileSystemOperationsMocked > switching back to commit 13eda5000304099d and only rebuilding the > hadoop-azure module is enough for them to work again > so, possible cause HADOOP-14520, that being the only other commit to have > gone in. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-14654) Update httpclient version to 4.5.3
[ https://issues.apache.org/jira/browse/HADOOP-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened HADOOP-14654: - this breaks the azure tests; something which can be checked with a {{mvn test -Dtest=TestNativeAzureFileSystemMocked}} Interesting (and worrying) that the original change didn't catch this; makes me think that yetus is over-selective about test coverage. Which means: all JAR updates are going to have to be rigorously checked by the submitter before submission. For this patch, the azure tests are going to have to be working again before it can go it. That may be a matter of actually changing the test assumptions/assertions (given it's mocking, it is potentially overreacting). sorry > Update httpclient version to 4.5.3 > -- > > Key: HADOOP-14654 > URL: https://issues.apache.org/jira/browse/HADOOP-14654 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.0.0-beta1 >Reporter: Ray Chiang >Assignee: Ray Chiang > Fix For: 3.0.0-beta1 > > Attachments: HADOOP-14654.001.patch > > > Update the dependency > org.apache.httpcomponents:httpclient:4.5.2 > to the latest (4.5.3). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14868) regression: wasb tests failing
Steve Loughran created HADOOP-14868: --- Summary: regression: wasb tests failing Key: HADOOP-14868 URL: https://issues.apache.org/jira/browse/HADOOP-14868 Project: Hadoop Common Issue Type: Bug Components: fs/azure Affects Versions: 2.9.0, 3.0.0-beta1, 3.1.0 Reporter: Steve Loughran Looks like some of the azure tests are failing, including the mock test TestNativeAzureFileSystemOperationsMocked switching back to commit 13eda5000304099d and only rebuilding the hadoop-azure module is enough for them to work again so, possible cause HADOOP-1452, that being the only other commit to have gone in. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/523/ [Sep 13, 2017 5:49:34 PM] (cliang) HADOOP-14804. correct wrong parameters format order in core-default.xml. [Sep 13, 2017 5:59:04 PM] (wang) HADOOP-14857. Fix downstream shaded client integration test. Contributed [Sep 13, 2017 6:06:47 PM] (rohithsharmaks) YARN-7157. Add admin configuration to filter per-user's apps in secure [Sep 13, 2017 7:29:08 PM] (epayne) YARN-4727. Unable to override the /home/ericp/run/conf/ env variable for [Sep 13, 2017 7:38:58 PM] (epayne) Revert 'YARN-4727. Unable to override the $HADOOP_CONF_DIR env variable [Sep 13, 2017 7:41:55 PM] (epayne) YARN-4727. Unable to override the $HADOOP_CONF_DIR env variable for [Sep 13, 2017 7:54:02 PM] (arp) HADOOP-14867. Update HDFS Federation setup document, for incorrect -1 overall The following subsystems voted -1: findbugs unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: FindBugs : module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Hard coded reference to an absolute pathname in org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext) At DockerLinuxContainerRuntime.java:absolute pathname in org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext) At DockerLinuxContainerRuntime.java:[line 490] Failed junit tests : hadoop.hdfs.TestDFSInotifyEventInputStream hadoop.hdfs.server.namenode.TestReencryptionWithKMS hadoop.hdfs.TestClientProtocolForPipelineRecovery hadoop.hdfs.TestLeaseRecoveryStriped hadoop.hdfs.TestReconstructStripedFile hadoop.hdfs.TestDFSUpgradeFromImage hadoop.hdfs.server.datanode.TestDirectoryScanner hadoop.hdfs.TestFileAppendRestart hadoop.hdfs.TestDFSStripedOutputStreamWithFailure040 hadoop.yarn.server.nodemanager.containermanager.TestContainerManager hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler hadoop.yarn.client.api.impl.TestAMRMClientContainerRequest hadoop.mapreduce.v2.hs.webapp.TestHSWebApp hadoop.fs.azure.TestNativeAzureFileSystemConcurrency hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked hadoop.fs.azure.TestNativeAzureFileSystemFileNameCheck hadoop.fs.azure.TestOutOfBandAzureBlobOperations hadoop.fs.azure.TestNativeAzureFileSystemContractMocked hadoop.fs.azure.TestNativeAzureFileSystemMocked hadoop.fs.azure.TestWasbFsck hadoop.yarn.sls.TestSLSRunner hadoop.yarn.sls.TestReservationSystemInvariants hadoop.yarn.sls.nodemanager.TestNMSimulator Timed out junit tests : org.apache.hadoop.hdfs.TestWriteReadStripedFile org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/523/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/523/artifact/out/diff-compile-javac-root.txt [292K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/523/artifact/out/diff-checkstyle-root.txt [17M] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/523/artifact/out/diff-patch-pylint.txt [20K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/523/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/523/artifact/out/diff-patch-shelldocs.txt [12K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/523/artifact/out/whitespace-eol.txt [11M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/523/artifact/out/whitespace-tabs.txt [1.2M] findbugs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/523/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html [8.0K] javadoc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/523/artifact/out/diff-javadoc-javadoc-root.txt [1.9M] unit: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/523/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt