Re: [DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?

On Thu, Sep 14, 2017 at 4:23 PM, Andrew Wang wrote: > > > > > I discussed this on yetus-dev a while back and Allen thought it'd be > non-trivial: > > https://lists.apache.org/thread.html/552ad614d1b3d5226a656b60c01084 >

Re: [DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?

I actually like this idea: > One approach: do a dependency:list of each module and for those that show a change with the patch we run tests there. Can 'jdeps' be used to prune the list of sub modules on which we do pre-commit ? Essentially, we figure out which classes actually use the modified

Re: [DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?

On Thu, Sep 14, 2017 at 1:59 PM, Sean Busbey wrote: > > > On 2017-09-14 15:36, Chris Douglas wrote: > > This has gotten bad enough that people are dismissing legitimate test > > failures among the noise. > > > > On Thu, Sep 14, 2017 at 1:20 PM, Allen

Re: [DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?

On 2017-09-14 15:36, Chris Douglas wrote: > This has gotten bad enough that people are dismissing legitimate test > failures among the noise. > > On Thu, Sep 14, 2017 at 1:20 PM, Allen Wittenauer > wrote: > > Someone should probably

Re: [DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?

On Thu, Sep 14, 2017 at 1:43 PM, Ray Chiang wrote: > The other solution I've seen (from Oozie?) is to re-run just the subset of > failing tests once more. That should help cut down the failures except for > the mostly flaky of flakies. Many of our unit tests generate random

Re: [DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?

On 9/14/17 1:36 PM, Chris Douglas wrote: This has gotten bad enough that people are dismissing legitimate test failures among the noise. On Thu, Sep 14, 2017 at 1:20 PM, Allen Wittenauer wrote: Someone should probably invest some time into integrating the

Re: [DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?

This has gotten bad enough that people are dismissing legitimate test failures among the noise. On Thu, Sep 14, 2017 at 1:20 PM, Allen Wittenauer wrote: > Someone should probably invest some time into integrating the HBase > flaky test code a) into Yetus and

Re: [DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?

> On Sep 14, 2017, at 11:01 AM, Sean Busbey wrote: > >> Committers MUST check the qbt output after a commit. They MUST make sure > their commit didn’t break something new. > > How do we make this easier / more likely to happen? > > For example, I don't see any notice on

Re: [DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?

Thanks Sean busbey for starting the discussion.This was one of pain point even I notified to Allen couple of times.IIUC his point is committers and contributors should monitor qbt. IMHO,instead of post-commit try to fix in pre-commit only?? May be we can get dependcy module(dependency:list) and

Re: [DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?

> Committers MUST check the qbt output after a commit. They MUST make sure their commit didn’t break something new. How do we make this easier / more likely to happen? For example, I don't see any notice on HADOOP-14654 that the qbt post-commit failed. Is this a timing thing? Did Steve L just

[jira] [Resolved] (HADOOP-14647) Update third-party libraries for Hadoop 3

[ https://issues.apache.org/jira/browse/HADOOP-14647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang resolved HADOOP-14647. - Resolution: Done Fix Version/s: 3.0.0-beta1 Moved out unfinished subtasks to a later

[jira] [Resolved] (HADOOP-14654) Update httpclient version to 4.5.3

[ https://issues.apache.org/jira/browse/HADOOP-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang resolved HADOOP-14654. - Resolution: Won't Do Sorry about the breakage [~ste...@apache.org]. Going to close this w.r.t.

Re: [DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?

> On Sep 14, 2017, at 8:03 AM, Sean Busbey wrote: > > * HADOOP-14654 updated commons-httplient to a new patch release in > hadoop-project > * Precommit checked the modules that changed (i.e. not many) > * nightly had Azure support break due to a change in behavior.

[DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?

Moving discussion here from HADOOP-14654. Short synopsis: * HADOOP-14654 updated commons-httplient to a new patch release in hadoop-project * Precommit checked the modules that changed (i.e. not many) * nightly had Azure support break due to a change in behavior. Is this just the cost of our

[jira] [Reopened] (HADOOP-14868) regression: wasb tests failing

[ https://issues.apache.org/jira/browse/HADOOP-14868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened HADOOP-14868: - > regression: wasb tests failing > -- > > Key:

[jira] [Resolved] (HADOOP-14868) regression: wasb tests failing

[ https://issues.apache.org/jira/browse/HADOOP-14868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-14868. - Resolution: Fixed Fix Version/s: 3.1.0 3.0.0-beta1 reverted

[jira] [Reopened] (HADOOP-14654) Update httpclient version to 4.5.3

[ https://issues.apache.org/jira/browse/HADOOP-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened HADOOP-14654: - this breaks the azure tests; something which can be checked with a {{mvn test

[jira] [Created] (HADOOP-14868) regression: wasb tests failing

Steve Loughran created HADOOP-14868: --- Summary: regression: wasb tests failing Key: HADOOP-14868 URL: https://issues.apache.org/jira/browse/HADOOP-14868 Project: Hadoop Common Issue Type:

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/523/ [Sep 13, 2017 5:49:34 PM] (cliang) HADOOP-14804. correct wrong parameters format order in core-default.xml. [Sep 13, 2017 5:59:04 PM] (wang) HADOOP-14857. Fix downstream shaded client integration test.