Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)
Thanks Wangda for initiating the release. I tested this RC built from source file. - Tested MR apps (sleep, wc) and verified both new YARN UI and old RM UI. - Below feature sanity is done - Application priority - Application timeout - Intra Queue preemption with priority based - DS based affinity tests to verify placement constraints. - Tested basic NodeLabel scenarios. - Added couple of labels to few of nodes and behavior is coming correct. - Verified old UI and new YARN UI for labels. - Submitted apps to labelled cluster and it works fine. - Also performed few cli commands related to nodelabel. - Test basic HA cases and seems correct. - Tested new YARN UI . All pages are getting loaded correctly. - Sunil On Fri, Mar 30, 2018 at 9:45 AM Wangda Tan wrote: > Hi folks, > > Thanks to the many who helped with this release since Dec 2017 [1]. We've > created RC1 for Apache Hadoop 3.1.0. The artifacts are available here: > > http://people.apache.org/~wangda/hadoop-3.1.0-RC1 > > The RC tag in git is release-3.1.0-RC1. Last git commit SHA is > 16b70619a24cdcf5d3b0fcf4b58ca77238ccbe6d > > The maven artifacts are available via repository.apache.org at > https://repository.apache.org/content/repositories/orgapachehadoop-1090/ > This vote will run 5 days, ending on Apr 3 at 11:59 pm Pacific. > > 3.1.0 contains 766 [2] fixed JIRA issues since 3.0.0. Notable additions > include the first class GPU/FPGA support on YARN, Native services, Support > rich placement constraints in YARN, S3-related enhancements, allow HDFS > block replicas to be provided by an external storage system, etc. > > For 3.1.0 RC0 vote discussion, please see [3]. > > We’d like to use this as a starting release for 3.1.x [1], depending on how > it goes, get it stabilized and potentially use a 3.1.1 in several weeks as > the stable release. > > We have done testing with a pseudo cluster: > - Ran distributed job. > - GPU scheduling/isolation. > - Placement constraints (intra-application anti-affinity) by using > distributed shell. > > My +1 to start. > > Best, > Wangda/Vinod > > [1] > > https://lists.apache.org/thread.html/b3fb3b6da8b6357a68513a6dfd104bc9e19e559aedc5ebedb4ca08c8@%3Cyarn-dev.hadoop.apache.org%3E > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.1.0) > AND fixVersion not in (3.0.0, 3.0.0-beta1) AND status = Resolved ORDER BY > fixVersion ASC > [3] > > https://lists.apache.org/thread.html/b3a7dc075b7329fd660f65b48237d72d4061f26f83547e41d0983ea6@%3Cyarn-dev.hadoop.apache.org%3E >
Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)
Hi Wangda +1 (non-binding) - Smoke tests with teragen/terasort/DS jobs - Various of metrics, UI displays validation, compatible tests - Tested GPU resource-type - Verified RM fail-over, app-recovery - Verified 2 threads async-scheduling - Enabled placement constraints, tested affinity/anti-affinity allocations - SLS tests -- Weiwei On 2 Apr 2018, 1:13 PM +0800, Brahma Reddy Battula , wrote: Wangda thanks for driving this. +1(binding) --Built from source --Installed HA cluster --Verified Basic Shell commands --Ran Sample Jobs --Browsed the UI's. On Fri, Mar 30, 2018 at 9:45 AM, Wangda Tan wrote: Hi folks, Thanks to the many who helped with this release since Dec 2017 [1]. We've created RC1 for Apache Hadoop 3.1.0. The artifacts are available here: http://people.apache.org/~wangda/hadoop-3.1.0-RC1 The RC tag in git is release-3.1.0-RC1. Last git commit SHA is 16b70619a24cdcf5d3b0fcf4b58ca77238ccbe6d The maven artifacts are available via repository.apache.org at https://repository.apache.org/content/repositories/orgapachehadoop-1090/ This vote will run 5 days, ending on Apr 3 at 11:59 pm Pacific. 3.1.0 contains 766 [2] fixed JIRA issues since 3.0.0. Notable additions include the first class GPU/FPGA support on YARN, Native services, Support rich placement constraints in YARN, S3-related enhancements, allow HDFS block replicas to be provided by an external storage system, etc. For 3.1.0 RC0 vote discussion, please see [3]. We’d like to use this as a starting release for 3.1.x [1], depending on how it goes, get it stabilized and potentially use a 3.1.1 in several weeks as the stable release. We have done testing with a pseudo cluster: - Ran distributed job. - GPU scheduling/isolation. - Placement constraints (intra-application anti-affinity) by using distributed shell. My +1 to start. Best, Wangda/Vinod [1] https://lists.apache.org/thread.html/b3fb3b6da8b6357a68513a6dfd104b c9e19e559aedc5ebedb4ca08c8@%3Cyarn-dev.hadoop.apache.org%3E [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.1.0) AND fixVersion not in (3.0.0, 3.0.0-beta1) AND status = Resolved ORDER BY fixVersion ASC [3] https://lists.apache.org/thread.html/b3a7dc075b7329fd660f65b48237d7 2d4061f26f83547e41d0983ea6@%3Cyarn-dev.hadoop.apache.org%3E
Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)
Wangda thanks for driving this. +1(binding) --Built from source --Installed HA cluster --Verified Basic Shell commands --Ran Sample Jobs --Browsed the UI's. On Fri, Mar 30, 2018 at 9:45 AM, Wangda Tan wrote: > Hi folks, > > Thanks to the many who helped with this release since Dec 2017 [1]. We've > created RC1 for Apache Hadoop 3.1.0. The artifacts are available here: > > http://people.apache.org/~wangda/hadoop-3.1.0-RC1 > > The RC tag in git is release-3.1.0-RC1. Last git commit SHA is > 16b70619a24cdcf5d3b0fcf4b58ca77238ccbe6d > > The maven artifacts are available via repository.apache.org at > https://repository.apache.org/content/repositories/orgapachehadoop-1090/ > This vote will run 5 days, ending on Apr 3 at 11:59 pm Pacific. > > 3.1.0 contains 766 [2] fixed JIRA issues since 3.0.0. Notable additions > include the first class GPU/FPGA support on YARN, Native services, Support > rich placement constraints in YARN, S3-related enhancements, allow HDFS > block replicas to be provided by an external storage system, etc. > > For 3.1.0 RC0 vote discussion, please see [3]. > > We’d like to use this as a starting release for 3.1.x [1], depending on how > it goes, get it stabilized and potentially use a 3.1.1 in several weeks as > the stable release. > > We have done testing with a pseudo cluster: > - Ran distributed job. > - GPU scheduling/isolation. > - Placement constraints (intra-application anti-affinity) by using > distributed shell. > > My +1 to start. > > Best, > Wangda/Vinod > > [1] > https://lists.apache.org/thread.html/b3fb3b6da8b6357a68513a6dfd104b > c9e19e559aedc5ebedb4ca08c8@%3Cyarn-dev.hadoop.apache.org%3E > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.1.0) > AND fixVersion not in (3.0.0, 3.0.0-beta1) AND status = Resolved ORDER BY > fixVersion ASC > [3] > https://lists.apache.org/thread.html/b3a7dc075b7329fd660f65b48237d7 > 2d4061f26f83547e41d0983ea6@%3Cyarn-dev.hadoop.apache.org%3E >
Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/182/ No changes -1 overall The following subsystems voted -1: docker Powered by Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Windows/x64
For more details, see https://builds.apache.org/job/hadoop-trunk-win/424/ [Apr 1, 2018 10:26:38 AM] (aajisaka) YETUS-619. audience-annottaions-jdiff doclet does not work on Java 9 [Mar 31, 2018 4:46:13 PM] (inigoiri) HDFS-13289. RBF: TestConnectionManager#testCleanup() test case need -1 overall The following subsystems voted -1: maven Powered by Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/738/ [Mar 31, 2018 1:55:35 AM] (shv) HADOOP-15253. Should update maxQueueSize when refresh call queue. [Mar 31, 2018 4:46:13 PM] (inigoiri) HDFS-13289. RBF: TestConnectionManager#testCleanup() test case need -1 overall The following subsystems voted -1: findbugs unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: FindBugs : module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api org.apache.hadoop.yarn.api.records.Resource.getResources() may expose internal representation by returning Resource.resources At Resource.java:by returning Resource.resources At Resource.java:[line 234] Failed junit tests : hadoop.conf.TestCommonConfigurationFields hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy hadoop.hdfs.web.TestWebHdfsTimeouts hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting hadoop.hdfs.server.namenode.TestDecommissioningStatus cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/738/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/738/artifact/out/diff-compile-javac-root.txt [288K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/738/artifact/out/diff-checkstyle-root.txt [17M] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/738/artifact/out/diff-patch-pylint.txt [24K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/738/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/738/artifact/out/diff-patch-shelldocs.txt [12K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/738/artifact/out/whitespace-eol.txt [9.2M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/738/artifact/out/whitespace-tabs.txt [1.1M] xml: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/738/artifact/out/xml.txt [4.0K] findbugs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/738/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-warnings.html [8.0K] javadoc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/738/artifact/out/diff-javadoc-javadoc-root.txt [760K] unit: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/738/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt [160K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/738/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [308K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/738/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt [84K] Powered by Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
