Re: [VOTE] Release Apache Hadoop 3.0.0-beta1 RC0

[Xiao Chen]

Thanks Andrew!

+1 (binding)

   - Verified md5 matches
   - Built from src tarball
   - Started pseudo-distributed hdfs cluster with kms, ran basic operations
   - Sanity checked logs and NN webui

P.S. Obviously Andrew meant to say Oct. 3rd, not November. :)

-Xiao

On Thu, Sep 28, 2017 at 5:04 PM, Andrew Wang 
wrote:

> Hi all,
>
> Let me start, as always, by thanking the many, many contributors who helped
> with this release! I've prepared an RC0 for 3.0.0-beta1:
>
> http://home.apache.org/~wang/3.0.0-beta1-RC0/
>
> This vote will run five days, ending on Nov 3rd at 5PM Pacific.
>
> beta1 contains 576 fixed JIRA issues comprising a number of bug fixes,
> improvements, and feature enhancements. Notable additions include the
> addition of YARN Timeline Service v2 alpha2, S3Guard, completion of the
> shaded client, and HDFS erasure coding pluggable policy support.
>
> I've done the traditional testing of running a Pi job on a pseudo cluster.
> My +1 to start.
>
> We're working internally on getting this run through our integration test
> rig. I'm hoping Vijay or Ray can ring in with a +1 once that's complete.
>
> Best,
> Andrew
>

[VOTE] Release Apache Hadoop 3.0.0-beta1 RC0

[Andrew Wang]

Hi all,

Let me start, as always, by thanking the many, many contributors who helped
with this release! I've prepared an RC0 for 3.0.0-beta1:

http://home.apache.org/~wang/3.0.0-beta1-RC0/

This vote will run five days, ending on Nov 3rd at 5PM Pacific.

beta1 contains 576 fixed JIRA issues comprising a number of bug fixes,
improvements, and feature enhancements. Notable additions include the
addition of YARN Timeline Service v2 alpha2, S3Guard, completion of the
shaded client, and HDFS erasure coding pluggable policy support.

I've done the traditional testing of running a Pi job on a pseudo cluster.
My +1 to start.

We're working internally on getting this run through our integration test
rig. I'm hoping Vijay or Ray can ring in with a +1 once that's complete.

Best,
Andrew

[DISCUSS] Merging API-based scheduler configuration to trunk/branch-2

[Jonathan Hung]

Hi everyone,

Starting this thread to discuss merging API-based scheduler configuration
to trunk/branch-2. The feature adds the framework for allowing users to
modify scheduler configuration via REST or CLI using a configurable backend
(leveldb/zk are currently supported), and adds capacity scheduler support
for this. The umbrella JIRA is YARN-5734. All the required work for this
feature is done and committed to branch YARN-5734, and a full diff has been
generated at YARN-7241.

Regarding compatibility, this feature is configurable and turned off by
default.

The feature has been tested locally on a couple RMs (since it is an RM only
change), with queue addition/removal/updates tested on single RM (leveldb)
and two RMs (zk). Also we verified the original configuration update
mechanism (via refreshQueues) is unaffected when the feature is off/not
configured.

Our original plan was to merge this to trunk (which is what the YARN-7241
diff is based on), and port to branch-2 before the 2.9 release. @Andrew,
what are your thoughts on also merging this to branch-3.0?

Thanks!

Jonathan Hung

Re: Heads up: branching branch-3.0.0-beta1 off of branch-3.0

[Andrew Wang]

Branch has been cut, branch-3.0 is now open for commits for 3.0.0 GA.

HEAD of branch-3.0.0-beta1 is 2223393ad1d5ffdd62da79e1546de79c6259dc12.

On Thu, Sep 28, 2017 at 10:52 AM, Andrew Wang 
wrote:

> Hi folks,
>
> We've driven the blocker count down to 0, and I went through and made sure
> the fix versions and release notes and so on are all lined up.
>
> I'm going to cut branch-3.0.0-beta1 off branch-3.0 and try and get RC0 out
> today.
>
> Cheers,
> Andrew
>

Heads up: branching branch-3.0.0-beta1 off of branch-3.0

[Andrew Wang]

Hi folks,

We've driven the blocker count down to 0, and I went through and made sure
the fix versions and release notes and so on are all lined up.

I'm going to cut branch-3.0.0-beta1 off branch-3.0 and try and get RC0 out
today.

Cheers,
Andrew

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/541/

[Sep 26, 2017 4:29:22 PM] (brahma) HDFS-5040.Audit log for admin commands/ 
logging output of all DFS admin
[Sep 26, 2017 5:08:16 PM] (lei) HDFS-12523. Thread pools in ErasureCodingWorker 
do not shutdown.
[Sep 26, 2017 7:52:27 PM] (jlowe) MAPREDUCE-6968. Staging directory erasure 
coding config property has a
[Sep 26, 2017 8:15:24 PM] (daryn) [HDFS-12386] Add fsserver defaults call to 
WebhdfsFileSystem. (Rushabh
[Sep 27, 2017 5:25:07 AM] (aajisaka) HADOOP-14822. hadoop-project/pom.xml is 
executable. Contributed by Ajay
[Sep 27, 2017 9:07:32 AM] (sunilg) YARN-6871. Add additional deSelects params in
[Sep 27, 2017 6:32:09 PM] (ctrezzo) YARN-7253. Shared Cache Manager daemon 
command listed as admin subcmd in
[Sep 27, 2017 10:46:18 PM] (xgong) YARN-7257. AggregatedLogsBlock reports a bad 
'end' value as a bad




-1 overall


The following subsystems voted -1:
unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Failed junit tests :

   hadoop.net.TestDNS 
   hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks 
   hadoop.hdfs.server.datanode.TestDirectoryScanner 
   hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM 
   hadoop.hdfs.server.namenode.TestReencryptionWithKMS 
   
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation 
   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppStarvation 
   
hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy 
   hadoop.yarn.server.TestContainerManagerSecurity 
   hadoop.yarn.sls.TestReservationSystemInvariants 
   hadoop.yarn.sls.TestSLSRunner 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/541/artifact/out/diff-compile-cc-root.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/541/artifact/out/diff-compile-javac-root.txt
  [292K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/541/artifact/out/diff-checkstyle-root.txt
  [17M]

   pylint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/541/artifact/out/diff-patch-pylint.txt
  [20K]

   shellcheck:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/541/artifact/out/diff-patch-shellcheck.txt
  [20K]

   shelldocs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/541/artifact/out/diff-patch-shelldocs.txt
  [12K]

   whitespace:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/541/artifact/out/whitespace-eol.txt
  [11M]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/541/artifact/out/whitespace-tabs.txt
  [1.2M]

   javadoc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/541/artifact/out/diff-javadoc-javadoc-root.txt
  [1.9M]

   unit:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/541/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt
  [152K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/541/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
  [340K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/541/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
  [64K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/541/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
  [12K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/541/artifact/out/patch-unit-hadoop-tools_hadoop-sls.txt
  [20K]

Powered by Apache Yetus 0.6.0-SNAPSHOT   http://yetus.apache.org

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org