Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

2017-12-12 Thread Ajay Kumar
+1 (non-binding) Thanks for driving this, Andrew Wang!! - downloaded the src tarball and verified md5 checksum - built from source with jdk 1.8.0_111-b14 - brought up a pseudo distributed cluster - did basic file system operations (mkdir, list, put, cat) and confirmed that everything was working

Re: [VOTE] Release Apache Hadoop 2.8.3 (RC0)

2017-12-12 Thread Kuhu Shukla
+1 non-binding. - Built and installed from source on a pseudo distributed cluster. - Ran sample jobs like wordcount, sleep etc. - Ran Tez 0.9 sample jobs. Regards, Kuhu On Mon, Dec 11, 2017 at 7:31 PM, Brahma Reddy Battula wrote: > +1 (non-binding), thanks Junping for driving this. > > > --

Re: [VOTE] Release Apache Hadoop 2.8.3 (RC0)

2017-12-12 Thread Jason Lowe
Thanks for driving this release, Junping! +1 (binding) - Verified signatures and digests - Successfully performed native build from source - Deployed a single-node cluster - Ran some test jobs and examined the logs Jason On Tue, Dec 5, 2017 at 3:58 AM, Junping Du wrote: > Hi all, > I've

Re: [VOTE] Release Apache Hadoop 2.8.3 (RC0)

2017-12-12 Thread Ajay Kumar
+1 (non-binding) Thanks for driving this, Junping!! - downloaded the src tarball and verified md5 checksum - built from source with jdk 1.8.0_111-b14 - brought up a pseudo distributed cluster - did basic file system operations (mkdir, list, put,rmr) and confirmed that everything was working - Run

Re: [VOTE] Release Apache Hadoop 2.7.5 (RC1)

2017-12-12 Thread Jason Lowe
Thanks for driving the release, Konstantin! +1 (binding) - Verified signatures and digests - Successfully performed a native build from source - Deployed a single-node cluster - Ran some sample jobs and checked the logs Jason On Thu, Dec 7, 2017 at 9:22 PM, Konstantin Shvachko wrote: > Hi ev

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

2017-12-12 Thread Wei-Chiu Chuang
Hi Andrew, thanks the tremendous effort. I found an empty "patchprocess" directory in the source tarball, that is not there if you clone from github. Any chance you might have some leftover trash when you made the tarball? Not wanting to nitpicking, but you might want to double check so we don't sh

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

2017-12-12 Thread Andrew Wang
Hi Wei-Chiu, The patchprocess directory is left over from the create-release process, and it looks empty to me. We should still file a create-release JIRA to fix this, but I think this is not a blocker. Would you agree? Best, Andrew On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang wrote: > Hi

Re: [VOTE] Release Apache Hadoop 2.8.3 (RC0)

2017-12-12 Thread Konstantin Shvachko
Downloaded again, now the checksums look good. Sorry my fault Thanks, --Konstantin On Mon, Dec 11, 2017 at 5:03 PM, Junping Du wrote: > Hi Konstantin, > > Thanks for verification and comments. I was verifying your example > below but found it is actually matched: > > > *jduMBP:hadoop-2.8.3

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

2017-12-12 Thread Lei Xu
+1 (binding) * Verified src tarball and bin tarball, verified md5 of each. * Build source with -Pdist,native * Started a pseudo cluster * Run ec -listPolicies / -getPolicy / -setPolicy on / , and run hdfs dfs put/get/cat on "/" with XOR-2-1 policy. Thanks Andrew for this great effort! Best, O

[jira] [Created] (HDFS-12918) EC Policy defaults incorrectly to enabled in protobufs

2017-12-12 Thread Zach Amsden (JIRA)
Zach Amsden created HDFS-12918: -- Summary: EC Policy defaults incorrectly to enabled in protobufs Key: HDFS-12918 URL: https://issues.apache.org/jira/browse/HDFS-12918 Project: Hadoop HDFS Issue

Re: [VOTE] Release Apache Hadoop 2.8.3 (RC0)

2017-12-12 Thread Eric Badger
Thanks, Junping +1 (non-binding) looks good from my end - Verified all hashes and checksums - Built from source on macOS 10.12.6, Java 1.8.0u65 - Deployed a pseudo cluster - Ran some example jobs Eric On Tue, Dec 12, 2017 at 12:55 PM, Konstantin Shvachko wrote: > Downloaded again, now the che

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

2017-12-12 Thread Andrew Wang
Hi everyone, As a reminder, this vote closes tomorrow at 12:31pm, so please give it a whack if you have time. There are already enough binding +1s to pass this vote, but it'd be great to get additional validation. Thanks to everyone who's voted thus far! Best, Andrew On Tue, Dec 12, 2017 at 1

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

2017-12-12 Thread Arun Suresh
+1 (binding) - Verified signatures of the source tarball. - built from source - using the docker build environment. - set up a pseudo-distributed test cluster. - ran basic HDFS commands - ran some basic MR jobs Cheers -Arun On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang wrote: > Hi everyone, > >

Re: [VOTE] Release Apache Hadoop 2.7.5 (RC1)

2017-12-12 Thread Eric Badger
Thanks, Konstantin. Everything looks good to me +1 (non-binding) - Verified all signatures and digests - Built from source on macOS 10.12.6, Java 1.8.0u65 - Deployed a pseudo cluster - Ran some example jobs Eric On Tue, Dec 12, 2017 at 11:01 AM, Jason Lowe wrote: > Thanks for driving the rele

[jira] [Resolved] (HDFS-4411) Asserts are disabled in unit tests

2017-12-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee resolved HDFS-4411. -- Resolution: Invalid > Asserts are disabled in unit tests > -- > >

[jira] [Created] (HDFS-12919) RBF: MR sets erasure coding by default

2017-12-12 Thread JIRA
Íñigo Goiri created HDFS-12919: -- Summary: RBF: MR sets erasure coding by default Key: HDFS-12919 URL: https://issues.apache.org/jira/browse/HDFS-12919 Project: Hadoop HDFS Issue Type: Sub-task

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

2017-12-12 Thread Elek, Marton
+1 (non-binding) * built from the source tarball (archlinux) / verified signature * Deployed to a kubernetes cluster (10/10 datanode/nodemanager pods) * Enabled ec on hdfs directory (hdfs cli) * Started example yarn jobs (pi/terragen) * checked yarn ui/ui2 Thanks for all the efforts. Marto

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

2017-12-12 Thread Iñigo Goiri
+1 (non-binding) I tested it in a deployment with 24 nodes across 8 subclusters. Tested a few jobs reading and writing data through HDFS Router-based federation. However, jobs failed to run when setting RBF as the default filesystem because after MAPREDUCE-6954, it tries to invoke setErasureCoding

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

2017-12-12 Thread Robert Kanter
+1 (binding) + Downloaded the binary release + Deployed on a 3 node cluster on CentOS 7.3 + Ran some MR jobs, clicked around the UI, etc + Ran some CLI commands (yarn logs, etc) Good job everyone on Hadoop 3! - Robert On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh wrote: > +1 (binding) > > - V

Re: [VOTE] Release Apache Hadoop 2.8.3 (RC0)

2017-12-12 Thread John Zhuge
Thanks Junping for the great effort! - Verified checksums and signatures of all tarballs - Built source with native, Azul Java 1.7.0_161 on Mac OS X 10.13.2 - Verified cloud connectors: - All S3A integration tests - Deployed both binary and built source to a pseudo cluster, pass

[jira] [Created] (HDFS-12920) HDFS default value change (with adding time unit) breaks old version MR tarball work with new version (3.0) of hadoop

2017-12-12 Thread Junping Du (JIRA)
Junping Du created HDFS-12920: - Summary: HDFS default value change (with adding time unit) breaks old version MR tarball work with new version (3.0) of hadoop Key: HDFS-12920 URL: https://issues.apache.org/jira/browse

Re: [VOTE] Release Apache Hadoop 2.8.3 (RC0)

2017-12-12 Thread Chandni Singh
+1 Built the source code. Deployed a single node cluster and ran example jobs. Also tested RM recovery. Thanks, Chandni On Tue, Dec 12, 2017 at 4:37 PM, Jian He wrote: > +1 > I built from source code > deployed a cluster > and successfully ran jobs while restarting RM as well > > Jian > > > > O

[jira] [Created] (HDFS-12921) DFS.setReplication should throw IOE on EC files

2017-12-12 Thread Lei (Eddy) Xu (JIRA)
Lei (Eddy) Xu created HDFS-12921: Summary: DFS.setReplication should throw IOE on EC files Key: HDFS-12921 URL: https://issues.apache.org/jira/browse/HDFS-12921 Project: Hadoop HDFS Issue Typ

[jira] [Resolved] (HDFS-12918) NameNode fails to start after upgrade - Missing state in ECPolicy Proto

2017-12-12 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy resolved HDFS-12918. --- Resolution: Won't Fix We have an upgrade incompatible fix landed in C6 at e565b5277d5

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

2017-12-12 Thread Junping Du
Thanks Andrew for pushing new RC for 3.0.0. I was out last week, just get chance to validate new RC now. Basically, I found two critical issues with the same rolling upgrade scenario as where HADOOP-15059 get found previously: HDFS-12920, we changed value format for some hdfs configurations that

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

2017-12-12 Thread Jonathan Hung
Thanks Andrew for the huge effort. +1 (non-binding) - Downloaded binary tarball and verified md5 - Ran RM HA and verified manual failover - Verified add/remove/update scheduler configuration API (CLI/REST) works for leveldb/zookeeper backend - Verified scheduler configuration changes persisted on

Re: [VOTE] Release Apache Hadoop 2.8.3 (RC0)

2017-12-12 Thread Wangda Tan
+1 (Binding). Built from source and deployed a cluster, jobs can run successfully. Thanks Junping for driving this. Best, Wangda On Tue, Dec 12, 2017 at 5:18 PM, Chandni Singh wrote: > +1 > Built the source code. > Deployed a single node cluster and ran example jobs. Also tested RM > recover

[jira] [Created] (HDFS-12922) Arrays of length 1 cause 9.2% memory overhead

2017-12-12 Thread Misha Dmitriev (JIRA)
Misha Dmitriev created HDFS-12922: - Summary: Arrays of length 1 cause 9.2% memory overhead Key: HDFS-12922 URL: https://issues.apache.org/jira/browse/HDFS-12922 Project: Hadoop HDFS Issue Typ

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2017-12-12 Thread Apache Jenkins Server
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/620/ [Dec 11, 2017 6:58:02 PM] (surendralilhore) HDFS-12833. Distcp : Update the usage of delete option for dependency [Dec 11, 2017 10:00:42 PM] (rkanter) MAPREDUCE-7018. Apply erasure coding properly to frame

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

2017-12-12 Thread Sangjin Lee
+1 (binding) - downloaded the binary tarball and the source tarball and checked signatures - verified that the source builds cleanly - verified that the shaded client jars are correct - checked the basic pseudo-distributed cluster set-up and checked UI and logs (hdfs and YARN) - ran some test jobs

[jira] [Created] (HDFS-12923) DFS.concat should throw exception if files have different EC policies.

2017-12-12 Thread Lei (Eddy) Xu (JIRA)
Lei (Eddy) Xu created HDFS-12923: Summary: DFS.concat should throw exception if files have different EC policies. Key: HDFS-12923 URL: https://issues.apache.org/jira/browse/HDFS-12923 Project: Hadoop

Re: [VOTE] Release Apache Hadoop 2.7.5 (RC1)

2017-12-12 Thread John Zhuge
Thanks Konstantin for the great effort! +1 (binding) - Verified checksums and signatures of all tarballs - Built source with native, Azul Java 1.7.0_161 on Mac OS X 10.13.2 - Verified cloud connectors: - All S3A integration tests - Deployed both binary and built source to a pse

Re: [VOTE] Release Apache Hadoop 2.8.3 (RC0)

2017-12-12 Thread John Zhuge
Oops, the vote was meant for 2.7.5. Sorry for the confusion. My 2.8.3 vote coming up shortly. On Tue, Dec 12, 2017 at 4:28 PM, John Zhuge wrote: > Thanks Junping for the great effort! > > >- Verified checksums and signatures of all tarballs >- Built source with native, Azul Java 1.7.0_

Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

2017-12-12 Thread Apache Jenkins Server
For more details, see https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/67/ [Dec 11, 2017 11:17:17 PM] (inigoiri) HDFS-12875. RBF: Complete logic for -readonly option of dfsrouteradmin -1 overall The following subsystems voted -1: unit xml The following subsystems vote

[jira] [Resolved] (HDFS-12921) DFS.setReplication should throw exception on EC files

2017-12-12 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu resolved HDFS-12921. -- Resolution: Won't Fix Fix Version/s: 3.0.0 It was a no-op in {{FSDirAttrOp#unprotectedSet