Apache Hadoop qbt Report: trunk+JDK8 on Windows/x64

2018-04-05 Thread Apache Jenkins Server 
For more details, see https://builds.apache.org/job/hadoop-trunk-win/428/

[Apr 4, 2018 7:23:58 PM] (aw) YETUS-621. docker-cleanup doesn't support robot 
or sentinel correctly
[Apr 4, 2018 4:43:14 PM] (sunilg) YARN-8115. [UI2] URL data like 
nodeHTTPAddress must be encoded in UI
[Apr 4, 2018 5:51:58 PM] (kkaranasos) YARN-8013. Support application tags when 
defining application namespaces
[Apr 4, 2018 6:59:31 PM] (haibochen) YARN-7946. Update TimelineServerV2 doc as 
per YARN-7919. (Haibo Chen)
[Apr 4, 2018 10:08:03 PM] (vrushali) YARN-8073 TimelineClientImpl doesn't honor
[Apr 4, 2018 10:35:58 PM] (szegedim) HADOOP-14855. Hadoop scripts may errantly 
believe a daemon is still




-1 overall


The following subsystems voted -1:
compile mvninstall unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc javac


The following subsystems are considered long running:
(runtime bigger than 1h 00m 00s)
unit


Specific tests:

Failed junit tests :

   hadoop.crypto.TestCryptoStreamsWithOpensslAesCtrCryptoCodec 
   hadoop.fs.contract.rawlocal.TestRawlocalContractAppend 
   hadoop.fs.TestFileUtil 
   hadoop.fs.TestFsShellCopy 
   hadoop.fs.TestFsShellList 
   hadoop.fs.TestLocalFileSystem 
   hadoop.fs.TestRawLocalFileSystemContract 
   hadoop.fs.TestSymlinkLocalFSFileContext 
   hadoop.fs.TestTrash 
   hadoop.http.TestHttpServer 
   hadoop.http.TestHttpServerLogs 
   hadoop.io.nativeio.TestNativeIO 
   hadoop.ipc.TestIPC 
   hadoop.ipc.TestSocketFactory 
   hadoop.metrics2.impl.TestStatsDMetrics 
   hadoop.metrics2.sink.TestRollingFileSystemSinkWithLocal 
   hadoop.security.TestSecurityUtil 
   hadoop.security.TestShellBasedUnixGroupsMapping 
   hadoop.security.token.TestDtUtilShell 
   hadoop.util.TestNativeCodeLoader 
   hadoop.util.TestNodeHealthScriptRunner 
   hadoop.fs.TestResolveHdfsSymlink 
   hadoop.hdfs.client.impl.TestBlockReaderLocalLegacy 
   hadoop.hdfs.crypto.TestHdfsCryptoStreams 
   hadoop.hdfs.qjournal.client.TestQJMWithFaults 
   hadoop.hdfs.qjournal.client.TestQuorumJournalManager 
   hadoop.hdfs.qjournal.server.TestJournalNode 
   hadoop.hdfs.qjournal.server.TestJournalNodeSync 
   hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks 
   hadoop.hdfs.server.blockmanagement.TestNameNodePrunesMissingStorages 
   hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistLockedMemory 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistPolicy 
   
hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaPlacement 
   
hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyWriter 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestProvidedImpl 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestSpaceReservation 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestWriteToReplica 
   hadoop.hdfs.server.datanode.TestBlockPoolSliceStorage 
   hadoop.hdfs.server.datanode.TestBlockRecovery 
   hadoop.hdfs.server.datanode.TestBlockScanner 
   hadoop.hdfs.server.datanode.TestDataNodeFaultInjector 
   hadoop.hdfs.server.datanode.TestDataNodeMetrics 
   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure 
   hadoop.hdfs.server.datanode.TestDirectoryScanner 
   hadoop.hdfs.server.datanode.TestHSync 
   hadoop.hdfs.server.datanode.web.TestDatanodeHttpXFrame 
   hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand 
   hadoop.hdfs.server.diskbalancer.TestDiskBalancerRPC 
   hadoop.hdfs.server.mover.TestMover 
   hadoop.hdfs.server.mover.TestStorageMover 
   hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA 
   hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA 
   hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics 
   
hadoop.hdfs.server.namenode.snapshot.TestINodeFileUnderConstructionWithSnapshot 
   hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot 
   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots 
   hadoop.hdfs.server.namenode.snapshot.TestSnapRootDescendantDiff 
   hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport 
   hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength 
   hadoop.hdfs.server.namenode.TestAddBlock 
   hadoop.hdfs.server.namenode.TestAuditLoggerWithCommands 
   hadoop.hdfs.server.namenode.TestCheckpoint 
   hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate 
   hadoop.hdfs.server.namenode.TestEditLogRace 
   hadoop.hdfs.server.namenode.TestFileTruncate 
   hadoop.hdfs.server.namenode.TestFsck 
   hadoop.hdfs.serv

Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

2018-04-05 Thread Apache Jenkins Server 
For more details, see 
https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/186/

[Apr 5, 2018 6:45:01 PM] (haibochen) YARN-6936. [Atsv2] Retrospect storing 
entities into sub application
[Apr 5, 2018 8:23:07 PM] (weiy) HDFS-13353. RBF: 
TestRouterWebHDFSContractCreate failed. Contributed by




-1 overall


The following subsystems voted -1:
docker


Powered by Apache Yetus 0.8.0-SNAPSHOT   http://yetus.apache.org

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)

2018-04-05 Thread Gour Saha 
Sorry for the really late send.

+1 (non-binding)

-Verified checksums and sha1
-Built from source and ran YARN Service core and api tests successfully
-Installed in a 7-node cluster
-Launched, flexed up/down, stopped, started and destroyed the sample sleeper 
service
-Launched, flexed, stopped, started Hive/LLAP
-Tested the Service REST APIs and command line

-Gour

On 4/5/18, 10:49 AM, "Wangda Tan"  wrote:

Thanks everybody for voting!
The vote passes successfully with 11 binding +1 votes, 8 non-binding +1
votes and no -1s.

I will work on the staging and releases.

Best,
Wangda Tan


On Thu, Apr 5, 2018 at 10:46 AM, Vinod Kumar Vavilapalli  wrote:

> That is a great observation. And I missed your previous email about the
> shaded vs unshaded jars already getting fixed.
>
> I guess we are good to go.
>
> 
> --
>
> Looking at the RC. Went through my usual check-list. Here's my summary.
>
> Verification
> - [Check] Successful recompilation from source tar-ball
> - [Check] Signature verification
> -- Note: The format of the mds files changed a bit - not a biggie.
> -- For e.g, in 3.0.0 and 2.x releases, it has lines of the form "
> *hadoop-3.0.0-src.tar.gz*: SHA256 = 8B21AD79 50BD606B 2A7C91FB AE9FC279
> 7BCED50B B2600318 B7E0BE3A 74DFFF71"
> -- But in 3.1.0 RC it is, "
> */build/source/target/artifacts/hadoop-3.1.0.tar.gz*: SHA256 = 670D2CED
> 595FA42D 9FA1A93C 4E39B39F 47002CAD 1553D9DF 163EE828 CA5143E7"
> - [Check] Generating dist tarballs from source tar-ball
> - [Check] Testing
>-- Start NN, DN, RM, NM, JHS, Timeline Service
>-- Ran dist-shell example, MR sleep, wordcount, randomwriter, sort,
> grep, pi
>-- Tested CLIs to print nodes, apps etc and also navigated UIs
>
> +1 binding.
>
> Thanks
> +Vinod
>
> On Apr 3, 2018, at 8:13 PM, Wangda Tan  wrote:
>
> Hi Vinod / Arpit,
>
> I checked following versions:
> - 2.6.5 / 2.7.5 / 2.8.3 / 2.9.0 / 3.0.1:
>
> Jars in maven repo [1] are *always* different from jars in the binary
>
> tarball [2]: (I only checked hadoop-yarn-api-version.jar)
>
> (Following numbers are sizes of the jar)
> 2.6.5:
> - Jar in Maven: 1896185
> - Jar in tarball: 1891485
>
> 2.7.5:
> - Jar in Maven: 2039371 (md5: 15e76f7c734b49315ef2bce952509ddf)
> - Jar in tarball: 2039371 (md5: 0ef9f42f587401f5b49b39f27459f3ef)
> (Even size is same, md5 is different)
>
> 2.8.3:
> - Jar in Maven: 2451433
> - Jar in tarball: 2438975
>
> 2.9.0:
> - Jar in Maven: 2791477
> - Jar in tarball: 289
>
> 3.0.1:
> - Jar in Maven: 2852604
> - Jar in tarball: 2851373
>
> I guess the differences come from our release process.
>
> Thanks,
> Wangda
>
> [1] Maven jars are downloaded from
> https://repository.apache.org/service/local/repositories/
> releases/content/org/apache/hadoop/hadoop-yarn-api/
> /hadoop-yarn-api-.jar
> [2] Binary tarballs downloaded from http://apache.claz.org/hadoop/common/
>
>
> On Tue, Apr 3, 2018 at 4:25 PM, Vinod Kumar Vavilapalli <
> vino...@apache.org>
> wrote:
>
> We vote on the source code. The binaries are convenience artifacts.
>
> This is what I would do - (a) Just replace both the maven jars as well as
> the binaries to be consistent and correct. And then (b) Give a couple more
> days for folks who tested on the binaries to reverify - I count one such
> clear vote as of now.
>
> Thanks
> +Vinod
>
>
> On Apr 3, 2018, at 3:30 PM, Wangda Tan  wrote:
>
> HI Arpit,
>
> I think it won't match if we do rebuild. It should be fine as far as
> they're signed, correct? I don't see any policy doesn't allow this.
>
> Thanks,
> Wangda
>
>
> On Tue, Apr 3, 2018 at 9:33 AM, Arpit Agarwal 
> wrote:
>
> Thanks Wangda, I see the shaded jars now.
>
> Are the repo jars required to be the same as the binary release? They
> don’t match right now, probably they got rebuilt.
>
> +1 (binding), modulo that remaining question.
>
> * Verified signatures
> * Verified checksums for source and binary artefacts
> * Sanity checked jars on r.a.o.
> * Built from source
> * Deployed to 3 node secure cluster with NameNode HA
> * Verified HDFS web UIs
> * Tried out HDFS shell commands
> * Ran sample MapReduce jobs
>
> Thanks!
>
>
> --
> From: Wangda Tan 
> Date: Monday, April 2, 2018 at 9:25 PM
> To: Arpit Agarwal 
> Cc: Gera Shegalov , Sunil G , "

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)

2018-04-05 Thread Wangda Tan 
Thanks everybody for voting!
The vote passes successfully with 11 binding +1 votes, 8 non-binding +1
votes and no -1s.

I will work on the staging and releases.

Best,
Wangda Tan


On Thu, Apr 5, 2018 at 10:46 AM, Vinod Kumar Vavilapalli  wrote:

> That is a great observation. And I missed your previous email about the
> shaded vs unshaded jars already getting fixed.
>
> I guess we are good to go.
>
> 
> --
>
> Looking at the RC. Went through my usual check-list. Here's my summary.
>
> Verification
> - [Check] Successful recompilation from source tar-ball
> - [Check] Signature verification
> -- Note: The format of the mds files changed a bit - not a biggie.
> -- For e.g, in 3.0.0 and 2.x releases, it has lines of the form "
> *hadoop-3.0.0-src.tar.gz*: SHA256 = 8B21AD79 50BD606B 2A7C91FB AE9FC279
> 7BCED50B B2600318 B7E0BE3A 74DFFF71"
> -- But in 3.1.0 RC it is, "
> */build/source/target/artifacts/hadoop-3.1.0.tar.gz*: SHA256 = 670D2CED
> 595FA42D 9FA1A93C 4E39B39F 47002CAD 1553D9DF 163EE828 CA5143E7"
> - [Check] Generating dist tarballs from source tar-ball
> - [Check] Testing
>-- Start NN, DN, RM, NM, JHS, Timeline Service
>-- Ran dist-shell example, MR sleep, wordcount, randomwriter, sort,
> grep, pi
>-- Tested CLIs to print nodes, apps etc and also navigated UIs
>
> +1 binding.
>
> Thanks
> +Vinod
>
> On Apr 3, 2018, at 8:13 PM, Wangda Tan  wrote:
>
> Hi Vinod / Arpit,
>
> I checked following versions:
> - 2.6.5 / 2.7.5 / 2.8.3 / 2.9.0 / 3.0.1:
>
> Jars in maven repo [1] are *always* different from jars in the binary
>
> tarball [2]: (I only checked hadoop-yarn-api-version.jar)
>
> (Following numbers are sizes of the jar)
> 2.6.5:
> - Jar in Maven: 1896185
> - Jar in tarball: 1891485
>
> 2.7.5:
> - Jar in Maven: 2039371 (md5: 15e76f7c734b49315ef2bce952509ddf)
> - Jar in tarball: 2039371 (md5: 0ef9f42f587401f5b49b39f27459f3ef)
> (Even size is same, md5 is different)
>
> 2.8.3:
> - Jar in Maven: 2451433
> - Jar in tarball: 2438975
>
> 2.9.0:
> - Jar in Maven: 2791477
> - Jar in tarball: 289
>
> 3.0.1:
> - Jar in Maven: 2852604
> - Jar in tarball: 2851373
>
> I guess the differences come from our release process.
>
> Thanks,
> Wangda
>
> [1] Maven jars are downloaded from
> https://repository.apache.org/service/local/repositories/
> releases/content/org/apache/hadoop/hadoop-yarn-api/
> /hadoop-yarn-api-.jar
> [2] Binary tarballs downloaded from http://apache.claz.org/hadoop/common/
>
>
> On Tue, Apr 3, 2018 at 4:25 PM, Vinod Kumar Vavilapalli <
> vino...@apache.org>
> wrote:
>
> We vote on the source code. The binaries are convenience artifacts.
>
> This is what I would do - (a) Just replace both the maven jars as well as
> the binaries to be consistent and correct. And then (b) Give a couple more
> days for folks who tested on the binaries to reverify - I count one such
> clear vote as of now.
>
> Thanks
> +Vinod
>
>
> On Apr 3, 2018, at 3:30 PM, Wangda Tan  wrote:
>
> HI Arpit,
>
> I think it won't match if we do rebuild. It should be fine as far as
> they're signed, correct? I don't see any policy doesn't allow this.
>
> Thanks,
> Wangda
>
>
> On Tue, Apr 3, 2018 at 9:33 AM, Arpit Agarwal 
> wrote:
>
> Thanks Wangda, I see the shaded jars now.
>
> Are the repo jars required to be the same as the binary release? They
> don’t match right now, probably they got rebuilt.
>
> +1 (binding), modulo that remaining question.
>
> * Verified signatures
> * Verified checksums for source and binary artefacts
> * Sanity checked jars on r.a.o.
> * Built from source
> * Deployed to 3 node secure cluster with NameNode HA
> * Verified HDFS web UIs
> * Tried out HDFS shell commands
> * Ran sample MapReduce jobs
>
> Thanks!
>
>
> --
> From: Wangda Tan 
> Date: Monday, April 2, 2018 at 9:25 PM
> To: Arpit Agarwal 
> Cc: Gera Shegalov , Sunil G , "
> yarn-...@hadoop.apache.org" , Hdfs-dev <
> hdfs-...@hadoop.apache.org>, Hadoop Common ,
> "mapreduce-dev@hadoop.apache.org" ,
> Vinod Kumar Vavilapalli 
> Subject: Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)
>
> As pointed by Arpit, the previously deployed shared jars are incorrect.
> Just redeployed jars and staged. @Arpit, could you please check the updated
> Maven repo? https://repository.apache.org/content/repositories/
> orgapachehadoop-1092
>
> Since the jars inside binary tarballs are correct (
> http://people.apache.org/~wangda/hadoop-3.1.0-RC1/). I think we don't
> need roll another RC, just update Maven repo should be sufficient.
>
> Best,
> Wangda
>
>
> On Mon, Apr 2, 2018 at 2:39 PM, Wangda Tan 
> wrote:
> Hi Arpit,
>
> Thanks for pointing out this.
>
> I just removed all .md5 files from artifacts. I found md5 checksums still
> exist in .mds files and I didn't remove them from .mds file because it is
> generated by create-rel

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)

2018-04-05 Thread Vinod Kumar Vavilapalli 
That is a great observation. And I missed your previous email about the shaded 
vs unshaded jars already getting fixed.

I guess we are good to go.

--

Looking at the RC. Went through my usual check-list. Here's my summary.

Verification
- [Check] Successful recompilation from source tar-ball
- [Check] Signature verification
-- Note: The format of the mds files changed a bit - not a biggie.
-- For e.g, in 3.0.0 and 2.x releases, it has lines of the form 
"hadoop-3.0.0-src.tar.gz: SHA256 = 8B21AD79 50BD606B 2A7C91FB AE9FC279 7BCED50B 
B2600318 B7E0BE3A 74DFFF71"
-- But in 3.1.0 RC it is, 
"/build/source/target/artifacts/hadoop-3.1.0.tar.gz: SHA256 = 670D2CED 595FA42D 
9FA1A93C 4E39B39F 47002CAD 1553D9DF 163EE828 CA5143E7"
- [Check] Generating dist tarballs from source tar-ball
- [Check] Testing
   -- Start NN, DN, RM, NM, JHS, Timeline Service
   -- Ran dist-shell example, MR sleep, wordcount, randomwriter, sort, grep, pi
   -- Tested CLIs to print nodes, apps etc and also navigated UIs

+1 binding.

Thanks
+Vinod

> On Apr 3, 2018, at 8:13 PM, Wangda Tan  wrote:
> 
> Hi Vinod / Arpit,
> 
> I checked following versions:
> - 2.6.5 / 2.7.5 / 2.8.3 / 2.9.0 / 3.0.1:
> 
> Jars in maven repo [1] are *always* different from jars in the binary
> tarball [2]: (I only checked hadoop-yarn-api-version.jar)
> 
> (Following numbers are sizes of the jar)
> 2.6.5:
> - Jar in Maven: 1896185
> - Jar in tarball: 1891485
> 
> 2.7.5:
> - Jar in Maven: 2039371 (md5: 15e76f7c734b49315ef2bce952509ddf)
> - Jar in tarball: 2039371 (md5: 0ef9f42f587401f5b49b39f27459f3ef)
> (Even size is same, md5 is different)
> 
> 2.8.3:
> - Jar in Maven: 2451433
> - Jar in tarball: 2438975
> 
> 2.9.0:
> - Jar in Maven: 2791477
> - Jar in tarball: 289
> 
> 3.0.1:
> - Jar in Maven: 2852604
> - Jar in tarball: 2851373
> 
> I guess the differences come from our release process.
> 
> Thanks,
> Wangda
> 
> [1] Maven jars are downloaded from
> https://repository.apache.org/service/local/repositories/releases/content/org/apache/hadoop/hadoop-yarn-api/
> /hadoop-yarn-api-.jar
> [2] Binary tarballs downloaded from http://apache.claz.org/hadoop/common/
> 
> 
> On Tue, Apr 3, 2018 at 4:25 PM, Vinod Kumar Vavilapalli 
> wrote:
> 
>> We vote on the source code. The binaries are convenience artifacts.
>> 
>> This is what I would do - (a) Just replace both the maven jars as well as
>> the binaries to be consistent and correct. And then (b) Give a couple more
>> days for folks who tested on the binaries to reverify - I count one such
>> clear vote as of now.
>> 
>> Thanks
>> +Vinod
>> 
>> 
>> On Apr 3, 2018, at 3:30 PM, Wangda Tan  wrote:
>> 
>> HI Arpit,
>> 
>> I think it won't match if we do rebuild. It should be fine as far as
>> they're signed, correct? I don't see any policy doesn't allow this.
>> 
>> Thanks,
>> Wangda
>> 
>> 
>> On Tue, Apr 3, 2018 at 9:33 AM, Arpit Agarwal 
>> wrote:
>> 
>>> Thanks Wangda, I see the shaded jars now.
>>> 
>>> Are the repo jars required to be the same as the binary release? They
>>> don’t match right now, probably they got rebuilt.
>>> 
>>> +1 (binding), modulo that remaining question.
>>> 
>>> * Verified signatures
>>> * Verified checksums for source and binary artefacts
>>> * Sanity checked jars on r.a.o.
>>> * Built from source
>>> * Deployed to 3 node secure cluster with NameNode HA
>>> * Verified HDFS web UIs
>>> * Tried out HDFS shell commands
>>> * Ran sample MapReduce jobs
>>> 
>>> Thanks!
>>> 
>>> 
>>> --
>>> From: Wangda Tan 
>>> Date: Monday, April 2, 2018 at 9:25 PM
>>> To: Arpit Agarwal 
>>> Cc: Gera Shegalov , Sunil G , "
>>> yarn-...@hadoop.apache.org" , Hdfs-dev <
>>> hdfs-...@hadoop.apache.org>, Hadoop Common ,
>>> "mapreduce-dev@hadoop.apache.org" ,
>>> Vinod Kumar Vavilapalli 
>>> Subject: Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)
>>> 
>>> As pointed by Arpit, the previously deployed shared jars are incorrect.
>>> Just redeployed jars and staged. @Arpit, could you please check the updated
>>> Maven repo? https://repository.apache.org/content/repositories/
>>> orgapachehadoop-1092
>>> 
>>> Since the jars inside binary tarballs are correct (
>>> http://people.apache.org/~wangda/hadoop-3.1.0-RC1/). I think we don't
>>> need roll another RC, just update Maven repo should be sufficient.
>>> 
>>> Best,
>>> Wangda
>>> 
>>> 
>>> On Mon, Apr 2, 2018 at 2:39 PM, Wangda Tan 
>>> wrote:
>>> Hi Arpit,
>>> 
>>> Thanks for pointing out this.
>>> 
>>> I just removed all .md5 files from artifacts. I found md5 checksums still
>>> exist in .mds files and I didn't remove them from .mds file because it is
>>> generated by create-release script and Apache guidance is "should not"
>>> instead of "must not". Please let me know if you think they need to be
>>> removed as well.
>>> 
>>> - Wangda
>>> 
>

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)

2018-04-05 Thread Shane Kumpf 
Thanks for putting together the RC, Wangda! Sorry for the late addition.

+1 (non-binding)

- Verified checksums and signatures of the artifacts
- Deployed a pseudo distributed cluster using the binary tgz, source tgz,
and git tag
- Ran basic hdfs commands
- Ran pi, sleep, distributed shell, and a sleeper service using the default
and docker runtimes
- Ran dshell using the default and docker runtimes in a secure cluster
- Verified docker runtime and privileged containers can be disabled
- Verified YARN and HDFS UI's
- Validated the change log against the commits, all discrepancies are
documented or expected
- Basic validation of Registry DNS lookups

On Thu, Apr 5, 2018 at 12:26 AM, Naganarasimha Garla <
naganarasimha...@apache.org> wrote:

> +1 (binding)
>
> * Verified signatures
> * Verified checksums for source and binary artefacts
> * Built from source
> * Deployed a pseudo Hadoop cluster
> * Verified YARN & HDFS web UIs
> * Ran sample MapReduce jobs
> * Tested NodeLabels
>
> Regards,
> + Naga
>
> On Thu, Apr 5, 2018 at 2:59 AM, Suma Shivaprasad <
> sumasai.shivapra...@gmail.com> wrote:
>
> > +1 (non binding)
> >
> >
> > *Verified - User Group Queue mapping - Node labels with New UI- Dynamic
> > queuesThanksSuma*
> >
> >
> > On Wed, Apr 4, 2018 at 11:48 AM, Steve Loughran 
> > wrote:
> >
> > > that's "dangerously interesting". I think you are right, and I also
> think
> > > it'll just be the version files which get generated
> > >
> > > anyway, +1 binding
> > >
> > > * ran my new Hadoop-3 profile on spark (SPARK-23807), with the
> committer
> > > binding, then my downstream tests. All is well, provided you also have
> a
> > > spark hive JAR patched to accept hadoop 3 as a legitimate hadoop
> version.
> > > That's an ongoing issue in the Spark project. With that JAR on my CP my
> > > downstream tests were all happy (yesterday)
> > >
> > > * today the staging files seem to be missing, at least maven is unable
> to
> > > find them even when I turn the spark snapshots-and-staging profile on.
> > > That'll be the maven dist process at play, nothing else
> > >
> > > On 4 Apr 2018, at 04:13, Wangda Tan mailto:wh
> > > eele...@gmail.com>> wrote:
> > >
> > > Hi Vinod / Arpit,
> > >
> > > I checked following versions:
> > > - 2.6.5 / 2.7.5 / 2.8.3 / 2.9.0 / 3.0.1:
> > >
> > > Jars in maven repo [1] are *always* different from jars in the binary
> > > tarball [2]: (I only checked hadoop-yarn-api-version.jar)
> > >
> > > (Following numbers are sizes of the jar)
> > > 2.6.5:
> > > - Jar in Maven: 1896185
> > > - Jar in tarball: 1891485
> > >
> > > 2.7.5:
> > > - Jar in Maven: 2039371 (md5: 15e76f7c734b49315ef2bce952509ddf)
> > > - Jar in tarball: 2039371 (md5: 0ef9f42f587401f5b49b39f27459f3ef)
> > > (Even size is same, md5 is different)
> > >
> > > 2.8.3:
> > > - Jar in Maven: 2451433
> > > - Jar in tarball: 2438975
> > >
> > > 2.9.0:
> > > - Jar in Maven: 2791477
> > > - Jar in tarball: 289
> > >
> > > 3.0.1:
> > > - Jar in Maven: 2852604
> > > - Jar in tarball: 2851373
> > >
> > > I guess the differences come from our release process.
> > >
> > > Thanks,
> > > Wangda
> > >
> > > [1] Maven jars are downloaded from
> > > https://repository.apache.org/service/local/repositories/
> > > releases/content/org/apache/hadoop/hadoop-yarn-api/
> > > /hadoop-yarn-api-.jar
> > > [2] Binary tarballs downloaded from http://apache.claz.org/hadoop/
> > common/
> > >
> > >
> > > On Tue, Apr 3, 2018 at 4:25 PM, Vinod Kumar Vavilapalli <
> > > vino...@apache.org>
> > > wrote:
> > >
> > > We vote on the source code. The binaries are convenience artifacts.
> > >
> > > This is what I would do - (a) Just replace both the maven jars as well
> as
> > > the binaries to be consistent and correct. And then (b) Give a couple
> > more
> > > days for folks who tested on the binaries to reverify - I count one
> such
> > > clear vote as of now.
> > >
> > > Thanks
> > > +Vinod
> > >
> > >
> > > On Apr 3, 2018, at 3:30 PM, Wangda Tan mailto:wh
> > > eele...@gmail.com>> wrote:
> > >
> > > HI Arpit,
> > >
> > > I think it won't match if we do rebuild. It should be fine as far as
> > > they're signed, correct? I don't see any policy doesn't allow this.
> > >
> > > Thanks,
> > > Wangda
> > >
> > >
> > > On Tue, Apr 3, 2018 at 9:33 AM, Arpit Agarwal <
> aagar...@hortonworks.com<
> > > mailto:aagar...@hortonworks.com>>
> > > wrote:
> > >
> > > Thanks Wangda, I see the shaded jars now.
> > >
> > > Are the repo jars required to be the same as the binary release? They
> > > don’t match right now, probably they got rebuilt.
> > >
> > > +1 (binding), modulo that remaining question.
> > >
> > > * Verified signatures
> > > * Verified checksums for source and binary artefacts
> > > * Sanity checked jars on r.a.o.
> > > * Built from source
> > > * Deployed to 3 node secure cluster with NameNode HA
> > > * Verified HDFS web UIs
> > > * Tried out HDFS shell commands
> > > * Ran sample MapReduce jobs
> > >
> > > Thanks!
> > >
> > >
> >

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2018-04-05 Thread Apache Jenkins Server 
For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/742/

[Apr 4, 2018 3:52:35 AM] (sunilg) YARN-7764. Findbugs warning: 
Resource#getResources may expose internal
[Apr 4, 2018 4:06:24 AM] (wangda) YARN-8106. Update 
LogAggregationIndexedFileController to use readFull
[Apr 4, 2018 4:43:14 PM] (sunilg) YARN-8115. [UI2] URL data like 
nodeHTTPAddress must be encoded in UI
[Apr 4, 2018 5:51:58 PM] (kkaranasos) YARN-8013. Support application tags when 
defining application namespaces
[Apr 4, 2018 6:59:31 PM] (haibochen) YARN-7946. Update TimelineServerV2 doc as 
per YARN-7919. (Haibo Chen)
[Apr 4, 2018 10:08:03 PM] (vrushali) YARN-8073 TimelineClientImpl doesn't honor
[Apr 4, 2018 10:35:58 PM] (szegedim) HADOOP-14855. Hadoop scripts may errantly 
believe a daemon is still




-1 overall


The following subsystems voted -1:
unit xml


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Failed junit tests :

   hadoop.hdfs.TestLeaseRecovery2 
   hadoop.hdfs.web.TestWebHdfsTimeouts 
   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration 
   
hadoop.yarn.server.nodemanager.containermanager.scheduler.TestContainerSchedulerQueuing
 
   hadoop.yarn.applications.distributedshell.TestDistributedShell 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/742/artifact/out/diff-compile-cc-root.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/742/artifact/out/diff-compile-javac-root.txt
  [288K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/742/artifact/out/diff-checkstyle-root.txt
  [17M]

   pylint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/742/artifact/out/diff-patch-pylint.txt
  [24K]

   shellcheck:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/742/artifact/out/diff-patch-shellcheck.txt
  [20K]

   shelldocs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/742/artifact/out/diff-patch-shelldocs.txt
  [12K]

   whitespace:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/742/artifact/out/whitespace-eol.txt
  [9.2M]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/742/artifact/out/whitespace-tabs.txt
  [1.1M]

   xml:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/742/artifact/out/xml.txt
  [4.0K]

   javadoc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/742/artifact/out/diff-javadoc-javadoc-root.txt
  [760K]

   unit:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/742/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
  [296K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/742/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
  [48K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/742/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-distributedshell.txt
  [12K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/742/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
  [84K]

Powered by Apache Yetus 0.8.0-SNAPSHOT   http://yetus.apache.org

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org