date:20180410

Apache Hadoop qbt Report: trunk+JDK8 on Windows/x64

2018-04-10 Thread Apache Jenkins Server

For more details, see https://builds.apache.org/job/hadoop-trunk-win/433/

[Apr 9, 2018 3:47:22 PM] (sunilg) YARN-7574. Add support for Node Labels on 
Auto Created Leaf Queue
[Apr 9, 2018 4:16:48 PM] (inigoiri) HDFS-13388. RequestHedgingProxyProvider 
calls multiple configured NNs
[Apr 9, 2018 5:09:25 PM] (inigoiri) HDFS-13380. RBF: mv/rm fail after the 
directory exceeded the quota
[Apr 9, 2018 5:48:42 PM] (james.clampffer) HDFS-13376. Specify minimum GCC 
version to avoid TLS support error in
[Apr 9, 2018 10:19:21 PM] (jlowe) YARN-7667. Docker Stop grace period should be 
configurable. Contributed
[Apr 9, 2018 11:37:49 PM] (Bharat) HADOOP-15328. Fix the typo in 
HttpAuthentication.md. Contributed by fang
[Apr 10, 2018 6:29:28 AM] (yqlin) HDFS-13410. RBF: Support federation with no 
subclusters. Contributed by
[Apr 10, 2018 7:34:42 AM] (yqlin) HDFS-13384. RBF: Improve timeout RPC call 
mechanism. Contributed by
[Apr 10, 2018 9:59:40 AM] (aajisaka) HADOOP-15374. Add links of the new 
features of 3.1.0 to the top page
[Apr 10, 2018 10:39:09 AM] (sunilg) YARN-7804. [UI2] Refresh action on Grid 
view page should not be




-1 overall


The following subsystems voted -1:
compile mvninstall unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc javac


The following subsystems are considered long running:
(runtime bigger than 1h 00m 00s)
unit


Specific tests:

Failed junit tests :

   hadoop.crypto.TestCryptoStreamsWithOpensslAesCtrCryptoCodec 
   hadoop.fs.contract.rawlocal.TestRawlocalContractAppend 
   hadoop.fs.TestFileUtil 
   hadoop.fs.TestFsShellCopy 
   hadoop.fs.TestFsShellList 
   hadoop.fs.TestLocalFileSystem 
   hadoop.fs.TestRawLocalFileSystemContract 
   hadoop.fs.TestTrash 
   hadoop.fs.viewfs.TestViewFileSystemWithAuthorityLocalFileSystem 
   hadoop.http.TestHttpServer 
   hadoop.http.TestHttpServerLogs 
   hadoop.io.nativeio.TestNativeIO 
   hadoop.ipc.TestIPC 
   hadoop.ipc.TestSocketFactory 
   hadoop.metrics2.impl.TestStatsDMetrics 
   hadoop.metrics2.sink.TestRollingFileSystemSinkWithLocal 
   hadoop.security.TestSecurityUtil 
   hadoop.security.TestShellBasedUnixGroupsMapping 
   hadoop.security.token.TestDtUtilShell 
   hadoop.util.TestNativeCodeLoader 
   hadoop.fs.TestResolveHdfsSymlink 
   hadoop.hdfs.client.impl.TestBlockReaderLocalLegacy 
   hadoop.hdfs.crypto.TestHdfsCryptoStreams 
   hadoop.hdfs.qjournal.client.TestQuorumJournalManager 
   hadoop.hdfs.qjournal.server.TestJournalNode 
   hadoop.hdfs.qjournal.server.TestJournalNodeSync 
   hadoop.hdfs.server.balancer.TestBalancer 
   hadoop.hdfs.server.balancer.TestBalancerRPCDelay 
   hadoop.hdfs.server.blockmanagement.TestNameNodePrunesMissingStorages 
   hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistLockedMemory 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistPolicy 
   
hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaPlacement 
   
hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyWriter 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestProvidedImpl 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestSpaceReservation 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestWriteToReplica 
   hadoop.hdfs.server.datanode.TestBlockPoolSliceStorage 
   hadoop.hdfs.server.datanode.TestBlockRecovery 
   hadoop.hdfs.server.datanode.TestBlockScanner 
   hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics 
   hadoop.hdfs.server.datanode.TestDataNodeFaultInjector 
   hadoop.hdfs.server.datanode.TestDataNodeMetrics 
   hadoop.hdfs.server.datanode.TestDataNodeUUID 
   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure 
   hadoop.hdfs.server.datanode.TestDirectoryScanner 
   hadoop.hdfs.server.datanode.TestHSync 
   hadoop.hdfs.server.datanode.web.TestDatanodeHttpXFrame 
   hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand 
   hadoop.hdfs.server.diskbalancer.TestDiskBalancer 
   hadoop.hdfs.server.diskbalancer.TestDiskBalancerRPC 
   hadoop.hdfs.server.mover.TestMover 
   hadoop.hdfs.server.mover.TestStorageMover 
   hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA 
   hadoop.hdfs.server.namenode.ha.TestDNFencing 
   hadoop.hdfs.server.namenode.ha.TestHAAppend 
   hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA 
   hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics 
   
hadoop.hdfs.server.namenode.snapshot.TestINodeFileUnderConstructionWithSnapshot

[jira] [Created] (MAPREDUCE-7076) TestNNBench#testNNBenchCreateReadAndDelete failing in our internal build

2018-04-10 Thread Rushabh S Shah (JIRA)

Rushabh S Shah created MAPREDUCE-7076:
-

 Summary: TestNNBench#testNNBenchCreateReadAndDelete failing in our 
internal build
 Key: MAPREDUCE-7076
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7076
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: test
Affects Versions: 2.8.0
Reporter: Rushabh S Shah


TestNNBench#testNNBenchCreateReadAndDelete failed couple of times in our 
internal jenkins build.
{noformat}
java.lang.AssertionError: create_write should create the file
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at 
org.apache.hadoop.hdfs.TestNNBench.testNNBenchCreateReadAndDelete(TestNNBench.java:55)
{noformat}

Below is my analysis for why it didn't create the file.
{code:title=NNBench.java|borderStyle=solid}
// Some comments here
  public void map(Text key, 
LongWritable value,
OutputCollector output,
Reporter reporter) throws IOException {
  if (barrier()) {
String fileName = "file_" + value;
if (op.equals(OP_CREATE_WRITE)) {
  startTimeTPmS = System.currentTimeMillis();
  doCreateWriteOp(fileName, reporter);
} ...
  } else {
output.collect(new Text("l:latemaps"), new Text("1"));
  }
  // Below are the relevant parts of barrier() method
  private boolean barrier() {
..
// If the sleep time is greater than 0, then sleep and return
...
LOG.info("Waiting in barrier for: " + sleepTime + " ms");
return retVal;
  }
  // Below are the relevant parts of the doCreateWriteOp
  private void doCreateWriteOp(String name,
 Reporter reporter) {
FSDataOutputStream out;
byte[] buffer = new byte[bytesToWrite];  
for (long l = 0l; l < numberOfFiles; l++) {
  Path filePath = new Path(new Path(baseDir, dataDirName), 
  name + "_" + l);
}
  
  }
{code}   

This file {{BASE_DIR/data/file_0_0}} is getting created only if the map task 
starts before the time mentioned by {{startTime}}.
Refer the chunk which I pasted above.
{{map(..)}} --> {{barrier()}} and *only if* {{barrier()}} evaluates to true it 
will call {{doCreateWriteOp}} which will eventually create the file.
In test case, the delay value is 3 seconds as per {{"-startTime", "" + 
(Time.now() / 1000 + 3)}}
In this failing test case, I can see the task starting minimum 6 seconds after 
the test case started.
{noformat}
2017-01-27 03:11:15,387 INFO  [Thread-4] mapreduce.JobSubmitter 
(JobSubmitter.java:printTokens(289)) - Submitting tokens for job: 
job_local1711545156_0001
2017-01-27 03:11:23,405 INFO  [Thread-4] mapreduce.Job (Job.java:submit(1345)) 
- The url to track the job: http://localhost:8080/
{noformat}

Also when I run this test on my laptop, I see the following line being printed.
{noformat}
2017-01-27 17:09:27,982 INFO  [LocalJobRunner Map Task Executor #0] 
hdfs.NNBench (NNBench.java:barrier(676)) - Waiting in barrier for: 1018 ms
{noformat}
This line will be printed only in {{barrier()}} method and I don't see this 
line in the logs of failed test.
In our environment, the jenkins server was very slow and it took more than 6 
seconds to launch a map task.
The correct fix in my opinion would be to return true in case there is no sleep 
in {{barrier() method}}. Only in exception, it will return false.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)

2018-04-10 Thread Wangda Tan

Thanks @Konstantin for reporting this issue, I will post comments on the
JIRA (HADOOP-15205)

- Wangda

On Tue, Apr 10, 2018 at 12:08 PM, Konstantin Shvachko 
wrote:

> A note to release managers. As discussed in
> https://issues.apache.org/jira/browse/HADOOP-15205
> We are producing release artifacts without sources jars. See e.g.
> https://repository.apache.org/content/repositories/releases/
> org/apache/hadoop/hadoop-common/3.1.0/
> I believe this has something to do with maven deployment stage,
> potentially maven-source-plugin.
> This is similar for all releases now, and I believe it should be fixed.
>
> Thanks,
> --Konstantin
>
> On Fri, Apr 6, 2018 at 2:01 PM, Wangda Tan  wrote:
>
>> Thanks guys for the additional votes! I just sent out announcement email.
>>
>> Best,
>> Wangda
>>
>> On Fri, Apr 6, 2018 at 2:32 AM, 俊平堵  wrote:
>>
>> > Thanks Wangda for the great work! Sorry for my late coming +1 (binding),
>> > based on:
>> >
>> > - Verified signatures
>> >
>> > - Verified checksums for source and binary artifacts
>> >
>> > - Built from source
>> >
>> > - Deployed a single node cluster
>> >
>> > - Verified web UIs, include Namenode, RM, etc.
>> >
>> > * Tried shell commands of HDFS and YARN
>> >
>> > * Ran sample MR jobs, include PI, Sleep, Terasort, etc.
>> >
>> >
>> > Thanks,
>> >
>> >
>> > Junping
>> >
>> >
>> >
>> > Wangda Tan 于2018年3月30日 周五下午12:15写道：
>> >
>> >> Hi folks,
>> >>
>> >> Thanks to the many who helped with this release since Dec 2017 [1].
>> We've
>> >> created RC1 for Apache Hadoop 3.1.0. The artifacts are available here:
>> >>
>> >> http://people.apache.org/~wangda/hadoop-3.1.0-RC1
>> >>
>> >> The RC tag in git is release-3.1.0-RC1. Last git commit SHA is
>> >> 16b70619a24cdcf5d3b0fcf4b58ca77238ccbe6d
>> >>
>> >> The maven artifacts are available via repository.apache.org at
>> >> https://repository.apache.org/content/repositories/orgapache
>> hadoop-1090/
>> >> This vote will run 5 days, ending on Apr 3 at 11:59 pm Pacific.
>> >>
>> >> 3.1.0 contains 766 [2] fixed JIRA issues since 3.0.0. Notable additions
>> >> include the first class GPU/FPGA support on YARN, Native services,
>> Support
>> >> rich placement constraints in YARN, S3-related enhancements, allow HDFS
>> >> block replicas to be provided by an external storage system, etc.
>> >>
>> >> For 3.1.0 RC0 vote discussion, please see [3].
>> >>
>> >> We’d like to use this as a starting release for 3.1.x [1], depending on
>> >> how
>> >> it goes, get it stabilized and potentially use a 3.1.1 in several
>> weeks as
>> >> the stable release.
>> >>
>> >> We have done testing with a pseudo cluster:
>> >> - Ran distributed job.
>> >> - GPU scheduling/isolation.
>> >> - Placement constraints (intra-application anti-affinity) by using
>> >> distributed shell.
>> >>
>> >> My +1 to start.
>> >>
>> >> Best,
>> >> Wangda/Vinod
>> >>
>> >> [1]
>> >> https://lists.apache.org/thread.html/b3fb3b6da8b6357a68513a6dfd104b
>> >> c9e19e559aedc5ebedb4ca08c8@%3Cyarn-dev.hadoop.apache.org%3E
>> >> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in
>> (3.1.0)
>> >> AND fixVersion not in (3.0.0, 3.0.0-beta1) AND status = Resolved ORDER
>> BY
>> >> fixVersion ASC
>> >> [3]
>> >> https://lists.apache.org/thread.html/b3a7dc075b7329fd660f65b48237d7
>> >> 2d4061f26f83547e41d0983ea6@%3Cyarn-dev.hadoop.apache.org%3E
>> >>
>> >
>>
>
>

Re: [VOTE] Release Apache Hadoop 3.0.2 (RC0)

2018-04-10 Thread Konstantin Shvachko

A note to release managers. As discussed in
https://issues.apache.org/jira/browse/HADOOP-15205
We are producing release artifacts without sources jars. See e.g.
https://repository.apache.org/content/repositories/releases/org/apache/hadoop/hadoop-common/3.0.1/
Based on what is staged on Nexus for 3.0.2 (RC0) I see the next release
will not have sources as well.
https://repository.apache.org/#stagingRepositories
I believe this has something to do with maven deployment stage, potentially
maven-source-plugin.
This is similar for all releases now, and I believe it should be fixed.

Thanks,
--Konstantin

On Tue, Apr 10, 2018 at 8:32 AM, Ajay Kumar 
wrote:

> Thanks Lie for working on this.
>
>- Downloaded src tarball and verified checksums
>- Built from src on mac with java 1.8.0_111
>- Built a pseudo distributed hdfs cluster
>- Run test mr jobs (pi, dfsio,wordcount
>- Verified basic hdfs operations
>- Basic validation for webui
>
> ** I checked maven artifacts and it seems source jars are not there
> (checked hadoop-hdfs , hadoop-client). Not sure if they are required for
> release.
>
>
> On 4/9/18, 4:19 PM, "Xiao Chen"  wrote:
>
> Thanks Eddy for the effort!
>
> +1 (binding)
>
>- Downloaded src tarball and verified checksums
>- Built from src
>- Started a pseudo distributed hdfs cluster
>- Verified basic hdfs operations work
>- Sanity checked logs / webui
>
> Best,
> -Xiao
>
>
> On Mon, Apr 9, 2018 at 11:28 AM, Eric Payne  invalid>
> wrote:
>
> > Thanks a lot for working to produce this release.
> >
> > +1 (binding)
> > Tested the following:
> > - built from source and installed on 6-node pseudo-cluster
> > - tested Capacity Scheduler FairOrderingPolicy and
> FifoOrderingPolicy to
> > determine that capacity was assigned as expected in each case
> > - tested user weights with FifoOrderingPolicy to ensure that weights
> were
> > assigned to users as expected.
> >
> > Eric Payne
> >
> >
> >
> >
> >
> >
> > On Friday, April 6, 2018, 1:17:10 PM CDT, Lei Xu 
> wrote:
> >
> >
> >
> >
> >
> > Hi, All
> >
> > I've created release candidate RC-0 for Apache Hadoop 3.0.2.
> >
> > Please note: this is an amendment for Apache Hadoop 3.0.1 release to
> > fix shaded jars in apache maven repository. The codebase of 3.0.2
> > release is the same as 3.0.1.  New bug fixes will be included in
> > Apache Hadoop 3.0.3 instead.
> >
> > The release page is:
> > https://cwiki.apache.org/confluence/display/HADOOP/
> Hadoop+3.0+Release
> >
> > New RC is available at: http://home.apache.org/~lei/
> hadoop-3.0.2-RC0/
> >
> > The git tag is release-3.0.2-RC0, and the latest commit is
> > 5c141f7c0f24c12cb8704a6ccc1ff8ec991f41ee
> >
> > The maven artifacts are available at
> > https://repository.apache.org/content/repositories/
> orgapachehadoop-1096/
> >
> > Please try the release, especially, *verify the maven artifacts*,
> and vote.
> >
> > The vote will run 5 days, ending 4/11/2018.
> >
> > Thanks for everyone who helped to spot the error and proposed fixes!
> >
> > 
> -
> > To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: mapreduce-dev-help@hadoop.
> apache.org
> >
> >
> > 
> -
> > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> >
> >
>
>
>
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>

Re: [VOTE] Release Apache Hadoop 3.0.2 (RC0)

2018-04-10 Thread Lei Xu

Ajay, thanks for spotting this.

I am working on fix the deploy.

On Tue, Apr 10, 2018 at 8:32 AM, Ajay Kumar  wrote:
> Thanks Lie for working on this.
>
>- Downloaded src tarball and verified checksums
>- Built from src on mac with java 1.8.0_111
>- Built a pseudo distributed hdfs cluster
>- Run test mr jobs (pi, dfsio,wordcount
>- Verified basic hdfs operations
>- Basic validation for webui
>
> ** I checked maven artifacts and it seems source jars are not there (checked 
> hadoop-hdfs , hadoop-client). Not sure if they are required for release.
>
>
> On 4/9/18, 4:19 PM, "Xiao Chen"  wrote:
>
> Thanks Eddy for the effort!
>
> +1 (binding)
>
>- Downloaded src tarball and verified checksums
>- Built from src
>- Started a pseudo distributed hdfs cluster
>- Verified basic hdfs operations work
>- Sanity checked logs / webui
>
> Best,
> -Xiao
>
>
> On Mon, Apr 9, 2018 at 11:28 AM, Eric Payne 
> 
> wrote:
>
> > Thanks a lot for working to produce this release.
> >
> > +1 (binding)
> > Tested the following:
> > - built from source and installed on 6-node pseudo-cluster
> > - tested Capacity Scheduler FairOrderingPolicy and FifoOrderingPolicy to
> > determine that capacity was assigned as expected in each case
> > - tested user weights with FifoOrderingPolicy to ensure that weights 
> were
> > assigned to users as expected.
> >
> > Eric Payne
> >
> >
> >
> >
> >
> >
> > On Friday, April 6, 2018, 1:17:10 PM CDT, Lei Xu  
> wrote:
> >
> >
> >
> >
> >
> > Hi, All
> >
> > I've created release candidate RC-0 for Apache Hadoop 3.0.2.
> >
> > Please note: this is an amendment for Apache Hadoop 3.0.1 release to
> > fix shaded jars in apache maven repository. The codebase of 3.0.2
> > release is the same as 3.0.1.  New bug fixes will be included in
> > Apache Hadoop 3.0.3 instead.
> >
> > The release page is:
> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.0+Release
> >
> > New RC is available at: http://home.apache.org/~lei/hadoop-3.0.2-RC0/
> >
> > The git tag is release-3.0.2-RC0, and the latest commit is
> > 5c141f7c0f24c12cb8704a6ccc1ff8ec991f41ee
> >
> > The maven artifacts are available at
> > https://repository.apache.org/content/repositories/orgapachehadoop-1096/
> >
> > Please try the release, especially, *verify the maven artifacts*, and 
> vote.
> >
> > The vote will run 5 days, ending 4/11/2018.
> >
> > Thanks for everyone who helped to spot the error and proposed fixes!
> >
> > -
> > To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
> >
> >
> > -
> > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> >
> >
>
>



-- 
Lei (Eddy) Xu
Software Engineer, Cloudera

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)

2018-04-10 Thread Konstantin Shvachko

A note to release managers. As discussed in
https://issues.apache.org/jira/browse/HADOOP-15205
We are producing release artifacts without sources jars. See e.g.
https://repository.apache.org/content/repositories/releases/org/apache/hadoop/hadoop-common/3.1.0/
I believe this has something to do with maven deployment stage, potentially
maven-source-plugin.
This is similar for all releases now, and I believe it should be fixed.

Thanks,
--Konstantin

On Fri, Apr 6, 2018 at 2:01 PM, Wangda Tan  wrote:

> Thanks guys for the additional votes! I just sent out announcement email.
>
> Best,
> Wangda
>
> On Fri, Apr 6, 2018 at 2:32 AM, 俊平堵  wrote:
>
> > Thanks Wangda for the great work! Sorry for my late coming +1 (binding),
> > based on:
> >
> > - Verified signatures
> >
> > - Verified checksums for source and binary artifacts
> >
> > - Built from source
> >
> > - Deployed a single node cluster
> >
> > - Verified web UIs, include Namenode, RM, etc.
> >
> > * Tried shell commands of HDFS and YARN
> >
> > * Ran sample MR jobs, include PI, Sleep, Terasort, etc.
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> >
> > Wangda Tan 于2018年3月30日 周五下午12:15写道：
> >
> >> Hi folks,
> >>
> >> Thanks to the many who helped with this release since Dec 2017 [1].
> We've
> >> created RC1 for Apache Hadoop 3.1.0. The artifacts are available here:
> >>
> >> http://people.apache.org/~wangda/hadoop-3.1.0-RC1
> >>
> >> The RC tag in git is release-3.1.0-RC1. Last git commit SHA is
> >> 16b70619a24cdcf5d3b0fcf4b58ca77238ccbe6d
> >>
> >> The maven artifacts are available via repository.apache.org at
> >> https://repository.apache.org/content/repositories/
> orgapachehadoop-1090/
> >> This vote will run 5 days, ending on Apr 3 at 11:59 pm Pacific.
> >>
> >> 3.1.0 contains 766 [2] fixed JIRA issues since 3.0.0. Notable additions
> >> include the first class GPU/FPGA support on YARN, Native services,
> Support
> >> rich placement constraints in YARN, S3-related enhancements, allow HDFS
> >> block replicas to be provided by an external storage system, etc.
> >>
> >> For 3.1.0 RC0 vote discussion, please see [3].
> >>
> >> We’d like to use this as a starting release for 3.1.x [1], depending on
> >> how
> >> it goes, get it stabilized and potentially use a 3.1.1 in several weeks
> as
> >> the stable release.
> >>
> >> We have done testing with a pseudo cluster:
> >> - Ran distributed job.
> >> - GPU scheduling/isolation.
> >> - Placement constraints (intra-application anti-affinity) by using
> >> distributed shell.
> >>
> >> My +1 to start.
> >>
> >> Best,
> >> Wangda/Vinod
> >>
> >> [1]
> >> https://lists.apache.org/thread.html/b3fb3b6da8b6357a68513a6dfd104b
> >> c9e19e559aedc5ebedb4ca08c8@%3Cyarn-dev.hadoop.apache.org%3E
> >> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.1.0)
> >> AND fixVersion not in (3.0.0, 3.0.0-beta1) AND status = Resolved ORDER
> BY
> >> fixVersion ASC
> >> [3]
> >> https://lists.apache.org/thread.html/b3a7dc075b7329fd660f65b48237d7
> >> 2d4061f26f83547e41d0983ea6@%3Cyarn-dev.hadoop.apache.org%3E
> >>
> >
>

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2018-04-10 Thread Apache Jenkins Server

For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/747/

[Apr 9, 2018 3:47:22 PM] (sunilg) YARN-7574. Add support for Node Labels on 
Auto Created Leaf Queue
[Apr 9, 2018 4:16:48 PM] (inigoiri) HDFS-13388. RequestHedgingProxyProvider 
calls multiple configured NNs
[Apr 9, 2018 5:09:25 PM] (inigoiri) HDFS-13380. RBF: mv/rm fail after the 
directory exceeded the quota
[Apr 9, 2018 5:48:42 PM] (james.clampffer) HDFS-13376. Specify minimum GCC 
version to avoid TLS support error in
[Apr 9, 2018 10:19:21 PM] (jlowe) YARN-7667. Docker Stop grace period should be 
configurable. Contributed
[Apr 9, 2018 11:37:49 PM] (Bharat) HADOOP-15328. Fix the typo in 
HttpAuthentication.md. Contributed by fang




-1 overall


The following subsystems voted -1:
unit xml


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Failed junit tests :

   hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA 
   hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy 
   hadoop.hdfs.web.TestWebHdfsTimeouts 
   hadoop.yarn.server.TestDiskFailures 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/747/artifact/out/diff-compile-cc-root.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/747/artifact/out/diff-compile-javac-root.txt
  [288K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/747/artifact/out/diff-checkstyle-root.txt
  [17M]

   pylint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/747/artifact/out/diff-patch-pylint.txt
  [24K]

   shellcheck:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/747/artifact/out/diff-patch-shellcheck.txt
  [20K]

   shelldocs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/747/artifact/out/diff-patch-shelldocs.txt
  [12K]

   whitespace:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/747/artifact/out/whitespace-eol.txt
  [9.4M]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/747/artifact/out/whitespace-tabs.txt
  [1.1M]

   xml:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/747/artifact/out/xml.txt
  [4.0K]

   javadoc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/747/artifact/out/diff-javadoc-javadoc-root.txt
  [760K]

   unit:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/747/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
  [300K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/747/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
  [12K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/747/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
  [84K]

Powered by Apache Yetus 0.8.0-SNAPSHOT   http://yetus.apache.org

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

[jira] [Created] (MAPREDUCE-7075) Change default configuration for mapreduce.reduce.shuffle.input.buffer.percent

2018-04-10 Thread Zhizhen Hou (JIRA)

Zhizhen Hou created MAPREDUCE-7075:
--

 Summary: Change default configuration for 
mapreduce.reduce.shuffle.input.buffer.percent
 Key: MAPREDUCE-7075
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7075
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
  Components: mrv2
Affects Versions: 2.9.1
Reporter: Zhizhen Hou


When the default value of mapreduce.reduce.shuffle.input.buffer.percent is 0.7, 
it may report OOM exception at shuffle stage.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: trunk+JDK8 on Windows/x64

[jira] [Created] (MAPREDUCE-7076) TestNNBench#testNNBenchCreateReadAndDelete failing in our internal build

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)

Re: [VOTE] Release Apache Hadoop 3.0.2 (RC0)

Re: [VOTE] Release Apache Hadoop 3.0.2 (RC0)

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

[jira] [Created] (MAPREDUCE-7075) Change default configuration for mapreduce.reduce.shuffle.input.buffer.percent

8 matches

Site Navigation

Mail list logo

Footer information