Build failed in Jenkins: Hadoop-common-trunk-Java8 #74

2015-01-13 Thread Apache Jenkins Server 
See 

Changes:

[rkanter] YARN-3027. Scheduler should use totalAvailable resource from node 
instead of availableResource for maxAllocation. (adhoot via rkanter)

[rkanter] YARN-2957. Create unit test to automatically compare 
YarnConfiguration and yarn-default.xml. (rchiang via rkanter)

[cmccabe] HDFS-7326. Add documentation for hdfs debug commands (Vijay Bhat via 
Colin P. McCabe)

[kihwal] HDFS-7533. Datanode sometimes does not shutdown on receiving upgrade 
shutdown command. Contributed by Eric Payne.

[raviprak] HADOOP-11473. test-patch says "-1 overall" even when all checks are 
+1

[cmccabe] HDFS-7598. Remove dependency on old version of Guava in 
TestDFSClientCache#testEviction (Sangjin Lee via Colin P. McCabe)

[cmccabe] HDFS-5445. PacketReceiver populates the packetLen field in 
PacketHeader incorrectly (Jonathan Mace via Colin P. McCabe)

[jing9] HDFS-7600. Refine hdfs admin classes to reuse common code. Contributed 
by Jing Zhao.

[rkanter] YARN-2643. Don't create a new DominantResourceCalculator on every 
FairScheduler.allocate call. (kasha via rkanter)

[arp] HADOOP-11465. Fix findbugs warnings in hadoop-gridmix. (Contributed by 
Varun Saxena)

[shv] HDFS-3107. Introduce truncate. Contributed by Plamen Jeliazkov.

[shv] HDFS-7056. Snapshot support for truncate. Contributed by Konstantin 
Shvachko and Plamen Jeliazkov.

--
[...truncated 5196 lines...]
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.173 sec - in 
org.apache.hadoop.net.TestNetworkTopologyWithNodeGroup
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Running org.apache.hadoop.net.TestClusterTopology
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.164 sec - in 
org.apache.hadoop.net.TestClusterTopology
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Running org.apache.hadoop.net.TestNetUtils
Tests run: 39, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.538 sec - in 
org.apache.hadoop.net.TestNetUtils
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Running org.apache.hadoop.net.TestScriptBasedMapping
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.363 sec - in 
org.apache.hadoop.net.TestScriptBasedMapping
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Running org.apache.hadoop.net.TestStaticMapping
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.87 sec - in 
org.apache.hadoop.net.TestStaticMapping
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Running org.apache.hadoop.log.TestLogLevel
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.637 sec - in 
org.apache.hadoop.log.TestLogLevel
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Running org.apache.hadoop.log.TestLog4Json
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.353 sec - in 
org.apache.hadoop.log.TestLog4Json
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Running org.apache.hadoop.ipc.TestRetryCache
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.383 sec - in 
org.apache.hadoop.ipc.TestRetryCache
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Running org.apache.hadoop.ipc.TestRPC
Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.293 sec - 
in org.apache.hadoop.ipc.TestRPC
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Running org.apache.hadoop.ipc.TestMultipleProtocolServer
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.806 sec - in 
org.apache.hadoop.ipc.TestMultipleProtocolServer
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Running org.apache.hadoop.ipc.TestCallQueueManager
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.414 sec - in 
org.apache.hadoop.ipc.TestCallQueueManager
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Running org.apache.hadoop.ipc.TestSaslRPC
Tests run: 85, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.028 sec - 
in org.apache.hadoop.ipc.TestSaslRPC
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Running org.apache.hadoop.ipc.TestDecayRpcScheduler
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.805 sec - in 
org.apache.hadoop.ipc.TestDecayRpcScheduler
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Runni

Build failed in Jenkins: Hadoop-Common-trunk #1374

2015-01-13 Thread Apache Jenkins Server 
See 

Changes:

[rkanter] YARN-3027. Scheduler should use totalAvailable resource from node 
instead of availableResource for maxAllocation. (adhoot via rkanter)

[rkanter] YARN-2957. Create unit test to automatically compare 
YarnConfiguration and yarn-default.xml. (rchiang via rkanter)

[cmccabe] HDFS-7326. Add documentation for hdfs debug commands (Vijay Bhat via 
Colin P. McCabe)

[kihwal] HDFS-7533. Datanode sometimes does not shutdown on receiving upgrade 
shutdown command. Contributed by Eric Payne.

[raviprak] HADOOP-11473. test-patch says "-1 overall" even when all checks are 
+1

[cmccabe] HDFS-7598. Remove dependency on old version of Guava in 
TestDFSClientCache#testEviction (Sangjin Lee via Colin P. McCabe)

[cmccabe] HDFS-5445. PacketReceiver populates the packetLen field in 
PacketHeader incorrectly (Jonathan Mace via Colin P. McCabe)

[jing9] HDFS-7600. Refine hdfs admin classes to reuse common code. Contributed 
by Jing Zhao.

[rkanter] YARN-2643. Don't create a new DominantResourceCalculator on every 
FairScheduler.allocate call. (kasha via rkanter)

[arp] HADOOP-11465. Fix findbugs warnings in hadoop-gridmix. (Contributed by 
Varun Saxena)

[shv] HDFS-3107. Introduce truncate. Contributed by Plamen Jeliazkov.

[shv] HDFS-7056. Snapshot support for truncate. Contributed by Konstantin 
Shvachko and Plamen Jeliazkov.

--
[...truncated 4729 lines...]
Running org.apache.hadoop.util.TestNativeCrc32
Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.829 sec - in 
org.apache.hadoop.util.TestNativeCrc32
Running org.apache.hadoop.util.TestGSet
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.064 sec - in 
org.apache.hadoop.util.TestGSet
Running org.apache.hadoop.util.TestNativeLibraryChecker
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.451 sec - in 
org.apache.hadoop.util.TestNativeLibraryChecker
Running org.apache.hadoop.util.TestIndexedSort
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.754 sec - in 
org.apache.hadoop.util.TestIndexedSort
Running org.apache.hadoop.util.bloom.TestBloomFilters
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.458 sec - in 
org.apache.hadoop.util.bloom.TestBloomFilters
Running org.apache.hadoop.util.TestStopWatch
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.071 sec - in 
org.apache.hadoop.util.TestStopWatch
Running org.apache.hadoop.util.TestMachineList
Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 123.992 sec - 
in org.apache.hadoop.util.TestMachineList
Running org.apache.hadoop.util.TestFileBasedIPList
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.176 sec - in 
org.apache.hadoop.util.TestFileBasedIPList
Running org.apache.hadoop.util.TestGenericsUtil
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.333 sec - in 
org.apache.hadoop.util.TestGenericsUtil
Running org.apache.hadoop.util.TestLightWeightGSet
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.176 sec - in 
org.apache.hadoop.util.TestLightWeightGSet
Running org.apache.hadoop.util.TestPureJavaCrc32
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.588 sec - in 
org.apache.hadoop.util.TestPureJavaCrc32
Running org.apache.hadoop.util.TestStringUtils
Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.293 sec - in 
org.apache.hadoop.util.TestStringUtils
Running org.apache.hadoop.util.TestProtoUtil
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.237 sec - in 
org.apache.hadoop.util.TestProtoUtil
Running org.apache.hadoop.util.TestSignalLogger
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.178 sec - in 
org.apache.hadoop.util.TestSignalLogger
Running org.apache.hadoop.util.TestDiskChecker
Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.123 sec - in 
org.apache.hadoop.util.TestDiskChecker
Running org.apache.hadoop.util.TestShutdownThreadsHelper
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.17 sec - in 
org.apache.hadoop.util.TestShutdownThreadsHelper
Running org.apache.hadoop.util.TestCacheableIPList
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.276 sec - in 
org.apache.hadoop.util.TestCacheableIPList
Running org.apache.hadoop.util.TestLineReader
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.197 sec - in 
org.apache.hadoop.util.TestLineReader
Running org.apache.hadoop.util.TestIdentityHashStore
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.189 sec - in 
org.apache.hadoop.util.TestIdentityHashStore
Running org.apache.hadoop.util.TestClasspath
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.343 sec - in 
org.apache.hadoop.util.TestClasspath
Running org.apache.hadoop.util.TestApplicationClassLoader
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time e

[jira] [Created] (HADOOP-11475) Utilize try-with-resource to close StopWatch

2015-01-13 Thread Ted Yu (JIRA) 
Ted Yu created HADOOP-11475:
---

 Summary: Utilize try-with-resource to close StopWatch
 Key: HADOOP-11475
 URL: https://issues.apache.org/jira/browse/HADOOP-11475
 Project: Hadoop Common
  Issue Type: Improvement
Reporter: Ted Yu
Priority: Minor


Currently the stop() method of StopWatch is called without using finally clause.
This can result in resource leak if there is IOE thrown.
Here is one example from Journal#journal():
{code}
StopWatch sw = new StopWatch();
sw.start();
curSegment.flush(shouldFsync);
sw.stop();
{code}
If curSegment.flush() throws IOE, sw would be left unclosed.

Propose using try-with-resource structure to close the StopWatch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

NFSv3 Filesystem Connector

2015-01-13 Thread Gokul Soundararajan 
Hi,

We (Jingxin Feng, Xing Lin, and I) have been working on providing a
FileSystem implementation that allows Hadoop to utilize a NFSv3 storage
server as a filesystem. It leverages code from hadoop-nfs project for all
the request/response handling. We would like your help to add it as part of
hadoop tools (similar to the way hadoop-aws and hadoop-azure).

In more detail, the Hadoop NFS Connector allows Apache Hadoop (2.2+) and
Apache Spark (1.2+) to use a NFSv3 storage server as a storage endpoint.
The NFS Connector can be run in two modes: (1) secondary filesystem - where
Hadoop/Spark runs using HDFS as its primary storage and can use NFS as a
second storage endpoint, and (2) primary filesystem - where Hadoop/Spark
runs entirely on a NFSv3 storage server.

The code is written in a way such that existing applications do not have to
change. All one has to do is to copy the connector jar into the lib/
directory of Hadoop/Spark. Then, modify core-site.xml to provide the
necessary details.

The current version can be seen at:
https://github.com/NetApp/NetApp-Hadoop-NFS-Connector

It is my first time contributing to the Hadoop codebase. It would be great
if someone on the Hadoop team can guide us through this process. I'm
willing to make the necessary changes to integrate the code. What are the
next steps? Should I create a JIRA entry?

Thanks,

Gokul

InterfaceStability and InterfaceAudience stability

2015-01-13 Thread Abraham Elmahrek 
Hey guys,

I've noticed the InterfaceStability (
https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/InterfaceStability.java)
and InterfaceAudience (
https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/InterfaceAudience.java)
classes are marked as "Evolving". These really haven't changed much in the
last few years, so I was wondering if it is reasonable to mark them as
stable?

-Abe

Re: InterfaceStability and InterfaceAudience stability

2015-01-13 Thread Ted Yu 
+1

On Tue, Jan 13, 2015 at 1:47 PM, Abraham Elmahrek  wrote:

> Hey guys,
>
> I've noticed the InterfaceStability (
>
> https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/InterfaceStability.java
> )
> and InterfaceAudience (
>
> https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/InterfaceAudience.java
> )
> classes are marked as "Evolving". These really haven't changed much in the
> last few years, so I was wondering if it is reasonable to mark them as
> stable?
>
> -Abe
>

Re: InterfaceStability and InterfaceAudience stability

2015-01-13 Thread Robert Kanter 
+1

Though it is kinda funny that the InterfaceStability annotation was marked
as Evolving :)
@InterfaceStability.Evolving
public class InterfaceStability {...}


On Tue, Jan 13, 2015 at 2:21 PM, Ted Yu  wrote:

> +1
>
> On Tue, Jan 13, 2015 at 1:47 PM, Abraham Elmahrek 
> wrote:
>
> > Hey guys,
> >
> > I've noticed the InterfaceStability (
> >
> >
> https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/InterfaceStability.java
> > )
> > and InterfaceAudience (
> >
> >
> https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/InterfaceAudience.java
> > )
> > classes are marked as "Evolving". These really haven't changed much in
> the
> > last few years, so I was wondering if it is reasonable to mark them as
> > stable?
> >
> > -Abe
> >
>

Re: InterfaceStability and InterfaceAudience stability

2015-01-13 Thread Karthik Kambatla 
+1

On Wed, Jan 14, 2015 at 4:04 AM, Robert Kanter  wrote:

> +1
>
> Though it is kinda funny that the InterfaceStability annotation was marked
> as Evolving :)
> @InterfaceStability.Evolving
> public class InterfaceStability {...}
>
>
> On Tue, Jan 13, 2015 at 2:21 PM, Ted Yu  wrote:
>
> > +1
> >
> > On Tue, Jan 13, 2015 at 1:47 PM, Abraham Elmahrek 
> > wrote:
> >
> > > Hey guys,
> > >
> > > I've noticed the InterfaceStability (
> > >
> > >
> >
> https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/InterfaceStability.java
> > > )
> > > and InterfaceAudience (
> > >
> > >
> >
> https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/InterfaceAudience.java
> > > )
> > > classes are marked as "Evolving". These really haven't changed much in
> > the
> > > last few years, so I was wondering if it is reasonable to mark them as
> > > stable?
> > >
> > > -Abe
> > >
> >
>



-- 
Karthik Kambatla
Software Engineer, Cloudera Inc.

http://five.sentenc.es

[jira] [Created] (HADOOP-11476) Mark InterfaceStability and InterfaceAudience as stable

2015-01-13 Thread Abraham Elmahrek (JIRA) 
Abraham Elmahrek created HADOOP-11476:
-

 Summary: Mark InterfaceStability and InterfaceAudience as stable
 Key: HADOOP-11476
 URL: https://issues.apache.org/jira/browse/HADOOP-11476
 Project: Hadoop Common
  Issue Type: Improvement
Reporter: Abraham Elmahrek


InterfaceStability 
(https://github.com/apache/hadoop/blob/branch-2.6.0/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/InterfaceStability.java)
 and InterfaceAudience 
(https://github.com/apache/hadoop/blob/branch-2.6.0/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/InterfaceAudience.java)
 are marked as "Evolving", but should not change in reality. Let's mark them as 
"Stable" so other projects can use them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: InterfaceStability and InterfaceAudience stability

2015-01-13 Thread Abraham Elmahrek 
Thanks for your thoughts guys. I've created
https://issues.apache.org/jira/browse/HADOOP-11476 to follow through on
this.

On Tue, Jan 13, 2015 at 3:55 PM, Karthik Kambatla 
wrote:

> +1
>
> On Wed, Jan 14, 2015 at 4:04 AM, Robert Kanter 
> wrote:
>
> > +1
> >
> > Though it is kinda funny that the InterfaceStability annotation was
> marked
> > as Evolving :)
> > @InterfaceStability.Evolving
> > public class InterfaceStability {...}
> >
> >
> > On Tue, Jan 13, 2015 at 2:21 PM, Ted Yu  wrote:
> >
> > > +1
> > >
> > > On Tue, Jan 13, 2015 at 1:47 PM, Abraham Elmahrek 
> > > wrote:
> > >
> > > > Hey guys,
> > > >
> > > > I've noticed the InterfaceStability (
> > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/InterfaceStability.java
> > > > )
> > > > and InterfaceAudience (
> > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/InterfaceAudience.java
> > > > )
> > > > classes are marked as "Evolving". These really haven't changed much
> in
> > > the
> > > > last few years, so I was wondering if it is reasonable to mark them
> as
> > > > stable?
> > > >
> > > > -Abe
> > > >
> > >
> >
>
>
>
> --
> Karthik Kambatla
> Software Engineer, Cloudera Inc.
> 
> http://five.sentenc.es
>