date:20170614

[jira] [Created] (HADOOP-14527) ITestS3GuardListConsistency is too slow

2017-06-14 Thread Aaron Fabbri (JIRA)

Aaron Fabbri created HADOOP-14527:
-

 Summary: ITestS3GuardListConsistency is too slow
 Key: HADOOP-14527
 URL: https://issues.apache.org/jira/browse/HADOOP-14527
 Project: Hadoop Common
  Issue Type: Sub-task
Affects Versions: HADOOP-13345
Reporter: Aaron Fabbri
Assignee: Aaron Fabbri
Priority: Minor


I'm really glad to see folks adopting the inconsistency injection stuff and 
adding test cases to ITestS3GuardListConsistency.  That test class has become 
very slow, however, mostly due to {{Thread.sleep()}} calls that wait for the 
inconsistency injection timers to expire.

I will take a stab at speeding up this test class.  As is it takes about 8 
minutes to run.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-14526) Examine code base for cases that exception is thrown from finally block

2017-06-14 Thread Yongjun Zhang (JIRA)

Yongjun Zhang created HADOOP-14526:
--

 Summary: Examine code base for cases that exception is thrown from 
finally block
 Key: HADOOP-14526
 URL: https://issues.apache.org/jira/browse/HADOOP-14526
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Yongjun Zhang


If exception X is thrown in try block, and exception Y is thrown is finally 
block, X will be swallowed. 

In addition, finally block is used to ensure resources are released properly in 
general. If we throw exception from there, some resources may be leaked. So 
it's not recommended to throw exception in the finally block

I caught one today and reported HDFS-11794, creating this jira as a master one 
to catch other similar cases.





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: trunk+JDK8 on Linux/ppc64le

2017-06-14 Thread Apache Jenkins Server

For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/345/

[Jun 13, 2017 8:21:23 PM] (templedf) MAPREDUCE-6896. Document wrong spelling in 
usage of MapredTestDriver
[Jun 13, 2017 10:21:04 PM] (junping_du) MAPREDUCE-6895. Job end notification 
not send due to




-1 overall


The following subsystems voted -1:
compile mvninstall unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc javac


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Failed junit tests :

   hadoop.fs.sftp.TestSFTPFileSystem 
   hadoop.hdfs.server.balancer.TestBalancer 
   hadoop.hdfs.server.datanode.TestNNHandlesBlockReportPerStorage 
   hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy 
   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer 
   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure 
   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting 
   hadoop.hdfs.server.diskbalancer.TestDiskBalancer 
   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure010 
   hadoop.hdfs.web.TestWebHdfsTimeouts 
   hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy 
   hadoop.yarn.server.nodemanager.recovery.TestNMLeveldbStateStoreService 
   hadoop.yarn.server.nodemanager.TestNodeManagerShutdown 
   hadoop.yarn.server.timeline.TestRollingLevelDB 
   hadoop.yarn.server.timeline.TestTimelineDataManager 
   hadoop.yarn.server.timeline.TestLeveldbTimelineStore 
   hadoop.yarn.server.timeline.recovery.TestLeveldbTimelineStateStore 
   hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore 
   
hadoop.yarn.server.applicationhistoryservice.TestApplicationHistoryServer 
   hadoop.yarn.server.resourcemanager.TestRMEmbeddedElector 
   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer 
   
hadoop.yarn.server.resourcemanager.scheduler.fair.TestContinuousScheduling 
   hadoop.yarn.server.resourcemanager.recovery.TestLeveldbRMStateStore 
   hadoop.yarn.server.resourcemanager.TestRMRestart 
   hadoop.yarn.server.TestDiskFailures 
   hadoop.yarn.server.TestMiniYarnClusterNodeUtilization 
   hadoop.yarn.server.TestContainerManagerSecurity 
   hadoop.yarn.client.api.impl.TestAMRMClient 
   hadoop.yarn.client.api.impl.TestNMClient 
   hadoop.yarn.server.timeline.TestLevelDBCacheTimelineStore 
   hadoop.yarn.server.timeline.TestOverrideTimelineStoreYarnClient 
   hadoop.yarn.server.timeline.TestEntityGroupFSTimelineStore 
   hadoop.yarn.applications.distributedshell.TestDistributedShell 
   hadoop.mapred.TestShuffleHandler 
   hadoop.mapreduce.v2.hs.TestHistoryServerLeveldbStateStoreService 

Timed out junit tests :

   org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean 
   org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCache 
   org.apache.hadoop.yarn.server.resourcemanager.TestRMStoreCommands 
   
org.apache.hadoop.yarn.server.resourcemanager.TestReservationSystemWithRMHA 
   
org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA 
   
org.apache.hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA 
   org.apache.hadoop.yarn.server.resourcemanager.TestRMHAForNodeLabels 
  

   mvninstall:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/345/artifact/out/patch-mvninstall-root.txt
  [496K]

   compile:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/345/artifact/out/patch-compile-root.txt
  [20K]

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/345/artifact/out/patch-compile-root.txt
  [20K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/345/artifact/out/patch-compile-root.txt
  [20K]

   unit:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/345/artifact/out/patch-unit-hadoop-assemblies.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/345/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt
  [144K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/345/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
  [928K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/345/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
  [56K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/345/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt
  [52K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/345/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
  [76K]

[jira] [Resolved] (HADOOP-9548) Uplift to commons-configuration 1.9

2017-06-14 Thread Andras Bokor (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-9548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Bokor resolved HADOOP-9548.
--
Resolution: Duplicate

> Uplift to commons-configuration 1.9
> ---
>
> Key: HADOOP-9548
> URL: https://issues.apache.org/jira/browse/HADOOP-9548
> Project: Hadoop Common
>  Issue Type: Wish
>Reporter: Matt Moser
>Assignee: Andras Bokor
>Priority: Minor
>
> The version of commons-configuration (1.6) currently in use is stale by 
> several years.  
> http://commons.apache.org/proper/commons-configuration/changes-report.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2017-06-14 Thread Apache Jenkins Server

For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/434/

[Jun 13, 2017 8:21:23 PM] (templedf) MAPREDUCE-6896. Document wrong spelling in 
usage of MapredTestDriver
[Jun 13, 2017 10:21:04 PM] (junping_du) MAPREDUCE-6895. Job end notification 
not send due to




-1 overall


The following subsystems voted -1:
findbugs unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

FindBugs :

   module:hadoop-common-project/hadoop-minikdc 
   Possible null pointer dereference in 
org.apache.hadoop.minikdc.MiniKdc.delete(File) due to return value of called 
method Dereferenced at 
MiniKdc.java:org.apache.hadoop.minikdc.MiniKdc.delete(File) due to return value 
of called method Dereferenced at MiniKdc.java:[line 368] 

FindBugs :

   module:hadoop-common-project/hadoop-auth 
   
org.apache.hadoop.security.authentication.server.MultiSchemeAuthenticationHandler.authenticate(HttpServletRequest,
 HttpServletResponse) makes inefficient use of keySet iterator instead of 
entrySet iterator At MultiSchemeAuthenticationHandler.java:of keySet iterator 
instead of entrySet iterator At MultiSchemeAuthenticationHandler.java:[line 
192] 

FindBugs :

   module:hadoop-common-project/hadoop-common 
   org.apache.hadoop.crypto.CipherSuite.setUnknownValue(int) 
unconditionally sets the field unknownValue At CipherSuite.java:unknownValue At 
CipherSuite.java:[line 44] 
   org.apache.hadoop.crypto.CryptoProtocolVersion.setUnknownValue(int) 
unconditionally sets the field unknownValue At 
CryptoProtocolVersion.java:unknownValue At CryptoProtocolVersion.java:[line 67] 
   Possible null pointer dereference in 
org.apache.hadoop.fs.FileUtil.fullyDeleteOnExit(File) due to return value of 
called method Dereferenced at 
FileUtil.java:org.apache.hadoop.fs.FileUtil.fullyDeleteOnExit(File) due to 
return value of called method Dereferenced at FileUtil.java:[line 118] 
   Possible null pointer dereference in 
org.apache.hadoop.fs.RawLocalFileSystem.handleEmptyDstDirectoryOnWindows(Path, 
File, Path, File) due to return value of called method Dereferenced at 
RawLocalFileSystem.java:org.apache.hadoop.fs.RawLocalFileSystem.handleEmptyDstDirectoryOnWindows(Path,
 File, Path, File) due to return value of called method Dereferenced at 
RawLocalFileSystem.java:[line 387] 
   Return value of org.apache.hadoop.fs.permission.FsAction.or(FsAction) 
ignored, but method has no side effect At FTPFileSystem.java:but method has no 
side effect At FTPFileSystem.java:[line 421] 
   Useless condition:lazyPersist == true at this point At 
CommandWithDestination.java:[line 502] 
   org.apache.hadoop.io.DoubleWritable.compareTo(DoubleWritable) 
incorrectly handles double value At DoubleWritable.java: At 
DoubleWritable.java:[line 78] 
   org.apache.hadoop.io.DoubleWritable$Comparator.compare(byte[], int, int, 
byte[], int, int) incorrectly handles double value At DoubleWritable.java:int) 
incorrectly handles double value At DoubleWritable.java:[line 97] 
   org.apache.hadoop.io.FloatWritable.compareTo(FloatWritable) incorrectly 
handles float value At FloatWritable.java: At FloatWritable.java:[line 71] 
   org.apache.hadoop.io.FloatWritable$Comparator.compare(byte[], int, int, 
byte[], int, int) incorrectly handles float value At FloatWritable.java:int) 
incorrectly handles float value At FloatWritable.java:[line 89] 
   Possible null pointer dereference in 
org.apache.hadoop.io.IOUtils.listDirectory(File, FilenameFilter) due to return 
value of called method Dereferenced at 
IOUtils.java:org.apache.hadoop.io.IOUtils.listDirectory(File, FilenameFilter) 
due to return value of called method Dereferenced at IOUtils.java:[line 351] 
   org.apache.hadoop.io.erasurecode.ECSchema.toString() makes inefficient 
use of keySet iterator instead of entrySet iterator At ECSchema.java:keySet 
iterator instead of entrySet iterator At ECSchema.java:[line 193] 
   Possible bad parsing of shift operation in 
org.apache.hadoop.io.file.tfile.Utils$Version.hashCode() At 
Utils.java:operation in 
org.apache.hadoop.io.file.tfile.Utils$Version.hashCode() At Utils.java:[line 
398] 
   
org.apache.hadoop.metrics2.lib.DefaultMetricsFactory.setInstance(MutableMetricsFactory)
 unconditionally sets the field mmfImpl At DefaultMetricsFactory.java:mmfImpl 
At DefaultMetricsFactory.java:[line 49] 
   
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.setMiniClusterMode(boolean) 
unconditionally sets the field miniClusterMode At 
DefaultMetricsSystem.java:miniClusterMode At DefaultMetricsSystem.java:[line 
100] 
   Useless object stored in variable seqOs of method

Re: Hadoop + OpenJDK 9 support

2017-06-14 Thread Gustavo Romero

Hi Chuang,

On 14-06-2017 11:12, Wei-Chiu Chuang wrote:
> Hi
> As far as I know Hadoop does not compile with JDK9. There’s development work 
> in trunk, please take a look at HADOOP-11123 
>  or search jira for 
> “hadoop jdk9”

Thanks for the information.

I'm getting an error related to the new module system on 9 when trying to start 
Ambari server:

# ambari-server start
Using python  /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start.Unable to determine server PID. Retrying...
..Unable to determine server PID. Retrying...
..Unable to determine server PID. Retrying...
ERROR: Exiting with exit code -1.
REASON: Ambari Server java process died with exitcode 1. Check 
/var/log/ambari-server/ambari-server.out for more information.

logs shows:

Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make 
protected final java.lang.Class
java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
 throws java.lang.ClassFormatError accessible: module java.base does not "opens 
java.lang" to unnamed
module @381cad29
   at 
java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:337)
   at 
java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:281)
   at 
java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:197)
   at java.base/java.lang.reflect.Method.setAccessible(Method.java:191)
   at 
com.google.inject.internal.cglib.core.$ReflectUtils$2.run(ReflectUtils.java:56)
   at java.base/java.security.AccessController.doPrivileged(Native Method)
   at 
com.google.inject.internal.cglib.core.$ReflectUtils.(ReflectUtils.java:46)

But I guess it can be an Ambari issue since running a simple standalone test on 
Hadoop with OpenJDK 9 like in [1] succeeds:

$ bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar 
grep input output '[a-z.]+'
$ cat ./output/*
1   dfsadmin

I think it's better to ask on Ambari's ML.

Thanks a lot.

Regards,
Gustavo

[1] 
https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SingleCluster.html

> Wei-Chiu Chuang
> 
>> On Jun 14, 2017, at 7:07 AM, Gustavo Romero  
>> wrote:
>>
>> Hi,
>>
>> Does Hadoop 2.7.3 or any on other recent version (maybe upstream) support
>> OpenJDK 9 (to be GA [1])?
>>
>> If none supports it, is it planned and a ticket is open to track that 
>> progress?
>>
>> Thank you.
>>
>> Best regards,
>> Gustavo
>>
>> [1] http://openjdk.java.net/projects/jdk9/
>>
>>
>> -
>> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
>> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>>
> 
> 

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: Hadoop + OpenJDK 9 support

2017-06-14 Thread Wei-Chiu Chuang

Hi
As far as I know Hadoop does not compile with JDK9. There’s development work in 
trunk, please take a look at HADOOP-11123 
 or search jira for “hadoop 
jdk9”

Wei-Chiu Chuang

> On Jun 14, 2017, at 7:07 AM, Gustavo Romero  
> wrote:
> 
> Hi,
> 
> Does Hadoop 2.7.3 or any on other recent version (maybe upstream) support
> OpenJDK 9 (to be GA [1])?
> 
> If none supports it, is it planned and a ticket is open to track that 
> progress?
> 
> Thank you.
> 
> Best regards,
> Gustavo
> 
> [1] http://openjdk.java.net/projects/jdk9/
> 
> 
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> 



signature.asc
Description: Message signed with OpenPGP

Hadoop + OpenJDK 9 support

2017-06-14 Thread Gustavo Romero

Hi,

Does Hadoop 2.7.3 or any on other recent version (maybe upstream) support
OpenJDK 9 (to be GA [1])?

If none supports it, is it planned and a ticket is open to track that progress?

Thank you.

Best regards,
Gustavo

[1] http://openjdk.java.net/projects/jdk9/


-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-14527) ITestS3GuardListConsistency is too slow

[jira] [Created] (HADOOP-14526) Examine code base for cases that exception is thrown from finally block

Apache Hadoop qbt Report: trunk+JDK8 on Linux/ppc64le

[jira] [Resolved] (HADOOP-9548) Uplift to commons-configuration 1.9

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

Re: Hadoop + OpenJDK 9 support

Re: Hadoop + OpenJDK 9 support

Hadoop + OpenJDK 9 support

8 matches

Site Navigation

Mail list logo

Footer information