[jira] [Created] (HADOOP-14064) Native maven build fails for non-default zlib location

[David James (JIRA)]

David James created HADOOP-14064:


 Summary: Native maven build fails for non-default zlib location
 Key: HADOOP-14064
 URL: https://issues.apache.org/jira/browse/HADOOP-14064
 Project: Hadoop Common
  Issue Type: Bug
  Components: build, native
Affects Versions: 2.7.3
Reporter: David James
 Fix For: 2.7.3


This bug prevents me from building a native Hadoop for my system. 

The current build fails for me if zlib is not located the usual location(s).

I looked into ways to pass the necessary information to the Maven build. I 
found nothing that worked short of modifying the pom.xml of hadoop-common.

Luckily, this is easy for the project maintainers to fix -- see patch below -- 
since the build already includes the FindZLIB module for cmake 
(https://cmake.org/cmake/help/v3.0/module/FindZLIB.html). All that is missing 
is to allow maven command line parameters pass through to FindZLIB.

I suggest the following patch (works for me) that accepts two command line 
arguments, `zlib.lib` and `zlib.include` and passes them to cmake as 
ZLIB_LIBRARY and ZLIB_INCLUDE_DIR, respectively.

It would be used like this:

{code:none}
mvn package -Pdist,native -DskipTests -Dtar \
-Dzlib.lib=$HOME/lib/libz.so \
-Dzlib.include=$HOME/include \
{code}

; e.g. "-Dzlib.lib=$HOME/lib/libz.dylib -Dzlib.include=$HOME/include".

{code:none}
diff -r hadoop-2.7.3-src/hadoop-common-project/hadoop-common/pom.xml 
hadoop-2.7.3-src-patched/hadoop-common-project/hadoop-common/pom.xml
521a522,523
> 
> 
605c607
<   
---
>   
643a646,647
> 
> 
{code}

Suggestions and improvements are welcome.




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-14063) Hadoop CredentialProvider fails to load list of keystore files

[ramtin (JIRA)]

ramtin created HADOOP-14063:
---

 Summary: Hadoop CredentialProvider fails to load list of keystore 
files
 Key: HADOOP-14063
 URL: https://issues.apache.org/jira/browse/HADOOP-14063
 Project: Hadoop Common
  Issue Type: Bug
Reporter: ramtin
Assignee: ramtin


The {{hadoop.security.credential.provider.path}} property can be a list of 
keystore files like this:
_jceks://hdfs/file1.jceks,jceks://hdfs/file2.jceks,jceks,jceks://hdfs/file3.jceks
 ..._

Each file can have different permissions set to limit the users that have 
access to the keys. Some users may not have access to all the keystore files.
Each keystore file in the list should be tried until one is found with the key 
needed.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: review for HADOOP-14028

[Aaron Fabbri]

I will take a look (though my review will be non-binding).

On Mon, Feb 6, 2017 at 8:28 AM, Steve Loughran 
wrote:

> can I get a review/feedback on, S3A block output streams don't delete
> temporary files in multipart uploads"
> https://issues.apache.org/jira/browse/HADOOP-14028
>
> ...this is pretty serious, and I'd like to get it into 2.8.0, for which I
> need reviewers. Tested against AWS S3, with a test which uses some new
> instrumentation to even verify that blocks are being released.
>
> thanks
>
> -Steve
>

requesting feedback on YARN-6013

[Steven Rand]

Hi all,

Would someone be able to provide a review/feedback on
https://issues.apache.org/jira/browse/YARN-6013?

As far as I can tell this issue breaks certain RPC calls in YARN when SASL
encryption is enabled for RPC, which seems worth fixing for the 2.8.0
release. Unfortunately I don't understand the code well, and am not
convinced that my (very naive) patch is the right approach to fixing the
problem, so I would greatly appreciate feedback/help from someone more
familiar with the code.

Thanks,
Steven

Apache Hadoop qbt Report: trunk+JDK8 on Linux/ppc64le

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/221/

[Feb 6, 2017 3:28:01 AM] (junping_du) MAPREDUCE-6338. MR AppMaster does not 
honor ephemeral port range.
[Feb 6, 2017 3:42:11 AM] (junping_du) HADOOP-12097. Allow port range to be 
specified while starting webapp.
[Feb 6, 2017 5:15:16 AM] (yqlin) HDFS-11377. Balancer hung due to no available 
mover threads. Contributed
[Feb 6, 2017 11:19:30 AM] (yqlin) HDFS-10219. Change the default value for




-1 overall


The following subsystems voted -1:
compile unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc javac


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Failed junit tests :

   hadoop.hdfs.TestDFSInputStream 
   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer 
   hadoop.fs.viewfs.TestViewFsHdfs 
   hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency 
   hadoop.hdfs.server.datanode.checker.TestThrottledAsyncChecker 
   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure 
   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting 
   hadoop.hdfs.TestRollingUpgrade 
   hadoop.hdfs.web.TestWebHdfsTimeouts 
   hadoop.yarn.server.timeline.TestRollingLevelDB 
   hadoop.yarn.server.timeline.TestTimelineDataManager 
   hadoop.yarn.server.timeline.TestLeveldbTimelineStore 
   hadoop.yarn.server.timeline.webapp.TestTimelineWebServices 
   hadoop.yarn.server.timeline.recovery.TestLeveldbTimelineStateStore 
   hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore 
   
hadoop.yarn.server.applicationhistoryservice.TestApplicationHistoryServer 
   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer 
   hadoop.yarn.server.resourcemanager.recovery.TestLeveldbRMStateStore 
   hadoop.yarn.server.resourcemanager.TestRMRestart 
   hadoop.yarn.server.TestMiniYarnClusterNodeUtilization 
   hadoop.yarn.server.TestContainerManagerSecurity 
   hadoop.yarn.server.timeline.TestLevelDBCacheTimelineStore 
   hadoop.yarn.server.timeline.TestOverrideTimelineStoreYarnClient 
   hadoop.yarn.server.timeline.TestEntityGroupFSTimelineStore 
   hadoop.yarn.applications.distributedshell.TestDistributedShell 
   hadoop.mapred.TestShuffleHandler 
   hadoop.mapreduce.v2.hs.TestHistoryServerLeveldbStateStoreService 

Timed out junit tests :

   org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean 
   org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCache 
  

   compile:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/221/artifact/out/patch-compile-root.txt
  [128K]

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/221/artifact/out/patch-compile-root.txt
  [128K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/221/artifact/out/patch-compile-root.txt
  [128K]

   unit:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/221/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
  [460K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/221/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
  [16K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/221/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt
  [52K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/221/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
  [72K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/221/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
  [324K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/221/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timeline-pluginstorage.txt
  [28K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/221/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-distributedshell.txt
  [12K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/221/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-ui.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/221/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-shuffle.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/221/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt
  [16K]
   

review for HADOOP-14028

[Steve Loughran]

can I get a review/feedback on, S3A block output streams don't delete temporary 
files in multipart uploads"
https://issues.apache.org/jira/browse/HADOOP-14028

...this is pretty serious, and I'd like to get it into 2.8.0, for which I need 
reviewers. Tested against AWS S3, with a test which uses some new 
instrumentation to even verify that blocks are being released.

thanks

-Steve

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/309/

[Feb 6, 2017 3:28:01 AM] (junping_du) MAPREDUCE-6338. MR AppMaster does not 
honor ephemeral port range.
[Feb 6, 2017 3:42:11 AM] (junping_du) HADOOP-12097. Allow port range to be 
specified while starting webapp.
[Feb 6, 2017 5:15:16 AM] (yqlin) HDFS-11377. Balancer hung due to no available 
mover threads. Contributed




-1 overall


The following subsystems voted -1:
asflicense unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Failed junit tests :

   hadoop.fs.sftp.TestSFTPFileSystem 
   hadoop.hdfs.server.datanode.checker.TestThrottledAsyncChecker 
   hadoop.hdfs.TestEncryptionZones 
   hadoop.yarn.server.timeline.webapp.TestTimelineWebServices 
   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer 
   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppStarvation 
   hadoop.yarn.server.TestDiskFailures 
   hadoop.yarn.server.TestContainerManagerSecurity 
   hadoop.yarn.server.TestMiniYarnClusterNodeUtilization 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/309/artifact/out/diff-compile-cc-root.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/309/artifact/out/diff-compile-javac-root.txt
  [168K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/309/artifact/out/diff-checkstyle-root.txt
  [16M]

   pylint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/309/artifact/out/diff-patch-pylint.txt
  [20K]

   shellcheck:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/309/artifact/out/diff-patch-shellcheck.txt
  [24K]

   shelldocs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/309/artifact/out/diff-patch-shelldocs.txt
  [16K]

   whitespace:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/309/artifact/out/whitespace-eol.txt
  [11M]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/309/artifact/out/whitespace-tabs.txt
  [1.3M]

   javadoc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/309/artifact/out/diff-javadoc-javadoc-root.txt
  [2.2M]

   unit:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/309/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt
  [128K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/309/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
  [220K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/309/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt
  [12K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/309/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
  [56K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/309/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
  [324K]

   asflicense:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/309/artifact/out/patch-asflicense-problems.txt
  [4.0K]

Powered by Apache Yetus 0.5.0-SNAPSHOT   http://yetus.apache.org



-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Resolved] (HADOOP-14061) Update checkstyle version to 6.16 or upper

[Akira Ajisaka (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-14061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akira Ajisaka resolved HADOOP-14061.

Resolution: Duplicate

> Update checkstyle version to 6.16 or upper
> --
>
> Key: HADOOP-14061
> URL: https://issues.apache.org/jira/browse/HADOOP-14061
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: build
>Reporter: Akira Ajisaka
>
> Lambda indentation issue was fixed by 
> https://github.com/checkstyle/checkstyle/issues/281 and the fix version is 
> 6.16.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Reopened] (HADOOP-14017) User friendly name for ADLS user and group

[Vishwajeet Dusane (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vishwajeet Dusane reopened HADOOP-14017:

  Assignee: Vishwajeet Dusane  (was: John Zhuge)

Based on the discussion on the same thread with [~jzhuge]. Reopening this issue 
instead of creating new feature request.

ADLS to support configuration toggle between OID and UPN (User friendly name).

> User friendly name for ADLS user and group
> --
>
> Key: HADOOP-14017
> URL: https://issues.apache.org/jira/browse/HADOOP-14017
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: fs/adl
>Affects Versions: 3.0.0-alpha3
>Reporter: John Zhuge
>Assignee: Vishwajeet Dusane
>
> ADLS displays GUID whenever user or group displayed, e.g., {{ls}}, 
> {{getfacl}}.
> ADLS requires GUID whenever user or group input is needed, e.g., {{setfacl}}, 
> {{chown}}.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org