[jira] [Created] (HDFS-13434) RBF: Fix dead links link RBF document

2018-04-11 Thread Akira Ajisaka (JIRA) 
Akira Ajisaka created HDFS-13434:


 Summary: RBF: Fix dead links link RBF document
 Key: HDFS-13434
 URL: https://issues.apache.org/jira/browse/HDFS-13434
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: documentation
Reporter: Akira Ajisaka






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: Apache Hadoop 2.9.1 Release Plan

2018-04-11 Thread Akira Ajisaka 

Hi Sammi,

Thank you for driving 2.9.1 release.

I've committed HDFS-13427 (only documentation bug fix) to branch-2.9.1.
Sorry for doing this without reading this thread.

Regards,
Akira

On 2018/04/07 4:12, Wei-Chiu Chuang wrote:

Done.

Both HDFS-11915 and HDFS-13364 are in branch-2.9 and branch-2.9.1.

Thank you Sammi for driving 2.9.1 release

On Fri, Apr 6, 2018 at 10:14 AM, Wei-Chiu Chuang 
wrote:



1.
   Looks like:
2. HDFS-13347  breaks
compilation, which is later fixed byHDFS-13364
. But HDFS-13364 is
only available in 2.9.2 (branch-2.9)





Before I cherrypick HDFS-11915, I will cherry pick HDFS-13364 first
into branch-2.9.1.


On Fri, Apr 6, 2018 at 9:46 AM, Chen, Sammi  wrote:


Hi Wei-Chiu,

HDFS-11915 improves the data reliability when there is power failure.
I'm happy to include it in 2.9.1 release.

Would you help to cherry-pick it to both branch-2.9 and branch-2.9.1?


Thanks,
Sammi

-Original Message-
From: Wei-Chiu Chuang [mailto:weic...@cloudera.com]
Sent: Thursday, April 5, 2018 1:30 AM
To: Chen, Sammi 
Cc: hdfs-dev ; mapreduce-...@hadoop.apache.or
g; common-...@hadoop.apache.org; yarn-...@hadoop.apache.org
Subject: Re: Apache Hadoop 2.9.1 Release Plan

Sorry Sammi I was late to this thread.
Please considering incorporating HDFS-11915. Sync rbw dir on the first
hsync() to avoid file lost on power failure.
I thought it was already in 2.9.1 but turns out it didn't land. The
cherry pick to branch-2.9 is conflict free.

On Mon, Apr 2, 2018 at 4:34 AM, Chen, Sammi  wrote:


Hi All,

Today I have created branch-2.9.1 from branch-2.9 and started creating

the

RC0  based on branch-2.9.1.   But due to the corporate network

conditions

and my not full privileges on Hadoop,   it will take a while for RC0 to
come out.

If you have anything want to commit to branch-2.9.1,  please let me

know.


Also I will update fix version of all  2.9.1 JIRAs and moved all
unresolved JIRA with target version = 2.9.1 to 2.9.2.



Bests,
Sammi Chen

-Original Message-
From: Chen, Sammi [mailto:sammi.c...@intel.com]
Sent: Friday, March 30, 2018 3:55 PM
To: hdfs-dev ;
mapreduce-...@hadoop.apache.org; common-...@hadoop.apache.org;
yarn-...@hadoop.apache.org
Subject: Apache Hadoop 2.9.1 Release Plan

Hi All,

We have 47 changes on 2.9 branch since last release on Nov. 2017.

  There

are 7 blockers, 5 critical issues and rest are normal bug fixes and
feature improvements.





Here are current tasks targeting for 2.9.1.  No critical and blockers
so far.

https://issues.apache.org/jira/issues/?jql=%22Target+
Version%2Fs%22+%3D+2.9.1+AND+%28project+%3D+hadoop+OR+
project+%3D+%22Hadoop+HDFS%22+OR+project+%3D+%22Hadoop+YARN%
22+OR+project+%3D+%22Hadoop+Map%2FReduce%22+OR+project+%
3D+%22Hadoop+Common%22%29+AND+status+%21%3D+resolved+ORDER+
BY+priority+DESC


I plan to cut the 2.9.1 branch today, and try to deliver the RC0  ASAP.
  Please let me know if you have any objections or suggestions.






Bests,

Sammi




-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org





--
A very happy Clouderan





--
A very happy Clouderan







-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-13433) webhdfs requests can be routed incorrectly in federated cluster

2018-04-11 Thread Arpit Agarwal (JIRA) 
Arpit Agarwal created HDFS-13433:


 Summary: webhdfs requests can be routed incorrectly in federated 
cluster
 Key: HDFS-13433
 URL: https://issues.apache.org/jira/browse/HDFS-13433
 Project: Hadoop HDFS
  Issue Type: Bug
Reporter: Arpit Agarwal
Assignee: Arpit Agarwal


In the following HA+Federated setup:
# NS1 -> namenodes nn1, nn2
# NS2 -> namenodes nn3, nn4
# fs.defaultFS is {{hdfs://ns1}}.

A webhdfs request issued to nn3 will be routed to nn1. This is because 
{{setClientNamenodeAddress}} initializes {{NameNode#clientNamenodeAddress}} 
using fs.defaultFS before the config is overriden.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Resolved] (HDFS-12707) Ozone: start-all script is missing ozone start

2018-04-11 Thread Bharat Viswanadham (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-12707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bharat Viswanadham resolved HDFS-12707.
---
Resolution: Won't Fix

> Ozone: start-all script is missing ozone start
> --
>
> Key: HDFS-12707
> URL: https://issues.apache.org/jira/browse/HDFS-12707
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
>
> start-all script is missing ozone start



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-13432) Ozone: When datanodes register, send NodeReport and ContainerReport

2018-04-11 Thread Bharat Viswanadham (JIRA) 
Bharat Viswanadham created HDFS-13432:
-

 Summary: Ozone: When datanodes register, send NodeReport and 
ContainerReport
 Key: HDFS-13432
 URL: https://issues.apache.org/jira/browse/HDFS-13432
 Project: Hadoop HDFS
  Issue Type: Bug
Reporter: Bharat Viswanadham
Assignee: Bharat Viswanadham


>From chillmode Deisgn Notes:

As part of this Jira, will update register to send NodeReport and 
ContaineReport.

Current Datanodes, send one heartbeat per 30 seconds. That means that even if 
the datanode is ready it will take around a 1 min or longer before the SCM sees 
the datanode container reports. We can address this partially be making sure 
that Register call contains both NodeReport and ContainerReport.

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-13431) Ozone: Ozone Shell should use RestClient and RpcClient

2018-04-11 Thread Lokesh Jain (JIRA) 
Lokesh Jain created HDFS-13431:
--

 Summary: Ozone: Ozone Shell should use RestClient and RpcClient
 Key: HDFS-13431
 URL: https://issues.apache.org/jira/browse/HDFS-13431
 Project: Hadoop HDFS
  Issue Type: Sub-task
Reporter: Lokesh Jain
Assignee: Lokesh Jain


Currently Ozone Shell uses OzoneRestClient. We should use both RestClient and 
RpcClient instead of OzoneRestClient.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-13430) Fix TestEncryptionZonesWithKMS failure due to HADOOP-14445

2018-04-11 Thread Xiao Chen (JIRA) 
Xiao Chen created HDFS-13430:


 Summary: Fix TestEncryptionZonesWithKMS failure due to HADOOP-14445
 Key: HDFS-13430
 URL: https://issues.apache.org/jira/browse/HDFS-13430
 Project: Hadoop HDFS
  Issue Type: Bug
Reporter: Xiao Chen
Assignee: Xiao Chen
 Attachments: HDFS-13430.01.patch

Unfortunately HADOOP-14445 had an HDFS test failure that's not caught in the 
hadoop-common precommit runs.

This is caught by our internal pre-commit using dist-test, and appears to be 
the only failure.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: trunk+JDK8 on Windows/x64

2018-04-11 Thread Apache Jenkins Server 
For more details, see https://builds.apache.org/job/hadoop-trunk-win/434/

[Apr 10, 2018 2:10:44 PM] (wwei) HDFS-13420. License header is displayed in 
ArchivalStorage/MemoryStorage
[Apr 10, 2018 2:58:26 PM] (aajisaka) HADOOP-15376. Remove double semi colons on 
imports that make Clover fall
[Apr 10, 2018 6:05:00 PM] (rakeshr) HDFS-13328. Abstract ReencryptionHandler 
recursive logic in separate
[Apr 10, 2018 6:19:48 PM] (xiao) HDFS-13363. Record file path when FSDirAclOp 
throws AclException.
[Apr 10, 2018 6:42:54 PM] (xyao) HADOOP-15340. Provide meaningful RPC server 
name for RpcMetrics.
[Apr 10, 2018 9:40:49 PM] (eyang) YARN-7984. Improved YARN service stop/destroy 
and clean up.   
[Apr 10, 2018 9:44:55 PM] (jlowe) HADOOP-15357. 
Configuration.getPropsWithPrefix no longer does variable
[Apr 10, 2018 10:38:25 PM] (xiao) HADOOP-14445. Delegation tokens are not 
shared between KMS instances.
[Apr 10, 2018 11:25:00 PM] (eyang) YARN-7973. Added ContainerRelaunch feature 
for Docker containers.   
[Apr 11, 2018 12:32:38 AM] (wangda) YARN-8116. Nodemanager fails with 
NumberFormatException: For input
[Apr 11, 2018 12:33:15 AM] (wangda) YARN-8133. Doc link broken for yarn-service 
from overview page. (Rohith
[Apr 11, 2018 3:48:50 AM] (rohithsharmaks) YARN-7941. Transitive dependencies 
for component are not resolved.
[Apr 11, 2018 4:15:39 AM] (rohithsharmaks) MAPREDUCE-7062. Update 
mapreduce.job.tags description for making use for
[Apr 11, 2018 4:31:48 AM] (xiao) HDFS-13056. Expose file-level composite CRCs 
in HDFS which are
[Apr 11, 2018 6:55:38 AM] (fabbri) HADOOP-15346 S3ARetryPolicy for 
400/BadArgument to be "fail".
[Apr 11, 2018 9:15:25 AM] (wwei) YARN-8127. Resource leak when async scheduling 
is enabled. Contributed




-1 overall


The following subsystems voted -1:
compile mvninstall unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc javac


Specific tests:

Failed junit tests :

   hadoop.security.authentication.util.TestZKSignerSecretProvider 
   hadoop.crypto.key.kms.server.TestKMS 
   hadoop.crypto.key.kms.server.TestKMSWithZK 
   hadoop.fs.http.client.TestHttpFSFileSystemLocalFileSystem 
   hadoop.fs.http.client.TestHttpFSFWithSWebhdfsFileSystem 
   hadoop.fs.http.client.TestHttpFSFWithWebhdfsFileSystem 
   hadoop.fs.http.client.TestHttpFSWithHttpFSFileSystem 
   hadoop.fs.http.server.TestHttpFSServer 
   hadoop.fs.http.server.TestHttpFSServerNoACLs 
   hadoop.fs.http.server.TestHttpFSServerNoXAttrs 
   hadoop.fs.http.server.TestHttpFSServerWebServer 
   hadoop.lib.service.hadoop.TestFileSystemAccessService 
   hadoop.lib.service.security.TestGroupsService 
   hadoop.test.TestHFSTestCase 
   hadoop.hdfs.nfs.nfs3.TestClientAccessPrivilege 
   hadoop.hdfs.nfs.nfs3.TestExportsTable 
   hadoop.hdfs.nfs.nfs3.TestNfs3HttpServer 
   hadoop.hdfs.nfs.nfs3.TestReaddir 
   hadoop.hdfs.nfs.nfs3.TestRpcProgramNfs3 
   hadoop.hdfs.nfs.nfs3.TestViewfsWithNfs3 
   hadoop.hdfs.nfs.nfs3.TestWrites 
   hadoop.hdfs.nfs.TestMountd 
   hadoop.fs.contract.router.TestRouterHDFSContractAppend 
   hadoop.fs.contract.router.TestRouterHDFSContractConcat 
   hadoop.fs.contract.router.TestRouterHDFSContractCreate 
   hadoop.fs.contract.router.TestRouterHDFSContractDelete 
   hadoop.fs.contract.router.TestRouterHDFSContractGetFileStatus 
   hadoop.fs.contract.router.TestRouterHDFSContractMkdir 
   hadoop.fs.contract.router.TestRouterHDFSContractOpen 
   hadoop.fs.contract.router.TestRouterHDFSContractRename 
   hadoop.fs.contract.router.TestRouterHDFSContractRootDirectory 
   hadoop.fs.contract.router.TestRouterHDFSContractSeek 
   hadoop.fs.contract.router.TestRouterHDFSContractSetTimes 
   hadoop.fs.contract.router.web.TestRouterWebHDFSContractAppend 
   hadoop.yarn.client.api.impl.TestTimelineClientForATS1_5 
   
hadoop.yarn.logaggregation.filecontroller.ifile.TestLogAggregationIndexFileController
 
   
hadoop.yarn.logaggregation.filecontroller.TestLogAggregationFileControllerFactory
 
   hadoop.yarn.logaggregation.TestAggregatedLogFormat 
   hadoop.yarn.logaggregation.TestAggregatedLogsBlock 
   hadoop.yarn.nodelabels.TestFileSystemNodeLabelsStore 
   hadoop.yarn.util.TestFSDownload 
   hadoop.yarn.util.TestWindowsBasedProcessTree 
   hadoop.yarn.server.federation.utils.TestFederationRegistryClient 
   hadoop.yarn.server.nodemanager.amrmproxy.TestFederationInterceptor 
   
hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch 
   
hadoop.yarn.server.nodemanager.containermanager.linux.privileged.TestPrivilegedOperationExecutor
 
   
hadoop.yarn.server.nodemanager.containermanager.linux.resources.TestCGroupsHandlerImpl
 
   
hadoop.yarn.server.nodemanager.containermanager.linux.runtime.docker.TestDockerCommandExecutor

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2018-04-11 Thread Apache Jenkins Server 
For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/748/

[Apr 10, 2018 6:29:28 AM] (yqlin) HDFS-13410. RBF: Support federation with no 
subclusters. Contributed by
[Apr 10, 2018 7:34:42 AM] (yqlin) HDFS-13384. RBF: Improve timeout RPC call 
mechanism. Contributed by
[Apr 10, 2018 9:59:40 AM] (aajisaka) HADOOP-15374. Add links of the new 
features of 3.1.0 to the top page
[Apr 10, 2018 10:39:09 AM] (sunilg) YARN-7804. [UI2] Refresh action on Grid 
view page should not be
[Apr 10, 2018 2:10:44 PM] (wwei) HDFS-13420. License header is displayed in 
ArchivalStorage/MemoryStorage
[Apr 10, 2018 2:58:26 PM] (aajisaka) HADOOP-15376. Remove double semi colons on 
imports that make Clover fall
[Apr 10, 2018 6:05:00 PM] (rakeshr) HDFS-13328. Abstract ReencryptionHandler 
recursive logic in separate
[Apr 10, 2018 6:19:48 PM] (xiao) HDFS-13363. Record file path when FSDirAclOp 
throws AclException.
[Apr 10, 2018 6:42:54 PM] (xyao) HADOOP-15340. Provide meaningful RPC server 
name for RpcMetrics.
[Apr 10, 2018 9:40:49 PM] (eyang) YARN-7984. Improved YARN service stop/destroy 
and clean up.   
[Apr 10, 2018 9:44:55 PM] (jlowe) HADOOP-15357. 
Configuration.getPropsWithPrefix no longer does variable
[Apr 10, 2018 10:38:25 PM] (xiao) HADOOP-14445. Delegation tokens are not 
shared between KMS instances.
[Apr 10, 2018 11:25:00 PM] (eyang) YARN-7973. Added ContainerRelaunch feature 
for Docker containers.   
[Apr 11, 2018 12:32:38 AM] (wangda) YARN-8116. Nodemanager fails with 
NumberFormatException: For input
[Apr 11, 2018 12:33:15 AM] (wangda) YARN-8133. Doc link broken for yarn-service 
from overview page. (Rohith




-1 overall


The following subsystems voted -1:
unit xml


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Failed junit tests :

   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure 
   hadoop.hdfs.TestEncryptionZonesWithKMS 
   hadoop.hdfs.web.TestWebHdfsTimeouts 
   hadoop.mapreduce.v2.TestSpeculativeExecution 
   hadoop.mapreduce.TestMapReduceLazyOutput 
   hadoop.mapreduce.v2.TestSpeculativeExecutionWithMRApp 
   hadoop.mapreduce.TestChild 
   hadoop.mapreduce.v2.TestMRJobs 
   hadoop.mapreduce.v2.TestNonExistentJob 
   hadoop.mapreduce.v2.TestRMNMInfo 
   hadoop.mapreduce.v2.TestUberAM 
   hadoop.mapreduce.v2.TestMRJobsWithHistoryService 
   hadoop.mapreduce.v2.TestMRJobsWithProfiler 
   hadoop.mapreduce.TestMROutputFormat 
   hadoop.mapreduce.v2.TestMRAMWithNonNormalizedCapabilities 
   hadoop.mapreduce.TestMapReduce 
   hadoop.mapreduce.TestLocalRunner 
   hadoop.conf.TestNoDefaultsJobConf 
   hadoop.mapreduce.v2.TestMRAppWithCombiner 
   hadoop.mapreduce.v2.TestMiniMRProxyUser 
   hadoop.mapreduce.security.TestMRCredentials 
   hadoop.mapreduce.security.TestBinaryTokenFile 
   hadoop.mapreduce.v2.TestMROldApiJobs 
   hadoop.streaming.TestSymLink 
   hadoop.mapred.gridmix.TestDistCacheEmulation 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/748/artifact/out/diff-compile-cc-root.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/748/artifact/out/diff-compile-javac-root.txt
  [288K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/748/artifact/out/diff-checkstyle-root.txt
  [17M]

   pylint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/748/artifact/out/diff-patch-pylint.txt
  [24K]

   shellcheck:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/748/artifact/out/diff-patch-shellcheck.txt
  [20K]

   shelldocs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/748/artifact/out/diff-patch-shelldocs.txt
  [12K]

   whitespace:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/748/artifact/out/whitespace-eol.txt
  [9.4M]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/748/artifact/out/whitespace-tabs.txt
  [1.1M]

   xml:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/748/artifact/out/xml.txt
  [4.0K]

   javadoc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/748/artifact/out/diff-javadoc-javadoc-root.txt
  [760K]

   unit:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/748/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
  [420K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/748/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
  [260K]

[jira] [Created] (HDFS-13429) libhdfs++ Expose a C++ logging API

2018-04-11 Thread James Clampffer (JIRA) 
James Clampffer created HDFS-13429:
--

 Summary: libhdfs++ Expose a C++ logging API
 Key: HDFS-13429
 URL: https://issues.apache.org/jira/browse/HDFS-13429
 Project: Hadoop HDFS
  Issue Type: Task
  Components: hdfs-client
Reporter: James Clampffer
Assignee: James Clampffer


The libhdfs++ C API supports taking function pointers for log plugins and 
defines levels for components and severity.  It'd be nice to have an idiomatic 
C++ logging interface.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Resolved] (HDFS-10205) libhdfs++: Integrate logging with the C API

2018-04-11 Thread James Clampffer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-10205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Clampffer resolved HDFS-10205.

Resolution: Won't Fix

Support for logging plugins for the C API were added a while ago 
(hdfsSetLogFunction and friends).  Closing this.

> libhdfs++: Integrate logging with the C API
> ---
>
> Key: HDFS-10205
> URL: https://issues.apache.org/jira/browse/HDFS-10205
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: hdfs-client
>Reporter: James Clampffer
>Priority: Major
>
> Logging was added in HDFS-9118.  The C API has it's own API based on setting 
> errno that is independent from the rest of the logs.  These should be tied 
> together so C API events get logged as well when it is used.
> It might also be useful to log the event handlers added in HDFS-9616 as part 
> of this work.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-13428) Remove LinkedList From StateStoreFileImpl.java

2018-04-11 Thread BELUGA BEHR (JIRA) 
BELUGA BEHR created HDFS-13428:
--

 Summary: Remove LinkedList From StateStoreFileImpl.java
 Key: HDFS-13428
 URL: https://issues.apache.org/jira/browse/HDFS-13428
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: federation
Affects Versions: 3.0.1
 Environment: Replace {{LinkedList}} with {{ArrayList}} implementation 
in the StateStoreFileImpl class.  This is especially advantageous because we 
can pre-allocate the internal array before a copy occurs.  {{ArrayList}} is 
faster for iterations and requires less memory than {{LinkedList}}.

 
{code:java}
  protected List getChildren(String path) {
List ret = new LinkedList<>();
File dir = new File(path);
File[] files = dir.listFiles();
if (files != null) {
  for (File file : files) {
String filename = file.getName();
ret.add(filename);
  }
}
return ret;
  }{code}
Reporter: BELUGA BEHR
 Attachments: HDFS-13428.1.patch





--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Reopened] (HDFS-13388) RequestHedgingProxyProvider calls multiple configured NNs all the time

2018-04-11 Thread Jinglun (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-13388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jinglun reopened HDFS-13388:


There is a problem with the patch. When it uses reflection to call the method, 
the InvocationTargetException isn't handled. We need reopen it and do some fix.

 

> RequestHedgingProxyProvider calls multiple configured NNs all the time
> --
>
> Key: HDFS-13388
> URL: https://issues.apache.org/jira/browse/HDFS-13388
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: hdfs-client
>Reporter: Jinglun
>Assignee: Jinglun
>Priority: Major
> Fix For: 2.10.0, 3.2.0, 3.1.1
>
> Attachments: HADOOP-13388.0001.patch, HADOOP-13388.0002.patch, 
> HADOOP-13388.0003.patch, HADOOP-13388.0004.patch, HADOOP-13388.0005.patch, 
> HADOOP-13388.0006.patch
>
>
> In HDFS-7858 RequestHedgingProxyProvider was designed to "first 
> simultaneously call multiple configured NNs to decide which is the active 
> Namenode and then for subsequent calls it will invoke the previously 
> successful NN ." But the current code call multiple configured NNs every time 
> even when we already got the successful NN. 
>  That's because in RetryInvocationHandler.java, ProxyDescriptor's member 
> proxyInfo is assigned only when it is constructed or when failover occurs. 
> RequestHedgingProxyProvider.currentUsedProxy is null in both cases, so the 
> only proxy we can get is always a dynamic proxy handled by 
> RequestHedgingInvocationHandler.class. RequestHedgingInvocationHandler.class 
> handles invoked method by calling multiple configured NNs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-13427) Fix transparent encryption document

2018-04-11 Thread Akira Ajisaka (JIRA) 
Akira Ajisaka created HDFS-13427:


 Summary: Fix transparent encryption document
 Key: HDFS-13427
 URL: https://issues.apache.org/jira/browse/HDFS-13427
 Project: Hadoop HDFS
  Issue Type: Bug
Reporter: Akira Ajisaka


{noformat}
The `crypto` command before Hadoop 2.8.0 does not provision the `.Trash` 
directory automatically. If an encryption zone is created before Hadoop 2.8.0, 
and then the cluster is upgraded to Hadoop 2.8.0 or above, the trash directory 
can be provisioned using `-provisionTrash` option (e.g., `hdfs crypto 
-provisionTrash -path /zone`).
Attack vectors
--
{noformat}
The long sentence starts with 'The crypto' wrongly become the title. We need to 
add a blank line between the sentence and 'Attack vectors' to fix this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-13426) Fixed javadoc in FsDatasetAsyncDiskService#removeVolume

2018-04-11 Thread Shashikant Banerjee (JIRA) 
Shashikant Banerjee created HDFS-13426:
--

 Summary: Fixed javadoc in FsDatasetAsyncDiskService#removeVolume
 Key: HDFS-13426
 URL: https://issues.apache.org/jira/browse/HDFS-13426
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs
Reporter: Shashikant Banerjee
Assignee: Shashikant Banerjee






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org