[jira] [Commented] (HDFS-10547) DiskBalancer: fix whitespace issue in doc files
[
https://issues.apache.org/jira/browse/HDFS-10547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338377#comment-15338377
]
Hadoop QA commented on HDFS-10547:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 28s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
55s {color} | {color:green} HDFS-1312 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 55s
{color} | {color:green} HDFS-1312 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 9m 44s {color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:c88012f |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12811695/HDFS-10547-HDFS-1312.001.patch
|
| JIRA Issue | HDFS-10547 |
| Optional Tests | asflicense mvnsite |
| uname | Linux 02c89823a1b9 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HDFS-1312 / 91d62aa |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/15827/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> DiskBalancer: fix whitespace issue in doc files
> ---
>
> Key: HDFS-10547
> URL: https://issues.apache.org/jira/browse/HDFS-10547
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: balancer & mover
>Affects Versions: HDFS-1312
>Reporter: Anu Engineer
>Assignee: Anu Engineer
>Priority: Minor
> Fix For: HDFS-1312
>
> Attachments: HDFS-10547-HDFS-1312.001.patch
>
>
> Fix the tabs are used warning in build.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10547) DiskBalancer: fix whitespace issue in doc files
[ https://issues.apache.org/jira/browse/HDFS-10547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-10547: Status: Patch Available (was: Open) > DiskBalancer: fix whitespace issue in doc files > --- > > Key: HDFS-10547 > URL: https://issues.apache.org/jira/browse/HDFS-10547 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: balancer & mover >Affects Versions: HDFS-1312 >Reporter: Anu Engineer >Assignee: Anu Engineer >Priority: Minor > Fix For: HDFS-1312 > > Attachments: HDFS-10547-HDFS-1312.001.patch > > > Fix the tabs are used warning in build. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10547) DiskBalancer: fix whitespace issue in doc files
[ https://issues.apache.org/jira/browse/HDFS-10547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-10547: Attachment: HDFS-10547-HDFS-1312.001.patch > DiskBalancer: fix whitespace issue in doc files > --- > > Key: HDFS-10547 > URL: https://issues.apache.org/jira/browse/HDFS-10547 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: balancer & mover >Affects Versions: HDFS-1312 >Reporter: Anu Engineer >Assignee: Anu Engineer >Priority: Minor > Fix For: HDFS-1312 > > Attachments: HDFS-10547-HDFS-1312.001.patch > > > Fix the tabs are used warning in build. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10547) DiskBalancer: fix whitespace issue in doc files
[ https://issues.apache.org/jira/browse/HDFS-10547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-10547: Status: Open (was: Patch Available) > DiskBalancer: fix whitespace issue in doc files > --- > > Key: HDFS-10547 > URL: https://issues.apache.org/jira/browse/HDFS-10547 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: balancer & mover >Affects Versions: HDFS-1312 >Reporter: Anu Engineer >Assignee: Anu Engineer >Priority: Minor > Fix For: HDFS-1312 > > > Fix the tabs are used warning in build. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10547) DiskBalancer: fix whitespace issue in doc files
[ https://issues.apache.org/jira/browse/HDFS-10547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-10547: Attachment: (was: HDFS-10547-HDFS-1312.001.patch) > DiskBalancer: fix whitespace issue in doc files > --- > > Key: HDFS-10547 > URL: https://issues.apache.org/jira/browse/HDFS-10547 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: balancer & mover >Affects Versions: HDFS-1312 >Reporter: Anu Engineer >Assignee: Anu Engineer >Priority: Minor > Fix For: HDFS-1312 > > > Fix the tabs are used warning in build. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10547) DiskBalancer: fix whitespace issue in doc files
[ https://issues.apache.org/jira/browse/HDFS-10547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-10547: Status: Patch Available (was: Open) > DiskBalancer: fix whitespace issue in doc files > --- > > Key: HDFS-10547 > URL: https://issues.apache.org/jira/browse/HDFS-10547 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: balancer & mover >Affects Versions: HDFS-1312 >Reporter: Anu Engineer >Assignee: Anu Engineer >Priority: Minor > Fix For: HDFS-1312 > > Attachments: HDFS-10547-HDFS-1312.001.patch > > > Fix the tabs are used warning in build. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10547) DiskBalancer: fix whitespace issue in doc files
[ https://issues.apache.org/jira/browse/HDFS-10547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-10547: Attachment: HDFS-10547-HDFS-1312.001.patch Remove tabs and white spaces. Added doc for Report command. > DiskBalancer: fix whitespace issue in doc files > --- > > Key: HDFS-10547 > URL: https://issues.apache.org/jira/browse/HDFS-10547 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: balancer & mover >Affects Versions: HDFS-1312 >Reporter: Anu Engineer >Assignee: Anu Engineer >Priority: Minor > Fix For: HDFS-1312 > > Attachments: HDFS-10547-HDFS-1312.001.patch > > > Fix the tabs are used warning in build. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10437) ReconfigurationProtocol not covered by HDFSPolicyProvider.
[
https://issues.apache.org/jira/browse/HDFS-10437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338269#comment-15338269
]
Hudson commented on HDFS-10437:
---
SUCCESS: Integrated in Hadoop-trunk-Commit #9983 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/9983/])
HDFS-10437. ReconfigurationProtocol not covered by HDFSPolicyProvider. (arp:
rev 0319d73c3b14c09da0f28371dc857b2cc5f6ec7a)
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HDFSPolicyProvider.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHDFSPolicyProvider.java
> ReconfigurationProtocol not covered by HDFSPolicyProvider.
> --
>
> Key: HDFS-10437
> URL: https://issues.apache.org/jira/browse/HDFS-10437
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.9.0
>Reporter: Chris Nauroth
>Assignee: Arpit Agarwal
> Fix For: 2.9.0
>
> Attachments: HDFS-10437.01.patch, HDFS-10437.02.patch,
> HDFS-10437.03.patch
>
>
> The {{HDFSPolicyProvider}} class contains an entry for defining the security
> policy of each HDFS RPC protocol interface. {{ReconfigurationProtocol}} is
> not listed currently. This may indicate that reconfiguration functionality
> is not working correctly in secured clusters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10437) ReconfigurationProtocol not covered by HDFSPolicyProvider.
[
https://issues.apache.org/jira/browse/HDFS-10437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arpit Agarwal updated HDFS-10437:
-
Resolution: Fixed
Fix Version/s: 2.9.0
Status: Resolved (was: Patch Available)
Pushed this to trunk and branch-2. Thank you for the reviews Chris and Xiaobing
and thanks Chris for reporting it.
> ReconfigurationProtocol not covered by HDFSPolicyProvider.
> --
>
> Key: HDFS-10437
> URL: https://issues.apache.org/jira/browse/HDFS-10437
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.9.0
>Reporter: Chris Nauroth
>Assignee: Arpit Agarwal
> Fix For: 2.9.0
>
> Attachments: HDFS-10437.01.patch, HDFS-10437.02.patch,
> HDFS-10437.03.patch
>
>
> The {{HDFSPolicyProvider}} class contains an entry for defining the security
> policy of each HDFS RPC protocol interface. {{ReconfigurationProtocol}} is
> not listed currently. This may indicate that reconfiguration functionality
> is not working correctly in secured clusters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10437) ReconfigurationProtocol not covered by HDFSPolicyProvider.
[
https://issues.apache.org/jira/browse/HDFS-10437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arpit Agarwal updated HDFS-10437:
-
Affects Version/s: (was: 2.8.0)
2.9.0
> ReconfigurationProtocol not covered by HDFSPolicyProvider.
> --
>
> Key: HDFS-10437
> URL: https://issues.apache.org/jira/browse/HDFS-10437
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.9.0
>Reporter: Chris Nauroth
>Assignee: Arpit Agarwal
> Attachments: HDFS-10437.01.patch, HDFS-10437.02.patch,
> HDFS-10437.03.patch
>
>
> The {{HDFSPolicyProvider}} class contains an entry for defining the security
> policy of each HDFS RPC protocol interface. {{ReconfigurationProtocol}} is
> not listed currently. This may indicate that reconfiguration functionality
> is not working correctly in secured clusters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10437) ReconfigurationProtocol not covered by HDFSPolicyProvider.
[
https://issues.apache.org/jira/browse/HDFS-10437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arpit Agarwal updated HDFS-10437:
-
Attachment: HDFS-10437.03.patch
Thanks for the reviews [~xiaobingo], [~cnauroth].
Yes the initial length makes little sense for a set construction, it was
inadvertently left in the test case. Thanks for catching it!
Attaching a v3 patch to fix that. The remaining checkstyle issues are due to
following the style of the original sources. I will commit this shortly.
> ReconfigurationProtocol not covered by HDFSPolicyProvider.
> --
>
> Key: HDFS-10437
> URL: https://issues.apache.org/jira/browse/HDFS-10437
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.8.0
>Reporter: Chris Nauroth
>Assignee: Arpit Agarwal
> Attachments: HDFS-10437.01.patch, HDFS-10437.02.patch,
> HDFS-10437.03.patch
>
>
> The {{HDFSPolicyProvider}} class contains an entry for defining the security
> policy of each HDFS RPC protocol interface. {{ReconfigurationProtocol}} is
> not listed currently. This may indicate that reconfiguration functionality
> is not working correctly in secured clusters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10545) DiskBalancer: PlanCommand should use -fs instead of -uri to be consistent with other hdfs commands
[
https://issues.apache.org/jira/browse/HDFS-10545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anu Engineer updated HDFS-10545:
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
[~arpitagarwal] Thanks for the review. I have committed this to the feature
branch.
> DiskBalancer: PlanCommand should use -fs instead of -uri to be consistent
> with other hdfs commands
> --
>
> Key: HDFS-10545
> URL: https://issues.apache.org/jira/browse/HDFS-10545
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
>Affects Versions: HDFS-1312
>Reporter: Lei (Eddy) Xu
>Assignee: Anu Engineer
>Priority: Minor
> Attachments: HDFS-10545-HDFS-1312.001.patch
>
>
> PlanCommand currently uses {{-uri}} to specify NameNode, while in all other
> hdfs commands (i.e., {{hdfs dfsadmin}} and {{hdfs balancer}})) they use
> {{-fs}} to specify NameNode.
> It'd be better to use {{-fs}} here.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10541) Diskbalancer: When no actions in plan, error message says "Plan was generated more than 24 hours ago"
[
https://issues.apache.org/jira/browse/HDFS-10541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anu Engineer updated HDFS-10541:
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
[~arpitagarwal] [~eddyxu] Thank you for the reviews. I have committed this to
the feature branch.
> Diskbalancer: When no actions in plan, error message says "Plan was generated
> more than 24 hours ago"
> -
>
> Key: HDFS-10541
> URL: https://issues.apache.org/jira/browse/HDFS-10541
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
>Affects Versions: HDFS-1312
>Reporter: Lei (Eddy) Xu
>Assignee: Anu Engineer
>Priority: Minor
> Attachments: HDFS-10541-HDFS-1312.001.patch
>
>
> The message is misleading. Instead, it should make it clear that there are no
> steps (or no action) to take in this plan - and should probably not error out.
> {code}
> 16/06/16 14:56:53 INFO command.Command: Executing "execute plan" command
> Plan was generated more than 24 hours ago.
> at
> org.apache.hadoop.hdfs.server.datanode.DiskBalancer.verifyTimeStamp(DiskBalancer.java:387)
> at
> org.apache.hadoop.hdfs.server.datanode.DiskBalancer.verifyPlan(DiskBalancer.java:315)
> at
> org.apache.hadoop.hdfs.server.datanode.DiskBalancer.submitPlan(DiskBalancer.java:173)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.submitDiskBalancerPlan(DataNode.java:3059)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientDatanodeProtocolServerSideTranslatorPB.submitDiskBalancerPlan(ClientDatanodeProtocolServerSideTranslatorPB.java:299)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientDatanodeProtocolProtos$ClientDatanodeProtocolService$2.callBlockingMethod(ClientDatanodeProtocolProtos.java:17509)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2086)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2082)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2080)
> {code}
> This happens when a plan looks like the following one:
> {code}
> {"volumeSetPlans":[],"nodeName":"a.b.c","nodeUUID":null,"port":20001,"timeStamp":0}
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10540) Diskbalancer: The CLI error message for disk balancer is not enabled is not clear.
[
https://issues.apache.org/jira/browse/HDFS-10540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anu Engineer updated HDFS-10540:
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
[~arpitagarwal] Thank you for review. I have committed this to the feature
branch.
> Diskbalancer: The CLI error message for disk balancer is not enabled is not
> clear.
> --
>
> Key: HDFS-10540
> URL: https://issues.apache.org/jira/browse/HDFS-10540
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
>Affects Versions: HDFS-1312
>Reporter: Lei (Eddy) Xu
>Assignee: Anu Engineer
> Attachments: HDFS-10540-HDFS-1312.001.patch
>
>
> When running the {{hdfs diskbalancer}} against a DN whose disk balancer
> feature is not enabled, it reports:
> {code}
> $ hdfs diskbalancer -plan 127.0.0.1 -uri hdfs://localhost
> 16/06/16 18:03:29 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Internal error, Unable to create JSON string.
> at
> org.apache.hadoop.hdfs.server.datanode.DiskBalancer.getVolumeNames(DiskBalancer.java:260)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.getDiskBalancerSetting(DataNode.java:3105)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientDatanodeProtocolServerSideTranslatorPB.getDiskBalancerSetting(ClientDatanodeProtocolServerSideTranslatorPB.java:359)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientDatanodeProtocolProtos$ClientDatanodeProtocolService$2.callBlockingMethod(ClientDatanodeProtocolProtos.java:17515)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2086)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2082)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2080)
> Caused by: org.apache.hadoop.hdfs.server.diskbalancer.DiskBalancerException:
> Disk Balancer is not enabled.
> at
> org.apache.hadoop.hdfs.server.datanode.DiskBalancer.checkDiskBalancerEnabled(DiskBalancer.java:293)
> at
> org.apache.hadoop.hdfs.server.datanode.DiskBalancer.getVolumeNames(DiskBalancer.java:251)
> ... 11 more
> {code}
> We should not directly throw IOE to the user. And it should explicitly
> explain the reason that the operation fails.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10533) Make DistCpOptions class immutable
[
https://issues.apache.org/jira/browse/HDFS-10533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337675#comment-15337675
]
Mingliang Liu commented on HDFS-10533:
--
Thanks [~cnauroth] and [~jingzhao] for the insights.
I strongly agree with Jing one the idea of making it hard, if not impossible
for downstream projects use the {{DistCpOptions}} wrongly. This code will go to
{{trunk}} branch, and I will address the interface audience and stability
annotations.
For the existing problems for downstream in using the {{DistCpOptions}} as Jing
proposed, I'll create another jira for tracking this. That should go to
{{branch-2}} and be backwards-compatible.
> Make DistCpOptions class immutable
> --
>
> Key: HDFS-10533
> URL: https://issues.apache.org/jira/browse/HDFS-10533
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: distcp
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
>
> Currently the {{DistCpOptions}} class encapsulates all DistCp options, which
> may be set from command-line (via the {{OptionsParser}}) or may be set
> manually (eg construct an instance and call setters). As there are multiple
> option fields and more (e.g. [HDFS-9868], [HDFS-10314]) to add, validating
> them can be cumbersome. Ideally, the {{DistCpOptions}} object should be
> immutable. The benefits are:
> # {{DistCpOptions}} is simple and easier to use and share, plus it scales well
> # validation is automatic, e.g. manually constructed {{DistCpOptions}} gets
> validated before usage
> # validation error message is well-defined which does not depend on the order
> of setters
> This jira is to track the effort of making the {{DistCpOptions}} immutable by
> using a Builder pattern for creation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10437) ReconfigurationProtocol not covered by HDFSPolicyProvider.
[
https://issues.apache.org/jira/browse/HDFS-10437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337613#comment-15337613
]
Hadoop QA commented on HDFS-10437:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 24s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 42s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
43s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m 23s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
36s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 19s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
35s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
30s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 8s
{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 16s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 1s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m 1s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 20s
{color} | {color:red} root: The patch generated 2 new + 100 unchanged - 0 fixed
= 102 total (was 100) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 42s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 42s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m 45s
{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 75m 45s {color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 129m 52s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.namenode.TestEditLog |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:e2f6409 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12811512/HDFS-10437.02.patch |
| JIRA Issue | HDFS-10437 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux b85b6eb7fcec 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 0761379 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/15824/artifact/patchprocess/diff-checkstyle-root.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/15824/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HDFS-Build/15824/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
[jira] [Updated] (HDFS-10437) ReconfigurationProtocol not covered by HDFSPolicyProvider.
[
https://issues.apache.org/jira/browse/HDFS-10437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-10437:
-
Assignee: Arpit Agarwal (was: Xiaobing Zhou)
Hadoop Flags: Reviewed
+1 for patch 02, pending pre-commit.
{code}
-policyProviderProtocols = new ArrayList<>(services.length);
+policyProviderProtocols = new HashSet<>(services.length);
{code}
Just a very minor nit: Passing a length to a collection constructor is usually
meant to avoid a memory realloc when the expected length is known. With the
switch from {{ArrayList}} to {{HashSet}}, that doesn't really work anymore
though, because the argument is now interpreted as the internal hash table's
capacity, with a realloc occurring after exceeding (capacity * load factor),
which defaults to 0.75. Things like Guava's
{{Maps#newHashMapWithExpectedSize}} internally do some math on the argument to
scale it up and try to stay ahead of the load factor to prevent a realloc.
It doesn't really matter much here, where it's just test code and the data set
is tiny, so I'm still +1 for the patch. It's just a common pitfall of the
{{HashMap}}/{{HashSet}} API.
Thanks for the patch!
> ReconfigurationProtocol not covered by HDFSPolicyProvider.
> --
>
> Key: HDFS-10437
> URL: https://issues.apache.org/jira/browse/HDFS-10437
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.8.0
>Reporter: Chris Nauroth
>Assignee: Arpit Agarwal
> Attachments: HDFS-10437.01.patch, HDFS-10437.02.patch
>
>
> The {{HDFSPolicyProvider}} class contains an entry for defining the security
> policy of each HDFS RPC protocol interface. {{ReconfigurationProtocol}} is
> not listed currently. This may indicate that reconfiguration functionality
> is not working correctly in secured clusters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
