[jira] [Updated] (HADOOP-13155) Implement TokenRenewer in KMS and HttpFS
[
https://issues.apache.org/jira/browse/HADOOP-13155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Chen updated HADOOP-13155:
---
Description:
Service DelegationToken (DT) renewal is done in Yarn by
{{org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer}},
where it calls {{Token#renew}} and uses ServiceLoader to find the renewer
class
([code|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/Token.java#L382]),
and invokes the renew method from it.
We seem to miss the token renewer class in KMS / HttpFSFileSystem, and hence
Yarn defaults to {{TrivialRenewer}} for DT of such kinds, resulting in the
token not being renewed.
As a side note, {{HttpFSFileSystem}} does have a {{renewDelegationToken}} API,
but I don't see it invoked in hadoop code base. KMS does not have any renew
hook.
was:
DelegationToken is done in Yarn by {{DelegationTokenRenewer}}, where it calls
{{Token#renew}} and uses ServiceLoader to get the renewer class
([code|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/Token.java#L382])
We seem to miss the token renewer class in KMS/HttpFSFileSystem, and hence Yarn
defaults to {{TrivialRenewer}} for DT of these kinds, resulting in the token
not renewed.
As a side note, {{HttpFSFileSystem}} does have a {{renewDelegationToken}} API,
but I don't see it invoked in hadoop code base. KMS does not have any renew
hook.
> Implement TokenRenewer in KMS and HttpFS
>
>
> Key: HADOOP-13155
> URL: https://issues.apache.org/jira/browse/HADOOP-13155
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Xiao Chen
>Assignee: Xiao Chen
>
> Service DelegationToken (DT) renewal is done in Yarn by
> {{org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer}},
> where it calls {{Token#renew}} and uses ServiceLoader to find the renewer
> class
> ([code|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/Token.java#L382]),
> and invokes the renew method from it.
> We seem to miss the token renewer class in KMS / HttpFSFileSystem, and hence
> Yarn defaults to {{TrivialRenewer}} for DT of such kinds, resulting in the
> token not being renewed.
> As a side note, {{HttpFSFileSystem}} does have a {{renewDelegationToken}}
> API, but I don't see it invoked in hadoop code base. KMS does not have any
> renew hook.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11858) [JDK8] Set minimum version of Hadoop 3 to JDK 8
[ https://issues.apache.org/jira/browse/HADOOP-11858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286081#comment-15286081 ] Akira AJISAKA commented on HADOOP-11858: Thanks Andrew and Robert for the work, and thanks all who discussed in the mailing list. > [JDK8] Set minimum version of Hadoop 3 to JDK 8 > --- > > Key: HADOOP-11858 > URL: https://issues.apache.org/jira/browse/HADOOP-11858 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Robert Kanter >Assignee: Robert Kanter >Priority: Blocker > Fix For: 3.0.0-alpha1 > > Attachments: HADOOP-11858.001.patch, HADOOP-11858.002.patch, > HADOOP-11858.003.patch, commit.ogv > > > Set minimum version of trunk to JDK 8 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13163) Reuse pre-computed filestatus in Distcp-CopyMapper
[ https://issues.apache.org/jira/browse/HADOOP-13163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13163: -- Attachment: HADOOP-13163.001.patch Attaching the patch for review. Existing test cases takes care of this code path. \cc [~cnauroth], [~steve_l] > Reuse pre-computed filestatus in Distcp-CopyMapper > -- > > Key: HADOOP-13163 > URL: https://issues.apache.org/jira/browse/HADOOP-13163 > Project: Hadoop Common > Issue Type: Bug >Reporter: Rajesh Balamohan > Attachments: HADOOP-13163.001.patch > > > https://github.com/apache/hadoop/blob/af942585a108d70e0946f6dd4c465a54d068eabf/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java#L185 > targetStatus is already computed and it can be reused in checkUpdate() > function. This wouldn't be a major issue in NN/HDFS, but in the case of S3 > getFileStatus calls can be expensive. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12666) Support Microsoft Azure Data Lake - as a file system in Hadoop
[
https://issues.apache.org/jira/browse/HADOOP-12666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286021#comment-15286021
]
Chris Douglas commented on HADOOP-12666:
Guessing that trainwreck on Jenkins was before HADOOP-13161. Will wait for an
all-clear before resubmitting.
[~vishwajeet.dusane], could you review the proposed changes to
{{CachedRefreshTokenBasedAccessTokenProvider}}? It does not reconfigure all
running clients on {{setConf}}; it only shares an instance among clients
created with the same ID. The replacement criteria were a guess based on the
parameters used by {{ConfRefreshTokenBasedAccessTokenProvider}}, so please
correct it as appropriate.
HADOOP-13037 will remove the dependency on WebHDFS, largely rewriting this
client. The buffering in {{PrivateAzureDataLakeFileSystem}} should also be
rewritten. It's implementing something like demand-paging, but some of the
[control
flow|https://issues.apache.org/jira/browse/HADOOP-12666?focusedCommentId=15283360=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15283360]
would be more powerful, and more understandable, if it were layered more
conventionally. Configuring the client is also very complex. I tried the
directions, but only arrived at a working client with Vishwajeet's help.
The target version is 2.9, but we should hold off on backporting this before
it's easier to use and maintain. I would like to commit the result of review
from [~cnauroth], [~eddyxu], [~twu], [~fabbri], and [~mackrorysd] to trunk.
It'll be easier to fixup the patch in targeted JIRAs. Committing the contract
tests in HADOOP-12875 would also be helpful. This would be with the caveats
from HDFS-9938: this module may be removed if it impedes WebHDFS development.
Further, it should be easier to configure before we include it in a release. Is
this an acceptable path forward?
> Support Microsoft Azure Data Lake - as a file system in Hadoop
> --
>
> Key: HADOOP-12666
> URL: https://issues.apache.org/jira/browse/HADOOP-12666
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs, fs/azure, tools
>Reporter: Vishwajeet Dusane
>Assignee: Vishwajeet Dusane
> Attachments: Create_Read_Hadoop_Adl_Store_Semantics.pdf,
> HADOOP-12666-002.patch, HADOOP-12666-003.patch, HADOOP-12666-004.patch,
> HADOOP-12666-005.patch, HADOOP-12666-006.patch, HADOOP-12666-007.patch,
> HADOOP-12666-008.patch, HADOOP-12666-009.patch, HADOOP-12666-010.patch,
> HADOOP-12666-011.patch, HADOOP-12666-012.patch, HADOOP-12666-013.patch,
> HADOOP-12666-014.patch, HADOOP-12666-1.patch
>
> Original Estimate: 336h
> Time Spent: 336h
> Remaining Estimate: 0h
>
> h2. Description
> This JIRA describes a new file system implementation for accessing Microsoft
> Azure Data Lake Store (ADL) from within Hadoop. This would enable existing
> Hadoop applications such has MR, HIVE, Hbase etc.., to use ADL store as
> input or output.
>
> ADL is ultra-high capacity, Optimized for massive throughput with rich
> management and security features. More details available at
> https://azure.microsoft.com/en-us/services/data-lake-store/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13163) Reuse pre-computed filestatus in Distcp-CopyMapper
Rajesh Balamohan created HADOOP-13163: - Summary: Reuse pre-computed filestatus in Distcp-CopyMapper Key: HADOOP-13163 URL: https://issues.apache.org/jira/browse/HADOOP-13163 Project: Hadoop Common Issue Type: Bug Reporter: Rajesh Balamohan https://github.com/apache/hadoop/blob/af942585a108d70e0946f6dd4c465a54d068eabf/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java#L185 targetStatus is already computed and it can be reused in checkUpdate() function. This wouldn't be a major issue in NN/HDFS, but in the case of S3 getFileStatus calls can be expensive. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13162) Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs
[ https://issues.apache.org/jira/browse/HADOOP-13162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13162: -- Attachment: HADOOP-13162.001.patch \cc [~steve_l] [~cnauroth] > Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs > --- > > Key: HADOOP-13162 > URL: https://issues.apache.org/jira/browse/HADOOP-13162 > Project: Hadoop Common > Issue Type: Bug > Components: fs/s3 >Reporter: Rajesh Balamohan > Attachments: HADOOP-13162.001.patch > > > getFileStatus is relatively expensive call and mkdirs invokes it multiple > times depending on how deep the directory structure is. It would be good to > reduce the number of getFileStatus calls in such cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13162) Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs
[ https://issues.apache.org/jira/browse/HADOOP-13162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13162: -- Description: getFileStatus is relatively expensive call and mkdirs invokes it multiple times depending on how deep the directory structure is. It would be good to reduce the number of getFileStatus calls in such cases. > Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs > --- > > Key: HADOOP-13162 > URL: https://issues.apache.org/jira/browse/HADOOP-13162 > Project: Hadoop Common > Issue Type: Bug > Components: fs/s3 >Reporter: Rajesh Balamohan > > getFileStatus is relatively expensive call and mkdirs invokes it multiple > times depending on how deep the directory structure is. It would be good to > reduce the number of getFileStatus calls in such cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13162) Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs
Rajesh Balamohan created HADOOP-13162: - Summary: Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs Key: HADOOP-13162 URL: https://issues.apache.org/jira/browse/HADOOP-13162 Project: Hadoop Common Issue Type: Bug Components: fs/s3 Reporter: Rajesh Balamohan -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13101) TestDNS#{testDefaultDnsServer,testNullDnsServer} failed intermittently
[
https://issues.apache.org/jira/browse/HADOOP-13101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286008#comment-15286008
]
Allen Wittenauer commented on HADOOP-13101:
---
I'm not sure how it can be one of the build machines when we run all of our
unit tests for precommit in docker containers
> TestDNS#{testDefaultDnsServer,testNullDnsServer} failed intermittently
> --
>
> Key: HADOOP-13101
> URL: https://issues.apache.org/jira/browse/HADOOP-13101
> Project: Hadoop Common
> Issue Type: Test
> Components: test
>Reporter: Xiaoyu Yao
>
> The test failed intermittently on
> [Jenkins|https://builds.apache.org/job/PreCommit-HDFS-Build/15366/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_91.txt]
> with the following error.
> {code}
> Failed tests:
> TestDNS.testDefaultDnsServer:134
> Expected: is "dd12a7999c74"
> but: was "localhost"
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13161) remove JDK7 from Dockerfile
[ https://issues.apache.org/jira/browse/HADOOP-13161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285955#comment-15285955 ] Hudson commented on HADOOP-13161: - FAILURE: Integrated in Hadoop-trunk-Commit #9776 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/9776/]) HADOOP-13161. remove JDK7 from Dockerfile (aw) (aw: rev 2c91fd82415a7b0fcdf91952bda98bea669f1646) * dev-support/docker/Dockerfile > remove JDK7 from Dockerfile > --- > > Key: HADOOP-13161 > URL: https://issues.apache.org/jira/browse/HADOOP-13161 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Fix For: 3.0.0-alpha1 > > Attachments: HADOOP-13161.00.patch, HADOOP-13161.01.patch, > HADOOP-13161.02.patch > > > We should slim down the Docker image by removing JDK7 now that trunk no > longer supports it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13161) remove JDK7 from Dockerfile
[ https://issues.apache.org/jira/browse/HADOOP-13161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-13161: -- Resolution: Fixed Fix Version/s: 3.0.0-alpha1 Status: Resolved (was: Patch Available) Thanks committed to trunk > remove JDK7 from Dockerfile > --- > > Key: HADOOP-13161 > URL: https://issues.apache.org/jira/browse/HADOOP-13161 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Fix For: 3.0.0-alpha1 > > Attachments: HADOOP-13161.00.patch, HADOOP-13161.01.patch, > HADOOP-13161.02.patch > > > We should slim down the Docker image by removing JDK7 now that trunk no > longer supports it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13161) remove JDK7 from Dockerfile
[ https://issues.apache.org/jira/browse/HADOOP-13161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285942#comment-15285942 ] Karthik Kambatla commented on HADOOP-13161: --- Patch looks good to me. +1. > remove JDK7 from Dockerfile > --- > > Key: HADOOP-13161 > URL: https://issues.apache.org/jira/browse/HADOOP-13161 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Attachments: HADOOP-13161.00.patch, HADOOP-13161.01.patch, > HADOOP-13161.02.patch > > > We should slim down the Docker image by removing JDK7 now that trunk no > longer supports it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12687) SecureUtil#getByName should also try to resolve direct hostname, incase multiple loopback addresses are present in /etc/hosts
[
https://issues.apache.org/jira/browse/HADOOP-12687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285931#comment-15285931
]
Rohith Sharma K S commented on HADOOP-12687:
I closed the INFRA JIRA as wont fix as per discussion with Allen
([comment-link|https://issues.apache.org/jira/browse/YARN-4478?focusedCommentId=15257550=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15257550])
in issue YARN-4478. I believe this JIRA should get in.
Given RFC standard are negotiable, Can other folks express their opinion on the
patch?
> SecureUtil#getByName should also try to resolve direct hostname, incase
> multiple loopback addresses are present in /etc/hosts
> -
>
> Key: HADOOP-12687
> URL: https://issues.apache.org/jira/browse/HADOOP-12687
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Junping Du
>Assignee: Sunil G
>Priority: Blocker
> Labels: security
> Attachments: 0001-YARN-4352.patch, 0002-YARN-4352.patch,
> 0003-HADOOP-12687.patch, 0004-HADOOP-12687.patch
>
>
> From
> https://builds.apache.org/job/PreCommit-YARN-Build/9661/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_79.txt,
> we can see the tests in TestYarnClient, TestAMRMClient and TestNMClient get
> timeout which can be reproduced locally.
> When {{/etc/hosts}} has multiple loopback entries,
> {{InetAddress.getByName(null)}} will be returning the first entry present in
> etc/hosts. Hence its possible that machine hostname can be second in list and
> cause {{UnKnownHostException}}.
> Suggesting a direct resolve for such hostname scenarios.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13161) remove JDK7 from Dockerfile
[
https://issues.apache.org/jira/browse/HADOOP-13161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285928#comment-15285928
]
Hadoop QA commented on HADOOP-13161:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 11m 11s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 46s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 14s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
12s {color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
10s {color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 1s {color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:cf2ee45 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804325/HADOOP-13161.02.patch
|
| JIRA Issue | HADOOP-13161 |
| Optional Tests | asflicense shellcheck shelldocs |
| uname | Linux a58dcee5bd58 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 61f46be |
| shellcheck | v0.4.4 |
| modules | C: U: |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9455/console |
| Powered by | Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> remove JDK7 from Dockerfile
> ---
>
> Key: HADOOP-13161
> URL: https://issues.apache.org/jira/browse/HADOOP-13161
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Affects Versions: 3.0.0-alpha1
>Reporter: Allen Wittenauer
>Assignee: Allen Wittenauer
> Attachments: HADOOP-13161.00.patch, HADOOP-13161.01.patch,
> HADOOP-13161.02.patch
>
>
> We should slim down the Docker image by removing JDK7 now that trunk no
> longer supports it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13161) remove JDK7 from Dockerfile
[
https://issues.apache.org/jira/browse/HADOOP-13161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285927#comment-15285927
]
Hadoop QA commented on HADOOP-13161:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 9m 59s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 1s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 14s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
12s {color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
11s {color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 5s {color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:cf2ee45 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804325/HADOOP-13161.02.patch
|
| JIRA Issue | HADOOP-13161 |
| Optional Tests | asflicense shellcheck shelldocs |
| uname | Linux 8af4941ac5a7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 61f46be |
| shellcheck | v0.4.4 |
| modules | C: U: |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9454/console |
| Powered by | Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> remove JDK7 from Dockerfile
> ---
>
> Key: HADOOP-13161
> URL: https://issues.apache.org/jira/browse/HADOOP-13161
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Affects Versions: 3.0.0-alpha1
>Reporter: Allen Wittenauer
>Assignee: Allen Wittenauer
> Attachments: HADOOP-13161.00.patch, HADOOP-13161.01.patch,
> HADOOP-13161.02.patch
>
>
> We should slim down the Docker image by removing JDK7 now that trunk no
> longer supports it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12687) SecureUtil#getByName should also try to resolve direct hostname, incase multiple loopback addresses are present in /etc/hosts
[
https://issues.apache.org/jira/browse/HADOOP-12687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285925#comment-15285925
]
Rohith Sharma K S commented on HADOOP-12687:
Linking to YARN-4478, few useful discussion happened related to this issue.
Discussion starts from this
[comment|https://issues.apache.org/jira/browse/YARN-4478?focusedCommentId=15174874=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15174874]
> SecureUtil#getByName should also try to resolve direct hostname, incase
> multiple loopback addresses are present in /etc/hosts
> -
>
> Key: HADOOP-12687
> URL: https://issues.apache.org/jira/browse/HADOOP-12687
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Junping Du
>Assignee: Sunil G
>Priority: Blocker
> Labels: security
> Attachments: 0001-YARN-4352.patch, 0002-YARN-4352.patch,
> 0003-HADOOP-12687.patch, 0004-HADOOP-12687.patch
>
>
> From
> https://builds.apache.org/job/PreCommit-YARN-Build/9661/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_79.txt,
> we can see the tests in TestYarnClient, TestAMRMClient and TestNMClient get
> timeout which can be reproduced locally.
> When {{/etc/hosts}} has multiple loopback entries,
> {{InetAddress.getByName(null)}} will be returning the first entry present in
> etc/hosts. Hence its possible that machine hostname can be second in list and
> cause {{UnKnownHostException}}.
> Suggesting a direct resolve for such hostname scenarios.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13161) remove JDK7 from Dockerfile
[ https://issues.apache.org/jira/browse/HADOOP-13161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285921#comment-15285921 ] Hadoop QA commented on HADOOP-13161: (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/PreCommit-HADOOP-Build/9455/console in case of problems. > remove JDK7 from Dockerfile > --- > > Key: HADOOP-13161 > URL: https://issues.apache.org/jira/browse/HADOOP-13161 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Attachments: HADOOP-13161.00.patch, HADOOP-13161.01.patch, > HADOOP-13161.02.patch > > > We should slim down the Docker image by removing JDK7 now that trunk no > longer supports it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13161) remove JDK7 from Dockerfile
[ https://issues.apache.org/jira/browse/HADOOP-13161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285920#comment-15285920 ] Hadoop QA commented on HADOOP-13161: (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/PreCommit-HADOOP-Build/9454/console in case of problems. > remove JDK7 from Dockerfile > --- > > Key: HADOOP-13161 > URL: https://issues.apache.org/jira/browse/HADOOP-13161 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Attachments: HADOOP-13161.00.patch, HADOOP-13161.01.patch, > HADOOP-13161.02.patch > > > We should slim down the Docker image by removing JDK7 now that trunk no > longer supports it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13161) remove JDK7 from Dockerfile
[ https://issues.apache.org/jira/browse/HADOOP-13161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-13161: -- Attachment: HADOOP-13161.02.patch -02: * fix maven link > remove JDK7 from Dockerfile > --- > > Key: HADOOP-13161 > URL: https://issues.apache.org/jira/browse/HADOOP-13161 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Attachments: HADOOP-13161.00.patch, HADOOP-13161.01.patch, > HADOOP-13161.02.patch > > > We should slim down the Docker image by removing JDK7 now that trunk no > longer supports it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13161) remove JDK7 from Dockerfile
[
https://issues.apache.org/jira/browse/HADOOP-13161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285916#comment-15285916
]
Hadoop QA commented on HADOOP-13161:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 2m 22s
{color} | {color:red} Docker failed to build yetus/hadoop:cf2ee45. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804323/HADOOP-13161.01.patch
|
| JIRA Issue | HADOOP-13161 |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9453/console |
| Powered by | Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> remove JDK7 from Dockerfile
> ---
>
> Key: HADOOP-13161
> URL: https://issues.apache.org/jira/browse/HADOOP-13161
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Affects Versions: 3.0.0-alpha1
>Reporter: Allen Wittenauer
>Assignee: Allen Wittenauer
> Attachments: HADOOP-13161.00.patch, HADOOP-13161.01.patch
>
>
> We should slim down the Docker image by removing JDK7 now that trunk no
> longer supports it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13161) remove JDK7 from Dockerfile
[ https://issues.apache.org/jira/browse/HADOOP-13161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285910#comment-15285910 ] Hadoop QA commented on HADOOP-13161: (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/PreCommit-HADOOP-Build/9453/console in case of problems. > remove JDK7 from Dockerfile > --- > > Key: HADOOP-13161 > URL: https://issues.apache.org/jira/browse/HADOOP-13161 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Attachments: HADOOP-13161.00.patch, HADOOP-13161.01.patch > > > We should slim down the Docker image by removing JDK7 now that trunk no > longer supports it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13161) remove JDK7 from Dockerfile
[ https://issues.apache.org/jira/browse/HADOOP-13161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-13161: -- Attachment: HADOOP-13161.01.patch -01: * move ant and maven out of top level apt-get to prevent default jre from being installed > remove JDK7 from Dockerfile > --- > > Key: HADOOP-13161 > URL: https://issues.apache.org/jira/browse/HADOOP-13161 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Attachments: HADOOP-13161.00.patch, HADOOP-13161.01.patch > > > We should slim down the Docker image by removing JDK7 now that trunk no > longer supports it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13161) remove JDK7 from Dockerfile
[
https://issues.apache.org/jira/browse/HADOOP-13161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285898#comment-15285898
]
Hadoop QA commented on HADOOP-13161:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 11m 51s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 1s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
11s {color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
11s {color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 4s {color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:cf2ee45 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804320/HADOOP-13161.00.patch
|
| JIRA Issue | HADOOP-13161 |
| Optional Tests | asflicense shellcheck shelldocs |
| uname | Linux b4b59a6d3fde 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 61f46be |
| shellcheck | v0.4.4 |
| modules | C: U: |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9452/console |
| Powered by | Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> remove JDK7 from Dockerfile
> ---
>
> Key: HADOOP-13161
> URL: https://issues.apache.org/jira/browse/HADOOP-13161
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Affects Versions: 3.0.0-alpha1
>Reporter: Allen Wittenauer
>Assignee: Allen Wittenauer
> Attachments: HADOOP-13161.00.patch
>
>
> We should slim down the Docker image by removing JDK7 now that trunk no
> longer supports it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13161) remove JDK7 from Dockerfile
[ https://issues.apache.org/jira/browse/HADOOP-13161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285887#comment-15285887 ] Hadoop QA commented on HADOOP-13161: (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/PreCommit-HADOOP-Build/9452/console in case of problems. > remove JDK7 from Dockerfile > --- > > Key: HADOOP-13161 > URL: https://issues.apache.org/jira/browse/HADOOP-13161 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Attachments: HADOOP-13161.00.patch > > > We should slim down the Docker image by removing JDK7 now that trunk no > longer supports it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13161) remove JDK7 from Dockerfile
[ https://issues.apache.org/jira/browse/HADOOP-13161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-13161: -- Status: Patch Available (was: Open) > remove JDK7 from Dockerfile > --- > > Key: HADOOP-13161 > URL: https://issues.apache.org/jira/browse/HADOOP-13161 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Attachments: HADOOP-13161.00.patch > > > We should slim down the Docker image by removing JDK7 now that trunk no > longer supports it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13161) remove JDK7 from Dockerfile
[ https://issues.apache.org/jira/browse/HADOOP-13161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-13161: -- Attachment: HADOOP-13161.00.patch > remove JDK7 from Dockerfile > --- > > Key: HADOOP-13161 > URL: https://issues.apache.org/jira/browse/HADOOP-13161 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Attachments: HADOOP-13161.00.patch > > > We should slim down the Docker image by removing JDK7 now that trunk no > longer supports it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13161) remove JDK7 from Dockerfile
[ https://issues.apache.org/jira/browse/HADOOP-13161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285880#comment-15285880 ] Allen Wittenauer commented on HADOOP-13161: --- It's important to note that Apache Yetus does not support different vendors/same version in MultiJDK mode (YETUS-13). Since OpenJDK 8 isn't (easily) available for our Ubuntu 14.04 docker image, I'm just going to go with plain old Oracle JDK 8. > remove JDK7 from Dockerfile > --- > > Key: HADOOP-13161 > URL: https://issues.apache.org/jira/browse/HADOOP-13161 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > > We should slim down the Docker image by removing JDK7 now that trunk no > longer supports it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-13161) remove JDK7 from Dockerfile
[ https://issues.apache.org/jira/browse/HADOOP-13161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer reassigned HADOOP-13161: - Assignee: Allen Wittenauer > remove JDK7 from Dockerfile > --- > > Key: HADOOP-13161 > URL: https://issues.apache.org/jira/browse/HADOOP-13161 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > > We should slim down the Docker image by removing JDK7 now that trunk no > longer supports it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13161) remove JDK7 from Dockerfile
Allen Wittenauer created HADOOP-13161: - Summary: remove JDK7 from Dockerfile Key: HADOOP-13161 URL: https://issues.apache.org/jira/browse/HADOOP-13161 Project: Hadoop Common Issue Type: Improvement Components: build Affects Versions: 3.0.0-alpha1 Reporter: Allen Wittenauer We should slim down the Docker image by removing JDK7 now that trunk no longer supports it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12666) Support Microsoft Azure Data Lake - as a file system in Hadoop
[
https://issues.apache.org/jira/browse/HADOOP-12666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285863#comment-15285863
]
Hadoop QA commented on HADOOP-12666:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 7s
{color} | {color:red} root in trunk failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 30s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 10s
{color} | {color:red} root in trunk failed with JDK v1.7.0_101. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
50s {color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m 18s
{color} | {color:red} hadoop-common in trunk failed. {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m 15s
{color} | {color:red} hadoop-tools in trunk failed. {color} |
| {color:red}-1{color} | {color:red} mvneclipse {color} | {color:red} 0m 9s
{color} | {color:red} hadoop-common in trunk failed. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s
{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-tools hadoop-tools/hadoop-tools-dist {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 8s
{color} | {color:red} hadoop-common in trunk failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 8s
{color} | {color:red} hadoop-common in trunk failed with JDK v1.8.0_91. {color}
|
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 9s
{color} | {color:red} hadoop-common in trunk failed with JDK v1.7.0_101.
{color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 8s
{color} | {color:red} hadoop-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 10s
{color} | {color:red} hadoop-project in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 19s
{color} | {color:red} hadoop-tools in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 11s
{color} | {color:red} hadoop-azure-datalake in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 10s
{color} | {color:red} hadoop-tools-dist in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 10s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 10s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 10s
{color} | {color:red} root in the patch failed with JDK v1.7.0_101. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 10s {color}
| {color:red} root in the patch failed with JDK v1.7.0_101. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 25s
{color} | {color:red} root: The patch generated 1 new + 5 unchanged - 0 fixed =
6 total (was 5) {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m 12s
{color} | {color:red} hadoop-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m 12s
{color} | {color:red} hadoop-tools in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m 12s
{color} | {color:red} hadoop-azure-datalake in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m 10s
{color} | {color:red} hadoop-tools-dist in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvneclipse {color} | {color:red} 0m 9s
{color} | {color:red} hadoop-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red}
[jira] [Commented] (HADOOP-13157) Follow-on improvements to HADOOP-12942
[
https://issues.apache.org/jira/browse/HADOOP-13157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285852#comment-15285852
]
Hadoop QA commented on HADOOP-13157:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 8s
{color} | {color:red} root in trunk failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 55s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 10s
{color} | {color:red} root in trunk failed with JDK v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s {color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m 17s
{color} | {color:red} hadoop-common in trunk failed. {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s {color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 11s
{color} | {color:red} hadoop-common in trunk failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 4s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 12s
{color} | {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 59s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 59s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 9s
{color} | {color:red} root in the patch failed with JDK v1.7.0_95. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 9s {color}
| {color:red} root in the patch failed with JDK v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s {color} | {color:green} hadoop-common-project/hadoop-common: The patch
generated 0 new + 59 unchanged - 2 fixed = 59 total (was 61) {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m 14s
{color} | {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 11s
{color} | {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 12s
{color} | {color:red} hadoop-common in the patch failed with JDK v1.7.0_95.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m 32s
{color} | {color:green} hadoop-common in the patch passed with JDK v1.8.0_91.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 11s {color}
| {color:red} hadoop-common in the patch failed with JDK v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 19s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:cf2ee45 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804308/HADOOP-13157.001.patch
|
| JIRA Issue | HADOOP-13157 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 565ab8e9dd1a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HADOOP-13160) Suppress checkstyle JavadocPackage error for test source files
[
https://issues.apache.org/jira/browse/HADOOP-13160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285844#comment-15285844
]
Hadoop QA commented on HADOOP-13160:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
31s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 36s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 41s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 10m
38s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
58s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m 50s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 10m
36s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
37s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 45s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m 9s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 10m 2s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
40s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 2s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m 32s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 9m 34s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 24m 36s {color}
| {color:red} root in the patch failed with JDK v1.8.0_91. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 125m 12s
{color} | {color:red} root in the patch failed with JDK v1.7.0_95. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 30s
{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 252m 57s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_91 Failed junit tests | hadoop.net.TestDNS |
| JDK v1.7.0_95 Failed junit tests | hadoop.hdfs.server.namenode.TestEditLog |
| | hadoop.metrics2.sink.TestRollingFileSystemSinkWithSecureHdfs |
| | hadoop.hdfs.server.namenode.TestFileTruncate |
| | hadoop.hdfs.TestAsyncDFSRename |
| |
hadoop.yarn.server.applicationhistoryservice.TestFileSystemApplicationHistoryStore
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:cf2ee45 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804260/HADOOP-13160.001.patch
|
| JIRA Issue | HADOOP-13160 |
| Optional Tests | asflicense xml compile javac javadoc mvninstall
mvnsite unit |
| uname | Linux f2e8b2dc3d8f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Updated] (HADOOP-13157) Follow-on improvements to HADOOP-12942
[
https://issues.apache.org/jira/browse/HADOOP-13157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Yoder updated HADOOP-13157:
Status: Patch Available (was: Open)
{quote}
File
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/JavaKeyStoreProvider.java:
Line 147:
Could this be a static helper?
{quote}
Changed. Now the env var and filename are passed in.
{quote}
Line 161: new
The javadoc says it returns null in this situation. This is also a difference
from the implementation in the AbstractJKSP. Intentional?
{quote}
This line came in as a part of
https://issues.apache.org/jira/browse/HADOOP-10224. With that work, the
JavaKeyStoreProvider was given a more sophisticated old/new corruption
prevention dance that the AbstractJKSP lacks. I'd lean towards leaving it alone
and using this version for both.
{quote}
Line 175: private void locateKeystore() throws IOException {
static helper? for the construct*Path methods too?
{quote}
locateKeystore hits a bunch of member variables: password, path, keyStore, fs,
permissions... so please no. construct*Path() - sure, changed.
{quote}
File
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/KeyShell.java:
Line 50: @VisibleForTesting public static final String NO_VALID_PROVIDERS =
FYI for the future, our coding style is to put annotations on their own
separate line.
File
{quote}
Done. Can this rule be added to the checkstyle rules?
{quote}
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/alias/AbstractJavaKeyStoreProvider.java:
Line 326: private char[] locatePassword() throws IOException {
this method looks very similar to the one in JavaKeyStoreProvider, except the
env var it looks for is different, is there potential for code reuse?
{quote}
Yes. Moved to ProviderUtils along with some other stuff.
{quote}
Line 394: " o In the environment variable " +
Using a "*" is the usual way of doing a bullet point, e.g. markdown and wiki
syntax.
{quote}
Fixed, both here and JavaKeyStoreProvider
{quote}
Line 399: "http://hadoop.apache.org/docs/current/hadoop-project-dist/; +
This link is not tied to a version, so could be inaccurate.
{quote}
Made generic without link.
> Follow-on improvements to HADOOP-12942
> --
>
> Key: HADOOP-13157
> URL: https://issues.apache.org/jira/browse/HADOOP-13157
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Affects Versions: 2.8.0
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-13157.001.patch
>
>
> [~andrew.wang] had some follow-up code review comments from HADOOP-12942.
> Hence this issue.
> Ping [~lmccay] as well.
> The comments:
> {quote}
> Overall this looks okay, the only correctness question I have is about the
> difference in behavior when the pwfile doesn't exist.
> The rest are all nits, would be nice to do these cleanups though.
> File
> hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/JavaKeyStoreProvider.java:
> Line 147:
> Could this be a static helper?
> Line 161: new
> The javadoc says it returns null in this situation. This is also a difference
> from the implementation in the AbstractJKSP. Intentional?
> Line 175: private void locateKeystore() throws IOException {
> static helper? for the construct*Path methods too?
> File
> hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/KeyShell.java:
> Line 50: @VisibleForTesting public static final String NO_VALID_PROVIDERS =
> FYI for the future, our coding style is to put annotations on their own
> separate line.
> File
> hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/alias/AbstractJavaKeyStoreProvider.java:
> Line 326: private char[] locatePassword() throws IOException {
> this method looks very similar to the one in JavaKeyStoreProvider, except the
> env var it looks for is different, is there potential for code reuse?
> Line 394: "o In the environment variable " +
> Using a "*" is the usual way of doing a bullet point, e.g. markdown and wiki
> syntax.
> Line 399:
> "http://hadoop.apache.org/docs/current/hadoop-project-dist/; +
> This link is not tied to a version, so could be inaccurate.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13157) Follow-on improvements to HADOOP-12942
[
https://issues.apache.org/jira/browse/HADOOP-13157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Yoder updated HADOOP-13157:
Attachment: HADOOP-13157.001.patch
> Follow-on improvements to HADOOP-12942
> --
>
> Key: HADOOP-13157
> URL: https://issues.apache.org/jira/browse/HADOOP-13157
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Affects Versions: 2.8.0
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-13157.001.patch
>
>
> [~andrew.wang] had some follow-up code review comments from HADOOP-12942.
> Hence this issue.
> Ping [~lmccay] as well.
> The comments:
> {quote}
> Overall this looks okay, the only correctness question I have is about the
> difference in behavior when the pwfile doesn't exist.
> The rest are all nits, would be nice to do these cleanups though.
> File
> hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/JavaKeyStoreProvider.java:
> Line 147:
> Could this be a static helper?
> Line 161: new
> The javadoc says it returns null in this situation. This is also a difference
> from the implementation in the AbstractJKSP. Intentional?
> Line 175: private void locateKeystore() throws IOException {
> static helper? for the construct*Path methods too?
> File
> hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/KeyShell.java:
> Line 50: @VisibleForTesting public static final String NO_VALID_PROVIDERS =
> FYI for the future, our coding style is to put annotations on their own
> separate line.
> File
> hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/alias/AbstractJavaKeyStoreProvider.java:
> Line 326: private char[] locatePassword() throws IOException {
> this method looks very similar to the one in JavaKeyStoreProvider, except the
> env var it looks for is different, is there potential for code reuse?
> Line 394: "o In the environment variable " +
> Using a "*" is the usual way of doing a bullet point, e.g. markdown and wiki
> syntax.
> Line 399:
> "http://hadoop.apache.org/docs/current/hadoop-project-dist/; +
> This link is not tied to a version, so could be inaccurate.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12930) [Umbrella] Dynamic subcommands for hadoop shell scripts
[ https://issues.apache.org/jira/browse/HADOOP-12930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285795#comment-15285795 ] Hudson commented on HADOOP-12930: - FAILURE: Integrated in Hadoop-trunk-Commit #9772 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/9772/]) HADOOP-12930. Dynamic subcommands for hadoop shell scripts (aw) (aw: rev 730bc746f9ac6e045e94dc2bc622b16de0159b4b) * hadoop-mapreduce-project/bin/mapred * hadoop-assemblies/src/main/resources/assemblies/hadoop-tools.xml * hadoop-hdfs-project/hadoop-hdfs/src/test/scripts/hdfs_subcommands.bats * hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs * hadoop-tools/hadoop-streaming/src/main/java/org/apache/hadoop/streaming/LoadTypedBytes.java * hadoop-tools/hadoop-streaming/src/main/shellprofile.d/hadoop-streaming.sh * hadoop-common-project/hadoop-common/src/site/markdown/UnixShellGuide.md * hadoop-tools/hadoop-gridmix/src/main/shellprofile.d/hadoop-gridmix.sh * hadoop-hdfs-project/hadoop-hdfs/src/main/shellprofile.d/hdfs.sh * hadoop-hdfs-project/hadoop-hdfs/src/test/scripts/hdfs-functions_test_helper.bash * hadoop-yarn-project/hadoop-yarn/shellprofile.d/hadoop-yarn.sh * hadoop-common-project/hadoop-common/src/main/bin/hadoop * hadoop-common-project/hadoop-common/src/test/scripts/hadoop-functions_test_helper.bash * hadoop-tools/hadoop-streaming/src/site/markdown/HadoopStreaming.md.vm * hadoop-hdfs-project/hadoop-hdfs/src/main/shellprofile.d/hadoop-hdfs.sh * hadoop-tools/hadoop-extras/src/main/shellprofile.d/hadoop-extras.sh * hadoop-hdfs-project/hadoop-hdfs/pom.xml * hadoop-common-project/hadoop-common/src/test/scripts/hadoop_subcommands.bats * hadoop-mapreduce-project/shellprofile.d/hadoop-mapreduce.sh * hadoop-yarn-project/hadoop-yarn/shellprofile.d/yarn.sh * hadoop-hdfs-project/hadoop-hdfs/src/test/scripts/run-bats.sh * hadoop-tools/hadoop-distcp/src/main/shellprofile.d/hadoop-distcp.sh * hadoop-tools/hadoop-rumen/src/main/shellprofile.d/hadoop-rumen.sh * hadoop-yarn-project/hadoop-yarn/bin/yarn * hadoop-tools/hadoop-streaming/src/main/java/org/apache/hadoop/streaming/HadoopStreaming.java * hadoop-tools/hadoop-rumen/src/site/markdown/Rumen.md.vm * hadoop-tools/hadoop-archives/src/main/shellprofile.d/hadoop-archives.sh * hadoop-mapreduce-project/shellprofile.d/mapreduce.sh * hadoop-tools/hadoop-archive-logs/src/main/shellprofile.d/hadoop-archive-logs.sh * hadoop-tools/hadoop-streaming/src/main/java/org/apache/hadoop/streaming/DumpTypedBytes.java > [Umbrella] Dynamic subcommands for hadoop shell scripts > --- > > Key: HADOOP-12930 > URL: https://issues.apache.org/jira/browse/HADOOP-12930 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 3.0.0-alpha1 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer >Priority: Critical > Fix For: 3.0.0-alpha1 > > Attachments: HADOOP-12930.00.patch > > > Umbrella for converting hadoop, hdfs, mapred, and yarn to allow for dynamic > subcommands. See first comment for more details. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13097) hadoop-mapred unit tests for dynamic commands
[ https://issues.apache.org/jira/browse/HADOOP-13097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-13097: -- Issue Type: Test (was: Sub-task) Parent: (was: HADOOP-12930) > hadoop-mapred unit tests for dynamic commands > - > > Key: HADOOP-13097 > URL: https://issues.apache.org/jira/browse/HADOOP-13097 > Project: Hadoop Common > Issue Type: Test > Components: scripts, test >Reporter: Allen Wittenauer > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13096) hadoop-yarn unit tests for dynamic commands
[ https://issues.apache.org/jira/browse/HADOOP-13096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-13096: -- Issue Type: Test (was: Sub-task) Parent: (was: HADOOP-12930) > hadoop-yarn unit tests for dynamic commands > --- > > Key: HADOOP-13096 > URL: https://issues.apache.org/jira/browse/HADOOP-13096 > Project: Hadoop Common > Issue Type: Test > Components: scripts, test >Reporter: Allen Wittenauer > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-12930) [Umbrella] Dynamic subcommands for hadoop shell scripts
[ https://issues.apache.org/jira/browse/HADOOP-12930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-12930: -- Fix Version/s: 3.0.0-alpha1 > [Umbrella] Dynamic subcommands for hadoop shell scripts > --- > > Key: HADOOP-12930 > URL: https://issues.apache.org/jira/browse/HADOOP-12930 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 3.0.0-alpha1 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer >Priority: Critical > Fix For: 3.0.0-alpha1 > > Attachments: HADOOP-12930.00.patch > > > Umbrella for converting hadoop, hdfs, mapred, and yarn to allow for dynamic > subcommands. See first comment for more details. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-12930) [Umbrella] Dynamic subcommands for hadoop shell scripts
[ https://issues.apache.org/jira/browse/HADOOP-12930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HADOOP-12930. --- Resolution: Fixed Committed to trunk > [Umbrella] Dynamic subcommands for hadoop shell scripts > --- > > Key: HADOOP-12930 > URL: https://issues.apache.org/jira/browse/HADOOP-12930 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 3.0.0-alpha1 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer >Priority: Critical > Fix For: 3.0.0-alpha1 > > Attachments: HADOOP-12930.00.patch > > > Umbrella for converting hadoop, hdfs, mapred, and yarn to allow for dynamic > subcommands. See first comment for more details. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-12930) [Umbrella] Dynamic subcommands for hadoop shell scripts
[ https://issues.apache.org/jira/browse/HADOOP-12930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-12930: -- Attachment: HADOOP-12930.00.patch Vote has passed. Here's the squashed commit of the sub-tasks that will be committed. Thanks everyone! -00: * squashed commit > [Umbrella] Dynamic subcommands for hadoop shell scripts > --- > > Key: HADOOP-12930 > URL: https://issues.apache.org/jira/browse/HADOOP-12930 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 3.0.0-alpha1 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer >Priority: Critical > Attachments: HADOOP-12930.00.patch > > > Umbrella for converting hadoop, hdfs, mapred, and yarn to allow for dynamic > subcommands. See first comment for more details. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13154) S3AFileSystem printAmazonServiceException/printAmazonClientException appear copy & paste of AWS examples
[
https://issues.apache.org/jira/browse/HADOOP-13154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285746#comment-15285746
]
Vinod Kumar Vavilapalli commented on HADOOP-13154:
--
[~steve_l], looks like they've been around for a while. git blame on both these
methods point to the following commit, back to you
{code}
commit 27c4e90efce04e1b1302f668b5eb22412e00d033
Author: Steve Loughran
Date: Thu May 12 19:24:20 2016 +0100
HADOOP-13028 add low level counter metrics for S3A; use in read performance
tests. contributed by: stevel
patch includes
HADOOP-12844 Recover when S3A fails on IOException in read()
HADOOP-13058 S3A FS fails during init against a read-only FS if multipart
purge
HADOOP-13047 S3a Forward seek in stream length to be configurable
{code}
> S3AFileSystem printAmazonServiceException/printAmazonClientException appear
> copy & paste of AWS examples
>
>
> Key: HADOOP-13154
> URL: https://issues.apache.org/jira/browse/HADOOP-13154
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 2.7.2
>Reporter: Steve Loughran
>Priority: Blocker
>
> The logging code in {{S3AFileSystem.printAmazonServiceException()}} and
> {{printAmazonClientException}} appear to be paste + edits of the example code
> in the amazon SDK, such as
> [http://docs.aws.amazon.com/AmazonS3/latest/dev/ListingObjectKeysUsingJava.html]]
> Either we review the license to validate it, and add credits to the code if
> compatible, or we rework. HADOOP-13130 would be the place to do that, as it
> is changing exception handling anyway.
> tagging as blocker as it is license related
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-12666) Support Microsoft Azure Data Lake - as a file system in Hadoop
[ https://issues.apache.org/jira/browse/HADOOP-12666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Douglas updated HADOOP-12666: --- Attachment: HADOOP-12666-014.patch Thanks, [~cnauroth]. The checkstyle warning is being addressed in HADOOP-13160. Updated the patch to fix the test for HADOOP-12738. > Support Microsoft Azure Data Lake - as a file system in Hadoop > -- > > Key: HADOOP-12666 > URL: https://issues.apache.org/jira/browse/HADOOP-12666 > Project: Hadoop Common > Issue Type: New Feature > Components: fs, fs/azure, tools >Reporter: Vishwajeet Dusane >Assignee: Vishwajeet Dusane > Attachments: Create_Read_Hadoop_Adl_Store_Semantics.pdf, > HADOOP-12666-002.patch, HADOOP-12666-003.patch, HADOOP-12666-004.patch, > HADOOP-12666-005.patch, HADOOP-12666-006.patch, HADOOP-12666-007.patch, > HADOOP-12666-008.patch, HADOOP-12666-009.patch, HADOOP-12666-010.patch, > HADOOP-12666-011.patch, HADOOP-12666-012.patch, HADOOP-12666-013.patch, > HADOOP-12666-014.patch, HADOOP-12666-1.patch > > Original Estimate: 336h > Time Spent: 336h > Remaining Estimate: 0h > > h2. Description > This JIRA describes a new file system implementation for accessing Microsoft > Azure Data Lake Store (ADL) from within Hadoop. This would enable existing > Hadoop applications such has MR, HIVE, Hbase etc.., to use ADL store as > input or output. > > ADL is ultra-high capacity, Optimized for massive throughput with rich > management and security features. More details available at > https://azure.microsoft.com/en-us/services/data-lake-store/ -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12782) Faster LDAP group name resolution with ActiveDirectory
[
https://issues.apache.org/jira/browse/HADOOP-12782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285717#comment-15285717
]
Hadoop QA commented on HADOOP-12782:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 7s
{color} | {color:red} root in trunk failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 47s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 10s
{color} | {color:red} root in trunk failed with JDK v1.7.0_101. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s {color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m 18s
{color} | {color:red} hadoop-common in trunk failed. {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 13s
{color} | {color:red} hadoop-common in trunk failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 0s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 12s
{color} | {color:red} hadoop-common in trunk failed with JDK v1.7.0_101.
{color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 12s
{color} | {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 50s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m 50s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 11s
{color} | {color:red} root in the patch failed with JDK v1.7.0_101. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 11s {color}
| {color:red} root in the patch failed with JDK v1.7.0_101. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s {color} | {color:green} hadoop-common-project/hadoop-common: The patch
generated 0 new + 40 unchanged - 7 fixed = 40 total (was 47) {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m 14s
{color} | {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 12s
{color} | {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 2s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 12s
{color} | {color:red} hadoop-common in the patch failed with JDK v1.7.0_101.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m 3s
{color} | {color:green} hadoop-common in the patch passed with JDK v1.8.0_91.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 11s {color}
| {color:red} hadoop-common in the patch failed with JDK v1.7.0_101. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 50s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:cf2ee45 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804294/HADOOP-12782.009.patch
|
| JIRA Issue | HADOOP-12782 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle xml |
| uname | Linux
[jira] [Commented] (HADOOP-13112) Change CredentialShell to use CommandShell base class
[
https://issues.apache.org/jira/browse/HADOOP-13112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285682#comment-15285682
]
Hadoop QA commented on HADOOP-13112:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 8s
{color} | {color:red} root in trunk failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 19s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 10s
{color} | {color:red} root in trunk failed with JDK v1.7.0_101. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s {color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m 17s
{color} | {color:red} hadoop-common in trunk failed. {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 11s
{color} | {color:red} hadoop-common in trunk failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 10s
{color} | {color:green} trunk passed with JDK v1.7.0_101 {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 13s
{color} | {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 40s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m 40s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 10s
{color} | {color:red} root in the patch failed with JDK v1.7.0_101. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 11s {color}
| {color:red} root in the patch failed with JDK v1.7.0_101. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s {color} | {color:green} hadoop-common-project/hadoop-common: The patch
generated 0 new + 3 unchanged - 1 fixed = 3 total (was 4) {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m 14s
{color} | {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 12s
{color} | {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 0s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 14s
{color} | {color:red} hadoop-common in the patch failed with JDK v1.7.0_101.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 32s {color}
| {color:red} hadoop-common in the patch failed with JDK v1.8.0_91. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 12s {color}
| {color:red} hadoop-common in the patch failed with JDK v1.7.0_101. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 27s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_91 Failed junit tests | hadoop.ha.TestZKFailoverController |
| | hadoop.security.ssl.TestReloadingX509TrustManager |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:cf2ee45 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804285/HADOOP-13112.04.patch
|
| JIRA Issue | HADOOP-13112 |
| Optional
[jira] [Updated] (HADOOP-12782) Faster LDAP group name resolution with ActiveDirectory
[ https://issues.apache.org/jira/browse/HADOOP-12782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HADOOP-12782: - Attachment: HADOOP-12782.009.patch Rev09: fixed checkstyle warning. The test failure is unrelated. > Faster LDAP group name resolution with ActiveDirectory > -- > > Key: HADOOP-12782 > URL: https://issues.apache.org/jira/browse/HADOOP-12782 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang > Attachments: HADOOP-12782.001.patch, HADOOP-12782.002.patch, > HADOOP-12782.003.patch, HADOOP-12782.004.patch, HADOOP-12782.005.patch, > HADOOP-12782.006.patch, HADOOP-12782.007.patch, HADOOP-12782.008.patch, > HADOOP-12782.009.patch > > > The typical LDAP group name resolution works well under typical scenarios. > However, we have seen cases where a user is mapped to many groups (in an > extreme case, a user is mapped to more than 100 groups). The way it's being > implemented now makes this case super slow resolving groups from > ActiveDirectory. > The current LDAP group resolution implementation sends two queries to a > ActiveDirectory server. The first query returns a user object, which contains > DN (distinguished name). The second query looks for groups where the user DN > is a member. If a user is mapped to many groups, the second query returns all > group objects associated with the user, and is thus very slow. > After studying a user object in ActiveDirectory, I found a user object > actually contains a "memberOf" field, which is the DN of all group objects > where the user belongs to. Assuming that an organization has no recursive > group relation (that is, a user A is a member of group G1, and group G1 is a > member of group G2), we can use this properties to avoid the second query, > which can potentially run very slow. > I propose that we add a configuration to only enable this feature for users > who want to reduce group resolution time and who does not have recursive > groups, so that existing behavior will not be broken. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12782) Faster LDAP group name resolution with ActiveDirectory
[
https://issues.apache.org/jira/browse/HADOOP-12782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285630#comment-15285630
]
Hadoop QA commented on HADOOP-12782:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
58s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 23s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 45s
{color} | {color:green} trunk passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 0s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
34s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 4s
{color} | {color:green} trunk passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 19s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 19s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 45s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 45s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 24s
{color} | {color:red} hadoop-common-project/hadoop-common: The patch generated
1 new + 40 unchanged - 7 fixed = 41 total (was 47) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 56s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 8s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 19s {color}
| {color:red} hadoop-common in the patch failed with JDK v1.8.0_91. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m 14s
{color} | {color:green} hadoop-common in the patch passed with JDK v1.7.0_101.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 62m 43s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_91 Failed junit tests | hadoop.ha.TestZKFailoverController |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:cf2ee45 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804272/HADOOP-12782.008.patch
|
| JIRA Issue | HADOOP-12782 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle xml |
| uname | Linux
[jira] [Commented] (HADOOP-12893) Verify LICENSE.txt and NOTICE.txt
[ https://issues.apache.org/jira/browse/HADOOP-12893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285616#comment-15285616 ] Andrew Wang commented on HADOOP-12893: -- Thanks Sean, that's the page I hadn't read yet :) I really wish the Licensing HOWTO gave examples for how this applies to a typical Java project. It talks a lot about source releases, which are the least common way for people to consume our artifacts. For a long time we were also under the impression that we could just provide the binary artifacts as a "convenience" while the source tarball was the only "official" release artifact. Now we know that's definitely not okay. I'm going to fix up the patch based on my above comments. So far no one has raised any objections on the dev list, so we'll have the "good faith" level of compliance to unblock the releases. I'll also attempt the pom work for generating specific L, but no promises. > Verify LICENSE.txt and NOTICE.txt > - > > Key: HADOOP-12893 > URL: https://issues.apache.org/jira/browse/HADOOP-12893 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.8.0, 2.7.3, 2.6.5, 3.0.0-alpha1 >Reporter: Allen Wittenauer >Assignee: Xiao Chen >Priority: Blocker > Attachments: HADOOP-12893.01.patch > > > We have many bundled dependencies in both the source and the binary artifacts > that are not in LICENSE.txt and NOTICE.txt. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11858) [JDK8] Set minimum version of Hadoop 3 to JDK 8
[ https://issues.apache.org/jira/browse/HADOOP-11858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285611#comment-15285611 ] Hudson commented on HADOOP-11858: - FAILURE: Integrated in Hadoop-trunk-Commit #9770 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/9770/]) HADOOP-11858. [JDK8] Set minimum version of Hadoop 3 to JDK 8. (wang: rev 4b55642b9d836691592405805c181d12c2ed7e50) * hadoop-project/pom.xml * pom.xml * BUILDING.txt > [JDK8] Set minimum version of Hadoop 3 to JDK 8 > --- > > Key: HADOOP-11858 > URL: https://issues.apache.org/jira/browse/HADOOP-11858 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Robert Kanter >Assignee: Robert Kanter >Priority: Blocker > Fix For: 3.0.0-alpha1 > > Attachments: HADOOP-11858.001.patch, HADOOP-11858.002.patch, > HADOOP-11858.003.patch, commit.ogv > > > Set minimum version of trunk to JDK 8 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13140) GlobalStorageStatistics should check null FileSystem scheme to avoid NPE
[
https://issues.apache.org/jira/browse/HADOOP-13140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285598#comment-15285598
]
Mingliang Liu commented on HADOOP-13140:
{{FileSystem#get}} has a configuration parameter (and
{{FileSystem#getDefaultUri}}). But the existing {{FileSystem#getStatistics()}}
does not have the access to the configuration object when initializing, so it's
hard to get default URI. Any thoughts?
> GlobalStorageStatistics should check null FileSystem scheme to avoid NPE
>
>
> Key: HADOOP-13140
> URL: https://issues.apache.org/jira/browse/HADOOP-13140
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 2.8.0
>Reporter: Brahma Reddy Battula
>Assignee: Mingliang Liu
> Attachments: HADOOP-13140.000.patch
>
>
> {{org.apache.hadoop.fs.GlobalStorageStatistics#put}} is not checking the null
> scheme, and the internal map will complain NPE. This was reported by a flaky
> test {{TestFileSystemApplicationHistoryStore}}. Thanks [~brahmareddy] for
> reporting.
> To address this,
> # Fix the test by providing a valid URI, e.g. {{file:///}}
> # Guard the null scheme in {{GlobalStorageStatistics#put}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13112) Change CredentialShell to use CommandShell base class
[
https://issues.apache.org/jira/browse/HADOOP-13112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthew Paduano updated HADOOP-13112:
-
Attachment: HADOOP-13112.04.patch
Patch 04 addresses the merge issue from HADOOP-12942
{code}
for patch #04:
.../org/apache/hadoop/crypto/key/KeyShell.java | 180 +++-
.../hadoop/security/alias/CredentialShell.java | 176
.../java/org/apache/hadoop/tools/CommandShell.java |4
3 files changed, 136 insertions(+), 224 deletions(-)
---
T E S T S
---
Running org.apache.hadoop.crypto.key.TestKeyShell
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0
Running org.apache.hadoop.security.alias.TestCredShell
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0
Running org.apache.hadoop.security.token.TestDtUtilShell
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0
Running org.apache.hadoop.tools.TestCommandShell
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
Results: Tests run: 31, Failures: 0, Errors: 0, Skipped: 0
{code}
> Change CredentialShell to use CommandShell base class
> -
>
> Key: HADOOP-13112
> URL: https://issues.apache.org/jira/browse/HADOOP-13112
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Matthew Paduano
>Assignee: Matthew Paduano
>Priority: Minor
> Attachments: HADOOP-13112.01.patch, HADOOP-13112.02.patch,
> HADOOP-13112.03.patch, HADOOP-13112.04.patch
>
>
> org.apache.hadoop.tools.CommandShell is a base class created for use by
> DtUtilShell. It was inspired by CredentialShell and much of it was taken
> verbatim. It should be a simple change to get CredentialShell to use the
> base class and simplify its code without changing its functionality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-11858) [JDK8] Set minimum version of Hadoop 3 to JDK 8
[ https://issues.apache.org/jira/browse/HADOOP-11858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HADOOP-11858: - Attachment: commit.ogv Per Steve's request on the dev list, I'm also attaching a video of this patch being committed for posterity. > [JDK8] Set minimum version of Hadoop 3 to JDK 8 > --- > > Key: HADOOP-11858 > URL: https://issues.apache.org/jira/browse/HADOOP-11858 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Robert Kanter >Assignee: Robert Kanter >Priority: Blocker > Fix For: 3.0.0-alpha1 > > Attachments: HADOOP-11858.001.patch, HADOOP-11858.002.patch, > HADOOP-11858.003.patch, commit.ogv > > > Set minimum version of trunk to JDK 8 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13146) Refactor RetryInvocationHandler
[ https://issues.apache.org/jira/browse/HADOOP-13146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285584#comment-15285584 ] Hudson commented on HADOOP-13146: - FAILURE: Integrated in Hadoop-trunk-Commit #9769 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/9769/]) HADOOP-13146. Refactor RetryInvocationHandler. Contributed by Tsz Wo (jing9: rev a9a8297cad4122961b34265c0a31d87134a4a028) * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/retry/TestRetryProxy.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/retry/RetryInvocationHandler.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/retry/FailoverProxyProvider.java > Refactor RetryInvocationHandler > --- > > Key: HADOOP-13146 > URL: https://issues.apache.org/jira/browse/HADOOP-13146 > Project: Hadoop Common > Issue Type: Improvement > Components: io >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze >Priority: Minor > Fix For: 2.9.0 > > Attachments: c13146_20160513.patch, c13146_20160513b.patch, > c13146_20160514.patch, c13146_20160514b.patch, c13146_20160516.patch > > > - The exception handling is quite long. It is better to refactor it to a > separated method. > - The failover logic and synchronization can be moved to a new inner class. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-11858) [JDK8] Set minimum version of Hadoop 3 to JDK 8
[ https://issues.apache.org/jira/browse/HADOOP-11858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-11858. -- Resolution: Fixed Fix Version/s: 3.0.0-alpha1 Target Version/s: (was: ) Committed to trunk. Thanks Robert for the patch and everyone for reviewing! > [JDK8] Set minimum version of Hadoop 3 to JDK 8 > --- > > Key: HADOOP-11858 > URL: https://issues.apache.org/jira/browse/HADOOP-11858 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Robert Kanter >Assignee: Robert Kanter >Priority: Blocker > Fix For: 3.0.0-alpha1 > > Attachments: HADOOP-11858.001.patch, HADOOP-11858.002.patch, > HADOOP-11858.003.patch > > > Set minimum version of trunk to JDK 8 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-12687) SecureUtil#getByName should also try to resolve direct hostname, incase multiple loopback addresses are present in /etc/hosts
[
https://issues.apache.org/jira/browse/HADOOP-12687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kumar Vavilapalli updated HADOOP-12687:
-
Target Version/s: 2.8.0, 3.0.0-alpha1
Priority: Blocker (was: Major)
I am going to bump this to be a blocker for all outstanding releases.
This is causing too much pain, many JIRAs are running into this, and we are
going ahead and committing patches relatively blindly.
Either this gets fixed or INFRA-11150 moves forward.
> SecureUtil#getByName should also try to resolve direct hostname, incase
> multiple loopback addresses are present in /etc/hosts
> -
>
> Key: HADOOP-12687
> URL: https://issues.apache.org/jira/browse/HADOOP-12687
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Junping Du
>Assignee: Sunil G
>Priority: Blocker
> Labels: security
> Attachments: 0001-YARN-4352.patch, 0002-YARN-4352.patch,
> 0003-HADOOP-12687.patch, 0004-HADOOP-12687.patch
>
>
> From
> https://builds.apache.org/job/PreCommit-YARN-Build/9661/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_79.txt,
> we can see the tests in TestYarnClient, TestAMRMClient and TestNMClient get
> timeout which can be reproduced locally.
> When {{/etc/hosts}} has multiple loopback entries,
> {{InetAddress.getByName(null)}} will be returning the first entry present in
> etc/hosts. Hence its possible that machine hostname can be second in list and
> cause {{UnKnownHostException}}.
> Suggesting a direct resolve for such hostname scenarios.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11858) [JDK8] Set minimum version of Hadoop 3 to JDK 8
[ https://issues.apache.org/jira/browse/HADOOP-11858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285575#comment-15285575 ] Andrew Wang commented on HADOOP-11858: -- I think we can use my +1 for 002 and your +1 for the 003 diff. Thanks Robert, I'll commit this shortly. > [JDK8] Set minimum version of Hadoop 3 to JDK 8 > --- > > Key: HADOOP-11858 > URL: https://issues.apache.org/jira/browse/HADOOP-11858 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Robert Kanter >Assignee: Robert Kanter >Priority: Blocker > Attachments: HADOOP-11858.001.patch, HADOOP-11858.002.patch, > HADOOP-11858.003.patch > > > Set minimum version of trunk to JDK 8 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11858) [JDK8] Set minimum version of Hadoop 3 to JDK 8
[ https://issues.apache.org/jira/browse/HADOOP-11858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285566#comment-15285566 ] Robert Kanter commented on HADOOP-11858: I've confirmed that with the 003 patch, the maxPermSize warning goes away and everything builds with Java 8, but the enforcer blocks Java 7 and 6. So +1 from me, but I don't know if you want to wait for a less "biased" +1 from someone else :) > [JDK8] Set minimum version of Hadoop 3 to JDK 8 > --- > > Key: HADOOP-11858 > URL: https://issues.apache.org/jira/browse/HADOOP-11858 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Robert Kanter >Assignee: Robert Kanter >Priority: Blocker > Attachments: HADOOP-11858.001.patch, HADOOP-11858.002.patch, > HADOOP-11858.003.patch > > > Set minimum version of trunk to JDK 8 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11858) [JDK8] Set minimum version of Hadoop 3 to JDK 8
[ https://issues.apache.org/jira/browse/HADOOP-11858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285550#comment-15285550 ] Andrew Wang commented on HADOOP-11858: -- Not sure if we can really run this through pre-commit since we can't run full test runs anymore, and all we're really looking for is validation that it builds. If someone else +1's, I'll go ahead and commit (full credit to Robert). > [JDK8] Set minimum version of Hadoop 3 to JDK 8 > --- > > Key: HADOOP-11858 > URL: https://issues.apache.org/jira/browse/HADOOP-11858 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Robert Kanter >Assignee: Robert Kanter >Priority: Blocker > Attachments: HADOOP-11858.001.patch, HADOOP-11858.002.patch, > HADOOP-11858.003.patch > > > Set minimum version of trunk to JDK 8 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13146) Refactor RetryInvocationHandler
[ https://issues.apache.org/jira/browse/HADOOP-13146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HADOOP-13146: --- Resolution: Fixed Fix Version/s: 2.9.0 Status: Resolved (was: Patch Available) I've committed this to trunk and branch-2. Thanks Nicholas for the contribution! > Refactor RetryInvocationHandler > --- > > Key: HADOOP-13146 > URL: https://issues.apache.org/jira/browse/HADOOP-13146 > Project: Hadoop Common > Issue Type: Improvement > Components: io >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze >Priority: Minor > Fix For: 2.9.0 > > Attachments: c13146_20160513.patch, c13146_20160513b.patch, > c13146_20160514.patch, c13146_20160514b.patch, c13146_20160516.patch > > > - The exception handling is quite long. It is better to refactor it to a > separated method. > - The failover logic and synchronization can be moved to a new inner class. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-11858) [JDK8] Set minimum version of Hadoop 3 to JDK 8
[ https://issues.apache.org/jira/browse/HADOOP-11858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HADOOP-11858: - Attachment: HADOOP-11858.003.patch Here's a patch getting rid of the MaxPermSize flag. I tested by doing a full build with this patch; enforcer rule triggered correctly. > [JDK8] Set minimum version of Hadoop 3 to JDK 8 > --- > > Key: HADOOP-11858 > URL: https://issues.apache.org/jira/browse/HADOOP-11858 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Robert Kanter >Assignee: Robert Kanter >Priority: Blocker > Attachments: HADOOP-11858.001.patch, HADOOP-11858.002.patch, > HADOOP-11858.003.patch > > > Set minimum version of trunk to JDK 8 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13146) Refactor RetryInvocationHandler
[ https://issues.apache.org/jira/browse/HADOOP-13146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285533#comment-15285533 ] Jing Zhao commented on HADOOP-13146: +1 for the latest patch. > Refactor RetryInvocationHandler > --- > > Key: HADOOP-13146 > URL: https://issues.apache.org/jira/browse/HADOOP-13146 > Project: Hadoop Common > Issue Type: Improvement > Components: io >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze >Priority: Minor > Attachments: c13146_20160513.patch, c13146_20160513b.patch, > c13146_20160514.patch, c13146_20160514b.patch, c13146_20160516.patch > > > - The exception handling is quite long. It is better to refactor it to a > separated method. > - The failover logic and synchronization can be moved to a new inner class. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-12782) Faster LDAP group name resolution with ActiveDirectory
[ https://issues.apache.org/jira/browse/HADOOP-12782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HADOOP-12782: - Attachment: HADOOP-12782.008.patch Hi Kai, thanks again for your review. Good catch! I've refactored the code to include the suggestion you mentioned. However, not all mock code is moved to the base test class, because some code is specific to one test, and might cause confusion if done so. > Faster LDAP group name resolution with ActiveDirectory > -- > > Key: HADOOP-12782 > URL: https://issues.apache.org/jira/browse/HADOOP-12782 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang > Attachments: HADOOP-12782.001.patch, HADOOP-12782.002.patch, > HADOOP-12782.003.patch, HADOOP-12782.004.patch, HADOOP-12782.005.patch, > HADOOP-12782.006.patch, HADOOP-12782.007.patch, HADOOP-12782.008.patch > > > The typical LDAP group name resolution works well under typical scenarios. > However, we have seen cases where a user is mapped to many groups (in an > extreme case, a user is mapped to more than 100 groups). The way it's being > implemented now makes this case super slow resolving groups from > ActiveDirectory. > The current LDAP group resolution implementation sends two queries to a > ActiveDirectory server. The first query returns a user object, which contains > DN (distinguished name). The second query looks for groups where the user DN > is a member. If a user is mapped to many groups, the second query returns all > group objects associated with the user, and is thus very slow. > After studying a user object in ActiveDirectory, I found a user object > actually contains a "memberOf" field, which is the DN of all group objects > where the user belongs to. Assuming that an organization has no recursive > group relation (that is, a user A is a member of group G1, and group G1 is a > member of group G2), we can use this properties to avoid the second query, > which can potentially run very slow. > I propose that we add a configuration to only enable this feature for users > who want to reduce group resolution time and who does not have recursive > groups, so that existing behavior will not be broken. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13146) Refactor RetryInvocationHandler
[
https://issues.apache.org/jira/browse/HADOOP-13146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285506#comment-15285506
]
Hadoop QA commented on HADOOP-13146:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
51s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 11s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 49s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 0s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
41s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 59s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 7s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 44s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 44s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 35s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m 35s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 26s
{color} | {color:red} hadoop-common-project/hadoop-common: The patch generated
1 new + 44 unchanged - 5 fixed = 45 total (was 49) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 1s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 0s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 10s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m 48s
{color} | {color:green} hadoop-common in the patch passed with JDK v1.8.0_91.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m 34s
{color} | {color:green} hadoop-common in the patch passed with JDK v1.7.0_95.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 65m 24s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:cf2ee45 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804249/c13146_20160516.patch
|
| JIRA Issue | HADOOP-13146 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 739e9571022e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk /
[jira] [Commented] (HADOOP-13157) Follow-on improvements to HADOOP-12942
[
https://issues.apache.org/jira/browse/HADOOP-13157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285504#comment-15285504
]
Larry McCay commented on HADOOP-13157:
--
1. We can consider adding it to providerUtils:
/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ProviderUtils.java
2. Wouldn't that require every release go and change the link? Perhaps we
should change to just directing them to the page rather than providing an
actual?
> Follow-on improvements to HADOOP-12942
> --
>
> Key: HADOOP-13157
> URL: https://issues.apache.org/jira/browse/HADOOP-13157
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Affects Versions: 2.8.0
>Reporter: Mike Yoder
>Assignee: Mike Yoder
>
> [~andrew.wang] had some follow-up code review comments from HADOOP-12942.
> Hence this issue.
> Ping [~lmccay] as well.
> The comments:
> {quote}
> Overall this looks okay, the only correctness question I have is about the
> difference in behavior when the pwfile doesn't exist.
> The rest are all nits, would be nice to do these cleanups though.
> File
> hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/JavaKeyStoreProvider.java:
> Line 147:
> Could this be a static helper?
> Line 161: new
> The javadoc says it returns null in this situation. This is also a difference
> from the implementation in the AbstractJKSP. Intentional?
> Line 175: private void locateKeystore() throws IOException {
> static helper? for the construct*Path methods too?
> File
> hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/KeyShell.java:
> Line 50: @VisibleForTesting public static final String NO_VALID_PROVIDERS =
> FYI for the future, our coding style is to put annotations on their own
> separate line.
> File
> hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/alias/AbstractJavaKeyStoreProvider.java:
> Line 326: private char[] locatePassword() throws IOException {
> this method looks very similar to the one in JavaKeyStoreProvider, except the
> env var it looks for is different, is there potential for code reuse?
> Line 394: "o In the environment variable " +
> Using a "*" is the usual way of doing a bullet point, e.g. markdown and wiki
> syntax.
> Line 399:
> "http://hadoop.apache.org/docs/current/hadoop-project-dist/; +
> This link is not tied to a version, so could be inaccurate.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13157) Follow-on improvements to HADOOP-12942
[
https://issues.apache.org/jira/browse/HADOOP-13157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285484#comment-15285484
]
Andrew Wang commented on HADOOP-13157:
--
1. If we make it a static helper method, it can go pretty much anywhere. Larry
knows this area of the code better than me, but I can search around if that'd
help.
2. I'd be happy if we just pin it to a given version of the docs. This is the
URL for the 2.7.2 docs, you can expect the 2.8.0 docs to use the same URL
format: http://hadoop.apache.org/docs/r2.7.2/
> Follow-on improvements to HADOOP-12942
> --
>
> Key: HADOOP-13157
> URL: https://issues.apache.org/jira/browse/HADOOP-13157
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Affects Versions: 2.8.0
>Reporter: Mike Yoder
>Assignee: Mike Yoder
>
> [~andrew.wang] had some follow-up code review comments from HADOOP-12942.
> Hence this issue.
> Ping [~lmccay] as well.
> The comments:
> {quote}
> Overall this looks okay, the only correctness question I have is about the
> difference in behavior when the pwfile doesn't exist.
> The rest are all nits, would be nice to do these cleanups though.
> File
> hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/JavaKeyStoreProvider.java:
> Line 147:
> Could this be a static helper?
> Line 161: new
> The javadoc says it returns null in this situation. This is also a difference
> from the implementation in the AbstractJKSP. Intentional?
> Line 175: private void locateKeystore() throws IOException {
> static helper? for the construct*Path methods too?
> File
> hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/KeyShell.java:
> Line 50: @VisibleForTesting public static final String NO_VALID_PROVIDERS =
> FYI for the future, our coding style is to put annotations on their own
> separate line.
> File
> hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/alias/AbstractJavaKeyStoreProvider.java:
> Line 326: private char[] locatePassword() throws IOException {
> this method looks very similar to the one in JavaKeyStoreProvider, except the
> env var it looks for is different, is there potential for code reuse?
> Line 394: "o In the environment variable " +
> Using a "*" is the usual way of doing a bullet point, e.g. markdown and wiki
> syntax.
> Line 399:
> "http://hadoop.apache.org/docs/current/hadoop-project-dist/; +
> This link is not tied to a version, so could be inaccurate.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13159) Fix potential NPE in Metrics2 source for DecayRpcScheduler
[ https://issues.apache.org/jira/browse/HADOOP-13159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285369#comment-15285369 ] Daniel Templeton commented on HADOOP-13159: --- Looks good to me. +1 (non-binding) > Fix potential NPE in Metrics2 source for DecayRpcScheduler > -- > > Key: HADOOP-13159 > URL: https://issues.apache.org/jira/browse/HADOOP-13159 > Project: Hadoop Common > Issue Type: Bug > Components: ipc >Affects Versions: 2.8.0 >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Attachments: HADOOP-13159.00.patch, HADOOP-13159.01.patch > > > HADOOP-12985 introduced a few metrics2 counters for DecayRpcScheduler. The > counters should be initialized before the updater thread is created/executed. > Otherwise, this is a chance of NPE if the updater thread get executed before > the counters are fully initialized. I will post a patch for it shortly. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13160) Suppress checkstyle JavadocPackage error for test source files
[ https://issues.apache.org/jira/browse/HADOOP-13160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285350#comment-15285350 ] John Zhuge commented on HADOOP-13160: - Tested the patch on Mac and Ubuntu. Not tested on Windows, could someone with Windows Hadoop env do a quick test? Thanks. > Suppress checkstyle JavadocPackage error for test source files > -- > > Key: HADOOP-13160 > URL: https://issues.apache.org/jira/browse/HADOOP-13160 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 2.7.2 >Reporter: John Zhuge >Assignee: John Zhuge >Priority: Minor > Attachments: HADOOP-13160.001.patch > > > Suppress "Missing package-info.java" checkstyle error for test source files. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13160) Suppress checkstyle JavadocPackage error for test source files
[
https://issues.apache.org/jira/browse/HADOOP-13160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285346#comment-15285346
]
John Zhuge commented on HADOOP-13160:
-
Run the following script to find out the number of all test files in all Hadoop
components with "Missing package-info" checkstyle error:
{code}
find * -name 'checkstyle-errors.xml' | xargs cat | grep '^ Suppress checkstyle JavadocPackage error for test source files
> --
>
> Key: HADOOP-13160
> URL: https://issues.apache.org/jira/browse/HADOOP-13160
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 2.7.2
>Reporter: John Zhuge
>Assignee: John Zhuge
>Priority: Minor
> Attachments: HADOOP-13160.001.patch
>
>
> Suppress "Missing package-info.java" checkstyle error for test source files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13160) Suppress checkstyle JavadocPackage error for test source files
[ https://issues.apache.org/jira/browse/HADOOP-13160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HADOOP-13160: Status: Patch Available (was: In Progress) > Suppress checkstyle JavadocPackage error for test source files > -- > > Key: HADOOP-13160 > URL: https://issues.apache.org/jira/browse/HADOOP-13160 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 2.7.2 >Reporter: John Zhuge >Assignee: John Zhuge >Priority: Minor > Attachments: HADOOP-13160.001.patch > > > Suppress "Missing package-info.java" checkstyle error for test source files. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13160) Suppress checkstyle JavadocPackage error for test source files
[ https://issues.apache.org/jira/browse/HADOOP-13160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HADOOP-13160: Attachment: HADOOP-13160.001.patch Patch 001: * Add checkstyle suppression filter XML * Suppress JavadocPackage check for test source files > Suppress checkstyle JavadocPackage error for test source files > -- > > Key: HADOOP-13160 > URL: https://issues.apache.org/jira/browse/HADOOP-13160 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 2.7.2 >Reporter: John Zhuge >Assignee: John Zhuge >Priority: Minor > Attachments: HADOOP-13160.001.patch > > > Suppress "Missing package-info.java" checkstyle error for test source files. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13159) Fix potential NPE in Metrics2 source for DecayRpcScheduler
[
https://issues.apache.org/jira/browse/HADOOP-13159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285323#comment-15285323
]
Hadoop QA commented on HADOOP-13159:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
4s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 8s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 52s
{color} | {color:green} trunk passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 48s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
20s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 4s
{color} | {color:green} trunk passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 2s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 2s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 55s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 55s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 58s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 6s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 6s {color} |
{color:red} hadoop-common in the patch failed with JDK v1.8.0_91. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 7s {color} |
{color:red} hadoop-common in the patch failed with JDK v1.7.0_101. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 65m 27s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_91 Failed junit tests | hadoop.ha.TestZKFailoverController |
| JDK v1.7.0_101 Failed junit tests | hadoop.net.TestDNS |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:cf2ee45 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804246/HADOOP-13159.01.patch
|
| JIRA Issue | HADOOP-13159 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 0b5f34288ed6 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
[jira] [Commented] (HADOOP-11505) Various native parts use bswap incorrectly and unportably
[ https://issues.apache.org/jira/browse/HADOOP-11505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285318#comment-15285318 ] Amir Sanjar commented on HADOOP-11505: -- hi Colin, fixing bugs and specially performance regression detection are our primary objective. I have a dedicated team of four getting trained for this tasks. > Various native parts use bswap incorrectly and unportably > - > > Key: HADOOP-11505 > URL: https://issues.apache.org/jira/browse/HADOOP-11505 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.0.0-alpha1 >Reporter: Colin Patrick McCabe >Assignee: Alan Burlison >Priority: Critical > Attachments: HADOOP-11505.001.patch, HADOOP-11505.003.patch, > HADOOP-11505.004.patch, HADOOP-11505.005.patch, HADOOP-11505.006.patch, > HADOOP-11505.007.patch, HADOOP-11505.008.patch > > > hadoop-mapreduce-client-nativetask fails to use x86 optimizations in some > cases. Also, on some alternate, non-x86, non-ARM architectures the generated > code is incorrect. Thanks to Steve Loughran and Edward Nevill for finding > this. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work started] (HADOOP-13160) Suppress checkstyle JavadocPackage error for test source files
[ https://issues.apache.org/jira/browse/HADOOP-13160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HADOOP-13160 started by John Zhuge. --- > Suppress checkstyle JavadocPackage error for test source files > -- > > Key: HADOOP-13160 > URL: https://issues.apache.org/jira/browse/HADOOP-13160 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 2.7.2 >Reporter: John Zhuge >Assignee: John Zhuge >Priority: Minor > > Suppress "Missing package-info.java" checkstyle error for test source files. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13160) Suppress checkstyle JavadocPackage error for test source files
John Zhuge created HADOOP-13160: --- Summary: Suppress checkstyle JavadocPackage error for test source files Key: HADOOP-13160 URL: https://issues.apache.org/jira/browse/HADOOP-13160 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.7.2 Reporter: John Zhuge Assignee: John Zhuge Priority: Minor Suppress "Missing package-info.java" checkstyle error for test source files. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13157) Follow-on improvements to HADOOP-12942
[
https://issues.apache.org/jira/browse/HADOOP-13157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285289#comment-15285289
]
Mike Yoder commented on HADOOP-13157:
-
Two questions:
# "locatePassword ... looks very similar to the one in JavaKeyStoreProvider ...
potential for code reuse?" Sure, and this crossed my mind, too. But where
would such a function live?
# "This link is not tied to a version..." Is there a canonical way of referring
to links that we can use?
> Follow-on improvements to HADOOP-12942
> --
>
> Key: HADOOP-13157
> URL: https://issues.apache.org/jira/browse/HADOOP-13157
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Affects Versions: 2.8.0
>Reporter: Mike Yoder
>Assignee: Mike Yoder
>
> [~andrew.wang] had some follow-up code review comments from HADOOP-12942.
> Hence this issue.
> Ping [~lmccay] as well.
> The comments:
> {quote}
> Overall this looks okay, the only correctness question I have is about the
> difference in behavior when the pwfile doesn't exist.
> The rest are all nits, would be nice to do these cleanups though.
> File
> hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/JavaKeyStoreProvider.java:
> Line 147:
> Could this be a static helper?
> Line 161: new
> The javadoc says it returns null in this situation. This is also a difference
> from the implementation in the AbstractJKSP. Intentional?
> Line 175: private void locateKeystore() throws IOException {
> static helper? for the construct*Path methods too?
> File
> hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/KeyShell.java:
> Line 50: @VisibleForTesting public static final String NO_VALID_PROVIDERS =
> FYI for the future, our coding style is to put annotations on their own
> separate line.
> File
> hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/alias/AbstractJavaKeyStoreProvider.java:
> Line 326: private char[] locatePassword() throws IOException {
> this method looks very similar to the one in JavaKeyStoreProvider, except the
> env var it looks for is different, is there potential for code reuse?
> Line 394: "o In the environment variable " +
> Using a "*" is the usual way of doing a bullet point, e.g. markdown and wiki
> syntax.
> Line 399:
> "http://hadoop.apache.org/docs/current/hadoop-project-dist/; +
> This link is not tied to a version, so could be inaccurate.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13159) Fix potential NPE in Metrics2 source for DecayRpcScheduler
[
https://issues.apache.org/jira/browse/HADOOP-13159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285282#comment-15285282
]
Hadoop QA commented on HADOOP-13159:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
10s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 15s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 54s
{color} | {color:green} trunk passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 1s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
40s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 5s
{color} | {color:green} trunk passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 4s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 4s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 55s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 55s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 8s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m 36s
{color} | {color:green} hadoop-common in the patch passed with JDK v1.8.0_91.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 22s {color}
| {color:red} hadoop-common in the patch failed with JDK v1.7.0_101. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 61m 41s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.7.0_101 Failed junit tests | hadoop.net.TestDNS |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:cf2ee45 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804240/HADOOP-13159.00.patch
|
| JIRA Issue | HADOOP-13159 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 0df0bca10315 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build
[jira] [Commented] (HADOOP-13065) Add a new interface for retrieving FS and FC Statistics
[ https://issues.apache.org/jira/browse/HADOOP-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285257#comment-15285257 ] Colin Patrick McCabe commented on HADOOP-13065: --- bq. I filed HADOOP-13140 to track the effort. Thanks, [~liuml07]. bq. [~mingma] wrote: BTW, is the network distance metrics something general for all file system or it is more specific to HDFS? For example, local file system doesn't need that. If it is more HDFS specific, wonder if we should move it to HDFS specific metrics. I agree that it's more conceptually consistent to put the distance-related metrics in HDFS-specific code. However, we would have to develop an optimized thread-local mechanism to do this, to avoid causing a performance regression in HDFS stream performance. Perhaps it would be better to simply move this to HDFS's existing per-stream ReadStatistics for now. > Add a new interface for retrieving FS and FC Statistics > --- > > Key: HADOOP-13065 > URL: https://issues.apache.org/jira/browse/HADOOP-13065 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Reporter: Ram Venkatesh >Assignee: Mingliang Liu > Fix For: 2.8.0 > > Attachments: HADOOP-13065-007.patch, HADOOP-13065.008.patch, > HADOOP-13065.009.patch, HADOOP-13065.010.patch, HADOOP-13065.011.patch, > HADOOP-13065.012.patch, HADOOP-13065.013.patch, HDFS-10175.000.patch, > HDFS-10175.001.patch, HDFS-10175.002.patch, HDFS-10175.003.patch, > HDFS-10175.004.patch, HDFS-10175.005.patch, HDFS-10175.006.patch, > TestStatisticsOverhead.java > > > Currently FileSystem.Statistics exposes the following statistics: > BytesRead > BytesWritten > ReadOps > LargeReadOps > WriteOps > These are in-turn exposed as job counters by MapReduce and other frameworks. > There is logic within DfsClient to map operations to these counters that can > be confusing, for instance, mkdirs counts as a writeOp. > Proposed enhancement: > Add a statistic for each DfsClient operation including create, append, > createSymlink, delete, exists, mkdirs, rename and expose them as new > properties on the Statistics object. The operation-specific counters can be > used for analyzing the load imposed by a particular job on HDFS. > For example, we can use them to identify jobs that end up creating a large > number of files. > Once this information is available in the Statistics object, the app > frameworks like MapReduce can expose them as additional counters to be > aggregated and recorded as part of job summary. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13140) GlobalStorageStatistics should check null FileSystem scheme to avoid NPE
[
https://issues.apache.org/jira/browse/HADOOP-13140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285252#comment-15285252
]
Colin Patrick McCabe commented on HADOOP-13140:
---
Thanks for looking at this. I don't think it makes sense to use a null, empty
string, or otherwise invalid string to identify a statistics object. It has no
meaning to the user. I think if the user passes a URL with a null schema, we
should call {{FileSystem#getDefaultUri}} and use the default schema, similar to
how {{FileSystem#get}} functions.
> GlobalStorageStatistics should check null FileSystem scheme to avoid NPE
>
>
> Key: HADOOP-13140
> URL: https://issues.apache.org/jira/browse/HADOOP-13140
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 2.8.0
>Reporter: Brahma Reddy Battula
>Assignee: Mingliang Liu
> Attachments: HADOOP-13140.000.patch
>
>
> {{org.apache.hadoop.fs.GlobalStorageStatistics#put}} is not checking the null
> scheme, and the internal map will complain NPE. This was reported by a flaky
> test {{TestFileSystemApplicationHistoryStore}}. Thanks [~brahmareddy] for
> reporting.
> To address this,
> # Fix the test by providing a valid URI, e.g. {{file:///}}
> # Guard the null scheme in {{GlobalStorageStatistics#put}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11505) Various native parts use bswap incorrectly and unportably
[ https://issues.apache.org/jira/browse/HADOOP-11505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285233#comment-15285233 ] Colin Patrick McCabe commented on HADOOP-11505: --- I think adding a PPC nightly build would be a step in the right direction. Of course, people interested in making Hadoop work well on PPC would still have to fix occasional breakages and performance regression. Apache is a do-ocracy so if people want to put in the work to do it, it will get done. > Various native parts use bswap incorrectly and unportably > - > > Key: HADOOP-11505 > URL: https://issues.apache.org/jira/browse/HADOOP-11505 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.0.0-alpha1 >Reporter: Colin Patrick McCabe >Assignee: Alan Burlison >Priority: Critical > Attachments: HADOOP-11505.001.patch, HADOOP-11505.003.patch, > HADOOP-11505.004.patch, HADOOP-11505.005.patch, HADOOP-11505.006.patch, > HADOOP-11505.007.patch, HADOOP-11505.008.patch > > > hadoop-mapreduce-client-nativetask fails to use x86 optimizations in some > cases. Also, on some alternate, non-x86, non-ARM architectures the generated > code is incorrect. Thanks to Steve Loughran and Edward Nevill for finding > this. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13146) Refactor RetryInvocationHandler
[ https://issues.apache.org/jira/browse/HADOOP-13146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HADOOP-13146: - Attachment: c13146_20160516.patch Thanks Jing for the review. Updated patch to address your comments. c13146_20160516.patch > Refactor RetryInvocationHandler > --- > > Key: HADOOP-13146 > URL: https://issues.apache.org/jira/browse/HADOOP-13146 > Project: Hadoop Common > Issue Type: Improvement > Components: io >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze >Priority: Minor > Attachments: c13146_20160513.patch, c13146_20160513b.patch, > c13146_20160514.patch, c13146_20160514b.patch, c13146_20160516.patch > > > - The exception handling is quite long. It is better to refactor it to a > separated method. > - The failover logic and synchronization can be moved to a new inner class. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13157) Follow-on improvements to HADOOP-12942
[
https://issues.apache.org/jira/browse/HADOOP-13157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wang updated HADOOP-13157:
-
Affects Version/s: 2.8.0
Target Version/s: 2.8.0
No worries, just some things I saw during internal review.
> Follow-on improvements to HADOOP-12942
> --
>
> Key: HADOOP-13157
> URL: https://issues.apache.org/jira/browse/HADOOP-13157
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Affects Versions: 2.8.0
>Reporter: Mike Yoder
>Assignee: Mike Yoder
>
> [~andrew.wang] had some follow-up code review comments from HADOOP-12942.
> Hence this issue.
> Ping [~lmccay] as well.
> The comments:
> {quote}
> Overall this looks okay, the only correctness question I have is about the
> difference in behavior when the pwfile doesn't exist.
> The rest are all nits, would be nice to do these cleanups though.
> File
> hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/JavaKeyStoreProvider.java:
> Line 147:
> Could this be a static helper?
> Line 161: new
> The javadoc says it returns null in this situation. This is also a difference
> from the implementation in the AbstractJKSP. Intentional?
> Line 175: private void locateKeystore() throws IOException {
> static helper? for the construct*Path methods too?
> File
> hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/KeyShell.java:
> Line 50: @VisibleForTesting public static final String NO_VALID_PROVIDERS =
> FYI for the future, our coding style is to put annotations on their own
> separate line.
> File
> hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/alias/AbstractJavaKeyStoreProvider.java:
> Line 326: private char[] locatePassword() throws IOException {
> this method looks very similar to the one in JavaKeyStoreProvider, except the
> env var it looks for is different, is there potential for code reuse?
> Line 394: "o In the environment variable " +
> Using a "*" is the usual way of doing a bullet point, e.g. markdown and wiki
> syntax.
> Line 399:
> "http://hadoop.apache.org/docs/current/hadoop-project-dist/; +
> This link is not tied to a version, so could be inaccurate.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13159) Fix potential NPE in Metrics2 source for DecayRpcScheduler
[ https://issues.apache.org/jira/browse/HADOOP-13159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HADOOP-13159: Attachment: HADOOP-13159.01.patch Fix a caption output of topN user. > Fix potential NPE in Metrics2 source for DecayRpcScheduler > -- > > Key: HADOOP-13159 > URL: https://issues.apache.org/jira/browse/HADOOP-13159 > Project: Hadoop Common > Issue Type: Bug > Components: ipc >Affects Versions: 2.8.0 >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Attachments: HADOOP-13159.00.patch, HADOOP-13159.01.patch > > > HADOOP-12985 introduced a few metrics2 counters for DecayRpcScheduler. The > counters should be initialized before the updater thread is created/executed. > Otherwise, this is a chance of NPE if the updater thread get executed before > the counters are fully initialized. I will post a patch for it shortly. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13130) s3a failures can surface as RTEs, not IOEs
[
https://issues.apache.org/jira/browse/HADOOP-13130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285182#comment-15285182
]
Hadoop QA commented on HADOOP-13130:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 49s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
11s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 41s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 44s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
26s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 20s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
21s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
32s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 6s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 22s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 14s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
57s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 43s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 43s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 40s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 40s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
27s {color} | {color:green} root: The patch generated 0 new + 24 unchanged - 5
fixed = 24 total (was 29) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 17s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 23s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 57s {color}
| {color:red} hadoop-common in the patch failed with JDK v1.8.0_91. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 15s
{color} | {color:green} hadoop-aws in the patch passed with JDK v1.8.0_91.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 44s {color}
| {color:red} hadoop-common in the patch failed with JDK v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 15s
{color} | {color:green} hadoop-aws in the patch passed with JDK v1.7.0_95.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 70m 35s {color}
| {color:black} {color} |
\\
\\
||
[jira] [Updated] (HADOOP-13159) Fix potential NPE in Metrics2 source for DecayRpcScheduler
[ https://issues.apache.org/jira/browse/HADOOP-13159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HADOOP-13159: Status: Patch Available (was: Open) > Fix potential NPE in Metrics2 source for DecayRpcScheduler > -- > > Key: HADOOP-13159 > URL: https://issues.apache.org/jira/browse/HADOOP-13159 > Project: Hadoop Common > Issue Type: Bug > Components: ipc >Affects Versions: 2.8.0 >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Attachments: HADOOP-13159.00.patch > > > HADOOP-12985 introduced a few metrics2 counters for DecayRpcScheduler. The > counters should be initialized before the updater thread is created/executed. > Otherwise, this is a chance of NPE if the updater thread get executed before > the counters are fully initialized. I will post a patch for it shortly. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13159) Fix potential NPE in Metrics2 source for DecayRpcScheduler
[ https://issues.apache.org/jira/browse/HADOOP-13159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HADOOP-13159: Attachment: HADOOP-13159.00.patch Attach a fix by changing the order of initialization. > Fix potential NPE in Metrics2 source for DecayRpcScheduler > -- > > Key: HADOOP-13159 > URL: https://issues.apache.org/jira/browse/HADOOP-13159 > Project: Hadoop Common > Issue Type: Bug > Components: ipc >Affects Versions: 2.8.0 >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Attachments: HADOOP-13159.00.patch > > > HADOOP-12985 introduced a few metrics2 counters for DecayRpcScheduler. The > counters should be initialized before the updater thread is created/executed. > Otherwise, this is a chance of NPE if the updater thread get executed before > the counters are fully initialized. I will post a patch for it shortly. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13159) Fix potential NPE in Metrics2 source for RPC scheduler
Xiaoyu Yao created HADOOP-13159: --- Summary: Fix potential NPE in Metrics2 source for RPC scheduler Key: HADOOP-13159 URL: https://issues.apache.org/jira/browse/HADOOP-13159 Project: Hadoop Common Issue Type: Bug Components: ipc Affects Versions: 2.8.0 Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao HADOOP-12985 introduced a few metrics2 counters for DecayRpcScheduler. The counters should be initialized before the updater thread is created/executed. Otherwise, this is a chance of NPE if the updater thread get executed before the counters are fully initialized. I will post a patch for it shortly. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13159) Fix potential NPE in Metrics2 source for DecayRpcScheduler
[ https://issues.apache.org/jira/browse/HADOOP-13159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HADOOP-13159: Summary: Fix potential NPE in Metrics2 source for DecayRpcScheduler (was: Fix potential NPE in Metrics2 source for RPC scheduler) > Fix potential NPE in Metrics2 source for DecayRpcScheduler > -- > > Key: HADOOP-13159 > URL: https://issues.apache.org/jira/browse/HADOOP-13159 > Project: Hadoop Common > Issue Type: Bug > Components: ipc >Affects Versions: 2.8.0 >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > > HADOOP-12985 introduced a few metrics2 counters for DecayRpcScheduler. The > counters should be initialized before the updater thread is created/executed. > Otherwise, this is a chance of NPE if the updater thread get executed before > the counters are fully initialized. I will post a patch for it shortly. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13148) TestDistCpViewFs to include IOExceptions in test error reports
[ https://issues.apache.org/jira/browse/HADOOP-13148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285118#comment-15285118 ] Hudson commented on HADOOP-13148: - FAILURE: Integrated in Hadoop-trunk-Commit #9767 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/9767/]) HADOOP-13148. TestDistCpViewFs to include IOExceptions in test error (cnauroth: rev b9685e85d59e69e5dd64802fa9025dad1b315be5) * hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCpViewFs.java > TestDistCpViewFs to include IOExceptions in test error reports > -- > > Key: HADOOP-13148 > URL: https://issues.apache.org/jira/browse/HADOOP-13148 > Project: Hadoop Common > Issue Type: Improvement > Components: tools/distcp >Affects Versions: 2.8.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Minor > Fix For: 2.8.0 > > Attachments: HADOOP-13148-001.patch > > Original Estimate: 0.25h > Remaining Estimate: 0.25h > > Minor test code style: all IOEs are caught and logged @ error, then test > failed with a simple assert. This will lose all failure data from the XML > test reports, hiding it in the logs. > Better to not catch the IOEs, just rethrow, and let JUnit handle them. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13149) Windows distro build fails on dist-copynativelibs.
[ https://issues.apache.org/jira/browse/HADOOP-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285117#comment-15285117 ] Hudson commented on HADOOP-13149: - FAILURE: Integrated in Hadoop-trunk-Commit #9767 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/9767/]) HADOOP-13149. Windows distro build fails on dist-copynativelibs. (cnauroth: rev 576e2d1278a584ef8e804832c852c3f8323bb64a) * hadoop-project-dist/pom.xml > Windows distro build fails on dist-copynativelibs. > -- > > Key: HADOOP-13149 > URL: https://issues.apache.org/jira/browse/HADOOP-13149 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: Chris Nauroth >Assignee: Chris Nauroth >Priority: Blocker > Fix For: 3.0.0-alpha1 > > Attachments: HADOOP-13149.001.patch > > > HADOOP-12892 pulled the dist-copynativelibs script into an external file. > The call to this script is failing when running a distro build on Windows. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13149) Windows distro build fails on dist-copynativelibs.
[ https://issues.apache.org/jira/browse/HADOOP-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HADOOP-13149: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha1 Target Version/s: 3.0.0-alpha1 (was: 2.8.0) Status: Resolved (was: Patch Available) Allen, thank you for the review. I committed this to trunk. > Windows distro build fails on dist-copynativelibs. > -- > > Key: HADOOP-13149 > URL: https://issues.apache.org/jira/browse/HADOOP-13149 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: Chris Nauroth >Assignee: Chris Nauroth >Priority: Blocker > Fix For: 3.0.0-alpha1 > > Attachments: HADOOP-13149.001.patch > > > HADOOP-12892 pulled the dist-copynativelibs script into an external file. > The call to this script is failing when running a distro build on Windows. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13148) TestDistCpViewFs to include IOExceptions in test error reports
[ https://issues.apache.org/jira/browse/HADOOP-13148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HADOOP-13148: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved (was: Patch Available) +1 for the patch. I committed this to trunk, branch-2 and branch-2.8. The Checkstyle warning was an unused import, and I removed it before committing. > TestDistCpViewFs to include IOExceptions in test error reports > -- > > Key: HADOOP-13148 > URL: https://issues.apache.org/jira/browse/HADOOP-13148 > Project: Hadoop Common > Issue Type: Improvement > Components: tools/distcp >Affects Versions: 2.8.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Minor > Fix For: 2.8.0 > > Attachments: HADOOP-13148-001.patch > > Original Estimate: 0.25h > Remaining Estimate: 0.25h > > Minor test code style: all IOEs are caught and logged @ error, then test > failed with a simple assert. This will lose all failure data from the XML > test reports, hiding it in the logs. > Better to not catch the IOEs, just rethrow, and let JUnit handle them. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13158) S3AFileSystem#toString might throw NullPointerException due to null cannedACL.
[
https://issues.apache.org/jira/browse/HADOOP-13158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285071#comment-15285071
]
Hadoop QA commented on HADOOP-13158:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
52s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 13s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 15s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 21s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 13s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 11s
{color} | {color:green} hadoop-aws in the patch passed with JDK v1.8.0_91.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 12s
{color} | {color:green} hadoop-aws in the patch passed with JDK v1.7.0_95.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 50s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:cf2ee45 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804227/HADOOP-13158.001.patch
|
| JIRA Issue | HADOOP-13158 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 5b385eae3f59 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HADOOP-13146) Refactor RetryInvocationHandler
[
https://issues.apache.org/jira/browse/HADOOP-13146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285063#comment-15285063
]
Jing Zhao commented on HADOOP-13146:
Thanks for the refactoring, Nicholas! The patch looks pretty good to me. Just
two minor comments:
# We can use {{Collections.singletonList}} instead of {{Arrays.asList}} in the
following code.
{code}
117 final Iterable exceptions = e instanceof
MultiException ?
118 ((MultiException) e).getExceptions().values() :
Arrays.asList(e);
{code}
# The follow java comment can be moved into the {{failover}} method.
{code}
// Make sure that concurrent failed method invocations only cause a
// single actual fail over.
{code}
+1 after addressing the comments.
> Refactor RetryInvocationHandler
> ---
>
> Key: HADOOP-13146
> URL: https://issues.apache.org/jira/browse/HADOOP-13146
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
>Reporter: Tsz Wo Nicholas Sze
>Assignee: Tsz Wo Nicholas Sze
>Priority: Minor
> Attachments: c13146_20160513.patch, c13146_20160513b.patch,
> c13146_20160514.patch, c13146_20160514b.patch
>
>
> - The exception handling is quite long. It is better to refactor it to a
> separated method.
> - The failover logic and synchronization can be moved to a new inner class.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13146) Refactor RetryInvocationHandler
[ https://issues.apache.org/jira/browse/HADOOP-13146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HADOOP-13146: --- Hadoop Flags: Reviewed > Refactor RetryInvocationHandler > --- > > Key: HADOOP-13146 > URL: https://issues.apache.org/jira/browse/HADOOP-13146 > Project: Hadoop Common > Issue Type: Improvement > Components: io >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze >Priority: Minor > Attachments: c13146_20160513.patch, c13146_20160513b.patch, > c13146_20160514.patch, c13146_20160514b.patch > > > - The exception handling is quite long. It is better to refactor it to a > separated method. > - The failover logic and synchronization can be moved to a new inner class. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13113) Enable parallel test execution for hadoop-aws.
[
https://issues.apache.org/jira/browse/HADOOP-13113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285053#comment-15285053
]
Chris Nauroth commented on HADOOP-13113:
bq. I should warn that I'm seeing errors here...
[~ste...@apache.org], here are my observations based on additional experiments.
# If I configure a dedicated bucket per file system (S3, S3N and S3A), then I
consistently get a full pass on parallel test runs. This was the setup I was
using when I developed the patch.
# All of my buckets are in AWS region US-west-2 (Oregon), which is very close
to my laptop physically.
# If I configure the same bucket to be used by the test suites for all 3 file
systems, then sometimes I see some flaky behavior. I don't know if this is due
to data problems, such as perhaps S3N tests leaving bits behind that confuse
S3A later, or if the increased activity on a shared bucket simply surfaces
eventual consistency problems with greater likelihood.
# One test run surfaced a bug, and I have submitted a patch for that on
HADOOP-13158.
# {{TestS3NContractRootDir#testListEmptyRootDirectory}} probably needs the same
retry loop that we put in place for the same test in {{TestS3AContractRootDir}}.
I'll continue to look for ways to stabilize this.
> Enable parallel test execution for hadoop-aws.
> --
>
> Key: HADOOP-13113
> URL: https://issues.apache.org/jira/browse/HADOOP-13113
> Project: Hadoop Common
> Issue Type: Improvement
> Components: test
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Fix For: 2.8.0
>
> Attachments: HADOOP-13113.001.patch, HADOOP-13113.002.patch,
> HADOOP-13113.003.patch, HADOOP-13113.004.patch
>
>
> The full hadoop-aws test suite takes ~30 minutes to execute. The tests spend
> most of their time blocked on network I/O with the S3 back-end, but they
> don't saturate the bandwidth of the NIC. We can improve overall execution
> time by enabling parallel test execution.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13157) Follow-on improvements to HADOOP-12942
[
https://issues.apache.org/jira/browse/HADOOP-13157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285039#comment-15285039
]
Larry McCay commented on HADOOP-13157:
--
[~andrew.wang] - thanks for the additional review! I should have pulled you in
earlier.
> Follow-on improvements to HADOOP-12942
> --
>
> Key: HADOOP-13157
> URL: https://issues.apache.org/jira/browse/HADOOP-13157
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
>
> [~andrew.wang] had some follow-up code review comments from HADOOP-12942.
> Hence this issue.
> Ping [~lmccay] as well.
> The comments:
> {quote}
> Overall this looks okay, the only correctness question I have is about the
> difference in behavior when the pwfile doesn't exist.
> The rest are all nits, would be nice to do these cleanups though.
> File
> hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/JavaKeyStoreProvider.java:
> Line 147:
> Could this be a static helper?
> Line 161: new
> The javadoc says it returns null in this situation. This is also a difference
> from the implementation in the AbstractJKSP. Intentional?
> Line 175: private void locateKeystore() throws IOException {
> static helper? for the construct*Path methods too?
> File
> hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/KeyShell.java:
> Line 50: @VisibleForTesting public static final String NO_VALID_PROVIDERS =
> FYI for the future, our coding style is to put annotations on their own
> separate line.
> File
> hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/alias/AbstractJavaKeyStoreProvider.java:
> Line 326: private char[] locatePassword() throws IOException {
> this method looks very similar to the one in JavaKeyStoreProvider, except the
> env var it looks for is different, is there potential for code reuse?
> Line 394: "o In the environment variable " +
> Using a "*" is the usual way of doing a bullet point, e.g. markdown and wiki
> syntax.
> Line 399:
> "http://hadoop.apache.org/docs/current/hadoop-project-dist/; +
> This link is not tied to a version, so could be inaccurate.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13028) add low level counter metrics for S3A; use in read performance tests
[ https://issues.apache.org/jira/browse/HADOOP-13028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285025#comment-15285025 ] Chris Nauroth commented on HADOOP-13028: HADOOP-13158 has a small follow-up bug fix. > add low level counter metrics for S3A; use in read performance tests > > > Key: HADOOP-13028 > URL: https://issues.apache.org/jira/browse/HADOOP-13028 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3, metrics >Affects Versions: 2.8.0 >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: 2.8.0 > > Attachments: HADOOP-13028-001.patch, HADOOP-13028-002.patch, > HADOOP-13028-004.patch, HADOOP-13028-005.patch, HADOOP-13028-006.patch, > HADOOP-13028-007.patch, HADOOP-13028-008.patch, HADOOP-13028-009.patch, > HADOOP-13028-012.patch, HADOOP-13028-013.patch, > HADOOP-13028-branch-2-008.patch, HADOOP-13028-branch-2-009.patch, > HADOOP-13028-branch-2-010.patch, HADOOP-13028-branch-2-011.patch, > HADOOP-13028-branch-2-012.patch, HADOOP-13028-branch-2-013.patch, > org.apache.hadoop.fs.s3a.scale.TestS3AInputStreamPerformance-output.txt, > org.apache.hadoop.fs.s3a.scale.TestS3AInputStreamPerformance-output.txt > > > against S3 (and other object stores), opening connections can be expensive, > closing connections may be expensive (a sign of a regression). > S3A FS and individual input streams should have counters of the # of > open/close/failure+reconnect operations, timers of how long things take. This > can be used downstream to measure efficiency of the code (how often > connections are being made), connection reliability, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13065) Add a new interface for retrieving FS and FC Statistics
[
https://issues.apache.org/jira/browse/HADOOP-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285026#comment-15285026
]
Mingliang Liu commented on HADOOP-13065:
Thanks [~mingma] for the kind remind.
IMHO the topology-aware read stats is HDFS only. You worked on the related
jiras and would give us more convincing opinions. If it's true, we can create a
new {{StorageStatistics}}, say {{RackAwareReadStorageStatistics}}, for the
stats in [HDFS-9579]. We can track this effort in [HADOOP-13031] after I modify
the description soon. For the implementation, I'm not quite sure whether we can
simply use the AtomicLongs instead of thread-local optimizations.
> Add a new interface for retrieving FS and FC Statistics
> ---
>
> Key: HADOOP-13065
> URL: https://issues.apache.org/jira/browse/HADOOP-13065
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
>Reporter: Ram Venkatesh
>Assignee: Mingliang Liu
> Fix For: 2.8.0
>
> Attachments: HADOOP-13065-007.patch, HADOOP-13065.008.patch,
> HADOOP-13065.009.patch, HADOOP-13065.010.patch, HADOOP-13065.011.patch,
> HADOOP-13065.012.patch, HADOOP-13065.013.patch, HDFS-10175.000.patch,
> HDFS-10175.001.patch, HDFS-10175.002.patch, HDFS-10175.003.patch,
> HDFS-10175.004.patch, HDFS-10175.005.patch, HDFS-10175.006.patch,
> TestStatisticsOverhead.java
>
>
> Currently FileSystem.Statistics exposes the following statistics:
> BytesRead
> BytesWritten
> ReadOps
> LargeReadOps
> WriteOps
> These are in-turn exposed as job counters by MapReduce and other frameworks.
> There is logic within DfsClient to map operations to these counters that can
> be confusing, for instance, mkdirs counts as a writeOp.
> Proposed enhancement:
> Add a statistic for each DfsClient operation including create, append,
> createSymlink, delete, exists, mkdirs, rename and expose them as new
> properties on the Statistics object. The operation-specific counters can be
> used for analyzing the load imposed by a particular job on HDFS.
> For example, we can use them to identify jobs that end up creating a large
> number of files.
> Once this information is available in the Statistics object, the app
> frameworks like MapReduce can expose them as additional counters to be
> aggregated and recorded as part of job summary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13158) S3AFileSystem#toString might throw NullPointerException due to null cannedACL.
[
https://issues.apache.org/jira/browse/HADOOP-13158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HADOOP-13158:
---
Attachment: HADOOP-13158.001.patch
I missed this in my code review of HADOOP-13028. I saw this happen in a test
run recently. Here is a patch to fix it. I reviewed the whole {{toString}}
method for similar bugs, and {{cannedACL}} is the only problem.
[~ste...@apache.org], would you please review?
> S3AFileSystem#toString might throw NullPointerException due to null cannedACL.
> --
>
> Key: HADOOP-13158
> URL: https://issues.apache.org/jira/browse/HADOOP-13158
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Attachments: HADOOP-13158.001.patch
>
>
> The {{cannedACL}} field of {{S3AFileSystem}} can be {{null}}. The
> {{toString}} implementation has an unguarded call to
> {{cannedACL.toString()}}, so there is a risk of {{NullPointerException}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13158) S3AFileSystem#toString might throw NullPointerException due to null cannedACL.
[
https://issues.apache.org/jira/browse/HADOOP-13158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HADOOP-13158:
---
Status: Patch Available (was: Open)
> S3AFileSystem#toString might throw NullPointerException due to null cannedACL.
> --
>
> Key: HADOOP-13158
> URL: https://issues.apache.org/jira/browse/HADOOP-13158
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Attachments: HADOOP-13158.001.patch
>
>
> The {{cannedACL}} field of {{S3AFileSystem}} can be {{null}}. The
> {{toString}} implementation has an unguarded call to
> {{cannedACL.toString()}}, so there is a risk of {{NullPointerException}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13158) S3AFileSystem#toString might throw NullPointerException due to null cannedACL.
Chris Nauroth created HADOOP-13158:
--
Summary: S3AFileSystem#toString might throw NullPointerException
due to null cannedACL.
Key: HADOOP-13158
URL: https://issues.apache.org/jira/browse/HADOOP-13158
Project: Hadoop Common
Issue Type: Bug
Components: fs/s3
Reporter: Chris Nauroth
Assignee: Chris Nauroth
The {{cannedACL}} field of {{S3AFileSystem}} can be {{null}}. The {{toString}}
implementation has an unguarded call to {{cannedACL.toString()}}, so there is a
risk of {{NullPointerException}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
