[jira] [Commented] (HADOOP-15379) Make IrqHandler.bind() public
[
https://issues.apache.org/jira/browse/HADOOP-15379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436842#comment-16436842
]
Hudson commented on HADOOP-15379:
-
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13988 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/13988/])
HADOOP-15379. Make IrqHandler.bind() public. Contributed by Ajay Kumar (Bharat:
rev ec1e8c1a8ca3d01b82ca82d73ba1132b6625d659)
* (edit)
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/service/launcher/IrqHandler.java
> Make IrqHandler.bind() public
> -
>
> Key: HADOOP-15379
> URL: https://issues.apache.org/jira/browse/HADOOP-15379
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 3.1.0
>Reporter: Steve Loughran
>Assignee: Ajay Kumar
>Priority: Minor
> Fix For: 3.2.0, 3.1.1
>
> Attachments: HADOOP-15379.00.patch
>
>
> {{org.apache.hadoop.service.launcher.IrqHandler.bind()}} is package private
> this means you can create an {{Interrupted}} handler in a different package,
> but you can't bind it to a signal.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15379) Make IrqHandler.bind() public
[
https://issues.apache.org/jira/browse/HADOOP-15379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436807#comment-16436807
]
Bharat Viswanadham commented on HADOOP-15379:
-
Committed this to trunk and branch-3.1.
Thank You [~ajayydv] for working on this and [~ste...@apache.org] for reporting
this.
> Make IrqHandler.bind() public
> -
>
> Key: HADOOP-15379
> URL: https://issues.apache.org/jira/browse/HADOOP-15379
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 3.1.0
>Reporter: Steve Loughran
>Assignee: Ajay Kumar
>Priority: Minor
> Fix For: 3.2.0, 3.1.1
>
> Attachments: HADOOP-15379.00.patch
>
>
> {{org.apache.hadoop.service.launcher.IrqHandler.bind()}} is package private
> this means you can create an {{Interrupted}} handler in a different package,
> but you can't bind it to a signal.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15379) Make IrqHandler.bind() public
[
https://issues.apache.org/jira/browse/HADOOP-15379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bharat Viswanadham updated HADOOP-15379:
Fix Version/s: 3.1.1
3.2.0
> Make IrqHandler.bind() public
> -
>
> Key: HADOOP-15379
> URL: https://issues.apache.org/jira/browse/HADOOP-15379
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 3.1.0
>Reporter: Steve Loughran
>Assignee: Ajay Kumar
>Priority: Minor
> Fix For: 3.2.0, 3.1.1
>
> Attachments: HADOOP-15379.00.patch
>
>
> {{org.apache.hadoop.service.launcher.IrqHandler.bind()}} is package private
> this means you can create an {{Interrupted}} handler in a different package,
> but you can't bind it to a signal.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15379) Make IrqHandler.bind() public
[
https://issues.apache.org/jira/browse/HADOOP-15379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bharat Viswanadham updated HADOOP-15379:
Resolution: Fixed
Target Version/s: 3.2.0, 3.1.1 (was: 3.2.0)
Status: Resolved (was: Patch Available)
> Make IrqHandler.bind() public
> -
>
> Key: HADOOP-15379
> URL: https://issues.apache.org/jira/browse/HADOOP-15379
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 3.1.0
>Reporter: Steve Loughran
>Assignee: Ajay Kumar
>Priority: Minor
> Attachments: HADOOP-15379.00.patch
>
>
> {{org.apache.hadoop.service.launcher.IrqHandler.bind()}} is package private
> this means you can create an {{Interrupted}} handler in a different package,
> but you can't bind it to a signal.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15379) Make IrqHandler.bind() public
[
https://issues.apache.org/jira/browse/HADOOP-15379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436800#comment-16436800
]
Bharat Viswanadham commented on HADOOP-15379:
-
I will commit this shortly.
> Make IrqHandler.bind() public
> -
>
> Key: HADOOP-15379
> URL: https://issues.apache.org/jira/browse/HADOOP-15379
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 3.1.0
>Reporter: Steve Loughran
>Assignee: Ajay Kumar
>Priority: Minor
> Attachments: HADOOP-15379.00.patch
>
>
> {{org.apache.hadoop.service.launcher.IrqHandler.bind()}} is package private
> this means you can create an {{Interrupted}} handler in a different package,
> but you can't bind it to a signal.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15304) [JDK10] Migrate from com.sun.tools.doclets to the replacement
[
https://issues.apache.org/jira/browse/HADOOP-15304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436779#comment-16436779
]
Akira Ajisaka commented on HADOOP-15304:
Umm. I ran
{noformat}
$ git apply
$ cd hadoop-common-project/hadoop-annotations
$ mvn install -DskipTests
$ mvn javadoc:javadoc
{noformat}
and it succeeded. No additional params.
> [JDK10] Migrate from com.sun.tools.doclets to the replacement
> -
>
> Key: HADOOP-15304
> URL: https://issues.apache.org/jira/browse/HADOOP-15304
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Attachments: HADOOP-15304.01.patch, HADOOP-15304.02.patch
>
>
> com.sun.tools.doclets.* packages were removed in Java 10.
> [https://bugs.openjdk.java.net/browse/JDK-8177511]
> This causes hadoop-annotations module to fail.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile)
> on project hadoop-annotations: Compilation failure: Compilation failure:
> [ERROR]
> /Users/ajisaka/git/hadoop/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/tools/IncludePublicAnnotationsStandardDoclet.java:[61,20]
> cannot find symbol
> [ERROR] symbol: method
> validOptions(java.lang.String[][],com.sun.javadoc.DocErrorReporter)
> [ERROR] location: class com.sun.tools.doclets.standard.Standard
> [ERROR]
> /Users/ajisaka/git/hadoop/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/tools/ExcludePrivateAnnotationsStandardDoclet.java:[56,20]
> cannot find symbol
> [ERROR] symbol: method
> validOptions(java.lang.String[][],com.sun.javadoc.DocErrorReporter)
> [ERROR] location: class com.sun.tools.doclets.standard.Standard
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-15304) [JDK10] Migrate from com.sun.tools.doclets to the replacement
[
https://issues.apache.org/jira/browse/HADOOP-15304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436294#comment-16436294
]
Ajay Kumar edited comment on HADOOP-15304 at 4/13/18 3:54 AM:
--
[~ajisakaa], [~tasanuma0829] I tried mvn javadoc:javadoc with patch v2 on
jdk10 but it failed with error about missing class
ExcludePrivateAnnotationsStandardDoclet, since it is excluded. Do i need some
additional parameter in mvn?
{code}javadoc: error - Cannot find doclet class
org.apache.hadoop.classification.tools.ExcludePrivateAnnotationsStandardDoclet{code}
was (Author: ajayydv):
[~ajisakaa], [~tasanuma0829] I tried mvn javadoc:javadoc with patch v2 on
jdk10 but it failed with error about missing class
ExcludePrivateAnnotationsStandardDoclet, since it is excluded. Do i need some
additional parameter in mvn?
{code}javadoc: error - Cannot find doclet class
org.apache.hadoop.classification.tools.ExcludePrivateAnnotationsStandardDoclet\{code}
> [JDK10] Migrate from com.sun.tools.doclets to the replacement
> -
>
> Key: HADOOP-15304
> URL: https://issues.apache.org/jira/browse/HADOOP-15304
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Attachments: HADOOP-15304.01.patch, HADOOP-15304.02.patch
>
>
> com.sun.tools.doclets.* packages were removed in Java 10.
> [https://bugs.openjdk.java.net/browse/JDK-8177511]
> This causes hadoop-annotations module to fail.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile)
> on project hadoop-annotations: Compilation failure: Compilation failure:
> [ERROR]
> /Users/ajisaka/git/hadoop/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/tools/IncludePublicAnnotationsStandardDoclet.java:[61,20]
> cannot find symbol
> [ERROR] symbol: method
> validOptions(java.lang.String[][],com.sun.javadoc.DocErrorReporter)
> [ERROR] location: class com.sun.tools.doclets.standard.Standard
> [ERROR]
> /Users/ajisaka/git/hadoop/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/tools/ExcludePrivateAnnotationsStandardDoclet.java:[56,20]
> cannot find symbol
> [ERROR] symbol: method
> validOptions(java.lang.String[][],com.sun.javadoc.DocErrorReporter)
> [ERROR] location: class com.sun.tools.doclets.standard.Standard
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15362) Review of Configuration.java
[
https://issues.apache.org/jira/browse/HADOOP-15362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436762#comment-16436762
]
Ajay Kumar commented on HADOOP-15362:
-
[~belugabehr] missed private identifier for class :). +1 once jenkins reported
checkstyle issues are fixed.
> Review of Configuration.java
>
>
> Key: HADOOP-15362
> URL: https://issues.apache.org/jira/browse/HADOOP-15362
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15362.1.patch, HADOOP-15362.2.patch,
> HADOOP-15362.3.patch, HADOOP-15362.4.patch, HADOOP-15362.5.patch
>
>
> * Various improvements
> * Fix a lot of checks style errors
> When I ran a recent debug log against a MR job, I was spammed from the
> following messages. I ask that we move them to 'trace' as there is already a
> debug level logging preceding them.
> {code:java}
> LOG.debug("Handling deprecation for all properties in config");
> foreach item {
> - LOG.debug("Handling deprecation for " + (String)item);
> + LOG.trace("Handling deprecation for {}", item);
> }{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14999) AliyunOSS: provide one asynchronous multi-part based uploading mechanism
[
https://issues.apache.org/jira/browse/HADOOP-14999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SammiChen updated HADOOP-14999:
---
Resolution: Fixed
Fix Version/s: 3.0.3
2.9.2
3.1.1
3.2.0
2.9.1
2.10.0
Status: Resolved (was: Patch Available)
> AliyunOSS: provide one asynchronous multi-part based uploading mechanism
>
>
> Key: HADOOP-14999
> URL: https://issues.apache.org/jira/browse/HADOOP-14999
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/oss
>Affects Versions: 3.0.0-beta1
>Reporter: Genmao Yu
>Assignee: Genmao Yu
>Priority: Major
> Fix For: 2.10.0, 2.9.1, 3.2.0, 3.1.1, 2.9.2, 3.0.3
>
> Attachments: HADOOP-14999-branch-2.001.patch,
> HADOOP-14999-branch-2.002.patch, HADOOP-14999.001.patch,
> HADOOP-14999.002.patch, HADOOP-14999.003.patch, HADOOP-14999.004.patch,
> HADOOP-14999.005.patch, HADOOP-14999.006.patch, HADOOP-14999.007.patch,
> HADOOP-14999.008.patch, HADOOP-14999.009.patch, HADOOP-14999.010.patch,
> HADOOP-14999.011.patch, asynchronous_file_uploading.pdf,
> diff-between-patch7-and-patch8.txt
>
>
> This mechanism is designed for uploading file in parallel and asynchronously:
> - improve the performance of uploading file to OSS server. Firstly, this
> mechanism splits result to multiple small blocks and upload them in parallel.
> Then, getting result and uploading blocks are asynchronous.
> - avoid buffering too large result into local disk. To cite an extreme
> example, there is a task which will output 100GB or even larger, we may need
> to output this 100GB to local disk and then upload it. Sometimes, it is
> inefficient and limited to disk space.
> This patch reuse {{SemaphoredDelegatingExecutor}} as executor service and
> depends on HADOOP-15039.
> Attached {{asynchronous_file_uploading.pdf}} illustrated the difference
> between previous {{AliyunOSSOutputStream}} and
> {{AliyunOSSBlockOutputStream}}, i.e. this asynchronous multi-part based
> uploading mechanism.
> 1. {{AliyunOSSOutputStream}}: we need to output the whole result to local
> disk before we can upload it to OSS. This will poses two problems：
> - if the output file is too large, it will run out of the local disk.
> - if the output file is too large, task will wait long time to upload result
> to OSS before finish, wasting much compute resource.
> 2. {{AliyunOSSBlockOutputStream}}: we cut the task output into small blocks,
> i.e. some small local file, and each block will be packaged into a uploading
> task. These tasks will be submitted into {{SemaphoredDelegatingExecutor}}.
> {{SemaphoredDelegatingExecutor}} will upload this blocks in parallel, this
> will improve performance greatly.
> 3. Each task will retry 3 times to upload block to Aliyun OSS. If one of
> those tasks failed, the whole file uploading will failed, and we will abort
> current uploading.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14999) AliyunOSS: provide one asynchronous multi-part based uploading mechanism
[
https://issues.apache.org/jira/browse/HADOOP-14999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436699#comment-16436699
]
SammiChen commented on HADOOP-14999:
Committed to trunk, branch-3.0, branch-3.1, branch-2, branch-2.9 & branch-2.9.1.
> AliyunOSS: provide one asynchronous multi-part based uploading mechanism
>
>
> Key: HADOOP-14999
> URL: https://issues.apache.org/jira/browse/HADOOP-14999
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/oss
>Affects Versions: 3.0.0-beta1
>Reporter: Genmao Yu
>Assignee: Genmao Yu
>Priority: Major
> Attachments: HADOOP-14999-branch-2.001.patch,
> HADOOP-14999-branch-2.002.patch, HADOOP-14999.001.patch,
> HADOOP-14999.002.patch, HADOOP-14999.003.patch, HADOOP-14999.004.patch,
> HADOOP-14999.005.patch, HADOOP-14999.006.patch, HADOOP-14999.007.patch,
> HADOOP-14999.008.patch, HADOOP-14999.009.patch, HADOOP-14999.010.patch,
> HADOOP-14999.011.patch, asynchronous_file_uploading.pdf,
> diff-between-patch7-and-patch8.txt
>
>
> This mechanism is designed for uploading file in parallel and asynchronously:
> - improve the performance of uploading file to OSS server. Firstly, this
> mechanism splits result to multiple small blocks and upload them in parallel.
> Then, getting result and uploading blocks are asynchronous.
> - avoid buffering too large result into local disk. To cite an extreme
> example, there is a task which will output 100GB or even larger, we may need
> to output this 100GB to local disk and then upload it. Sometimes, it is
> inefficient and limited to disk space.
> This patch reuse {{SemaphoredDelegatingExecutor}} as executor service and
> depends on HADOOP-15039.
> Attached {{asynchronous_file_uploading.pdf}} illustrated the difference
> between previous {{AliyunOSSOutputStream}} and
> {{AliyunOSSBlockOutputStream}}, i.e. this asynchronous multi-part based
> uploading mechanism.
> 1. {{AliyunOSSOutputStream}}: we need to output the whole result to local
> disk before we can upload it to OSS. This will poses two problems：
> - if the output file is too large, it will run out of the local disk.
> - if the output file is too large, task will wait long time to upload result
> to OSS before finish, wasting much compute resource.
> 2. {{AliyunOSSBlockOutputStream}}: we cut the task output into small blocks,
> i.e. some small local file, and each block will be packaged into a uploading
> task. These tasks will be submitted into {{SemaphoredDelegatingExecutor}}.
> {{SemaphoredDelegatingExecutor}} will upload this blocks in parallel, this
> will improve performance greatly.
> 3. Each task will retry 3 times to upload block to Aliyun OSS. If one of
> those tasks failed, the whole file uploading will failed, and we will abort
> current uploading.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14698) Make copyFromLocal's -t option available for put as well
[
https://issues.apache.org/jira/browse/HADOOP-14698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436548#comment-16436548
]
genericqa commented on HADOOP-14698:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 10m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 30m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 48s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 30m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 30m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s{color} | {color:green} hadoop-common-project/hadoop-common: The patch
generated 0 new + 54 unchanged - 12 fixed = 54 total (was 66) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 20s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
14s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
39s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}139m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8620d2b |
| JIRA Issue | HADOOP-14698 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12892046/HADOOP-14698.08.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle xml |
| uname | Linux 3321f4603e3c 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 53b3e59 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_162 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14485/testReport/ |
| Max. process+thread count | 1499 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14485/console |
| Powered by | Apache Yetus
[jira] [Commented] (HADOOP-14698) Make copyFromLocal's -t option available for put as well
[ https://issues.apache.org/jira/browse/HADOOP-14698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436296#comment-16436296 ] Jason Cwik commented on HADOOP-14698: - As mentioned above in https://issues.apache.org/jira/browse/HADOOP-14698?focusedCommentId=16107552=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16107552 the current threading model only works for the leaf nodes. In deep/wide tree structures, the enumeration can take a significant amount of time itself, especially when using other FileSystem implementations like S3A or other object store connectors. I started a patch in HDFS-13398 to address this (especially for `ls` or `du` commands) but it could likely be combined with this effort to parallelize the FsShell module in general. So far, we've tried two approaches. The first simply creates another executor in the base class and enqueues the child operations in processPaths. The second approach uses ForkJoinPool to crawl the tree and process subtrees in parallel. Currently, we have FJP working with `ls` and `du`, but not other operations. I think that FJP is the best route since we could do things like wait to delete a directory until all its children have been deleted, but in order to do this properly it might require a significant refactoring of the whole FsShell module to implement the correct ForkJoinTask structure. Thoughts? > Make copyFromLocal's -t option available for put as well > > > Key: HADOOP-14698 > URL: https://issues.apache.org/jira/browse/HADOOP-14698 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Andras Bokor >Assignee: Andras Bokor >Priority: Major > Attachments: HADOOP-14698.01.patch, HADOOP-14698.02.patch, > HADOOP-14698.03.patch, HADOOP-14698.04.patch, HADOOP-14698.05.patch, > HADOOP-14698.06.patch, HADOOP-14698.07.patch, HADOOP-14698.08.patch > > > After HDFS-11786 copyFromLocal and put are no longer identical. > I do not see any reason why not to add the new feature to put as well. > Being non-identical makes the understanding/usage of command more complicated > from user point of view. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15304) [JDK10] Migrate from com.sun.tools.doclets to the replacement
[
https://issues.apache.org/jira/browse/HADOOP-15304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436294#comment-16436294
]
Ajay Kumar commented on HADOOP-15304:
-
[~ajisakaa], [~tasanuma0829] I tried mvn javadoc:javadoc with patch v2 on
jdk10 but it failed with error about missing class
ExcludePrivateAnnotationsStandardDoclet, since it is excluded. Do i need some
additional parameter in mvn?
{code}javadoc: error - Cannot find doclet class
org.apache.hadoop.classification.tools.ExcludePrivateAnnotationsStandardDoclet\{code}
> [JDK10] Migrate from com.sun.tools.doclets to the replacement
> -
>
> Key: HADOOP-15304
> URL: https://issues.apache.org/jira/browse/HADOOP-15304
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Attachments: HADOOP-15304.01.patch, HADOOP-15304.02.patch
>
>
> com.sun.tools.doclets.* packages were removed in Java 10.
> [https://bugs.openjdk.java.net/browse/JDK-8177511]
> This causes hadoop-annotations module to fail.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile)
> on project hadoop-annotations: Compilation failure: Compilation failure:
> [ERROR]
> /Users/ajisaka/git/hadoop/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/tools/IncludePublicAnnotationsStandardDoclet.java:[61,20]
> cannot find symbol
> [ERROR] symbol: method
> validOptions(java.lang.String[][],com.sun.javadoc.DocErrorReporter)
> [ERROR] location: class com.sun.tools.doclets.standard.Standard
> [ERROR]
> /Users/ajisaka/git/hadoop/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/tools/ExcludePrivateAnnotationsStandardDoclet.java:[56,20]
> cannot find symbol
> [ERROR] symbol: method
> validOptions(java.lang.String[][],com.sun.javadoc.DocErrorReporter)
> [ERROR] location: class com.sun.tools.doclets.standard.Standard
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11640) add user defined delimiter support to Configuration
[
https://issues.apache.org/jira/browse/HADOOP-11640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436104#comment-16436104
]
Jim Brennan commented on HADOOP-11640:
--
MAPREDUCE-7069 provides a solution for this specifically for the mapreduce
environment properties ({{mapreduce.map.env}}, {{mapreduce.reduce.env}},
etc...) by providing the ability to specify variables individually. e.g.,
{{mapreduce.map.env.VARNAME=value}}.
> add user defined delimiter support to Configuration
> ---
>
> Key: HADOOP-11640
> URL: https://issues.apache.org/jira/browse/HADOOP-11640
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 2.6.0
>Reporter: Xiaoshuang LU
>Assignee: Xiaoshuang LU
>Priority: Major
> Labels: BB2015-05-TBR
> Attachments: HADOOP-11640.patch
>
>
> As mentioned by org.apache.hadoop.conf.Configuration.getStrings ("Get the
> comma delimited values of the name property as an array of Strings"), only
> comma separated strings can be used. It would be much better if user defined
> separators are supported.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15369) Avoid usage of ${project.version} in parent poms
[
https://issues.apache.org/jira/browse/HADOOP-15369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436060#comment-16436060
]
Bharat Viswanadham commented on HADOOP-15369:
-
LGTM.
Need to add documentation info, for the additional step where we need to modify
hadoop.version to project.version.
+1, after the above, is fixed.
> Avoid usage of ${project.version} in parent poms
>
>
> Key: HADOOP-15369
> URL: https://issues.apache.org/jira/browse/HADOOP-15369
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.2.0
>Reporter: Elek, Marton
>Assignee: Elek, Marton
>Priority: Major
> Attachments: HADOOP-15369-trnk.001.patch
>
>
> hadoop-project/pom.xml and hadoop-project-dist/pom.xml use
> _${project.version}_ variable in dependencyManagement and plugin dependencies.
> Unfortunatelly it could not work if we use different version in a child
> project as ${project.version} variable is resolved *after* the inheritance.
> From [maven
> doc|https://maven.apache.org/guides/introduction/introduction-to-the-pom.html#Project_Inheritance]:
> {quote}
> For example, to access the project.version variable, you would reference it
> like so:
> ${project.version}
> One factor to note is that these variables are processed after inheritance as
> outlined above. This means that if a parent project uses a variable, then its
> definition in the child, not the parent, will be the one eventually used.
> {quote}
> The community voted to keep ozone in-tree but use a different release cycle.
> To achieve this we need different version for selected subproject therefor we
> can't use ${project.version} any more.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14970) MiniHadoopClusterManager doesn't respect lack of format option
[
https://issues.apache.org/jira/browse/HADOOP-14970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436028#comment-16436028
]
Konstantin Shvachko commented on HADOOP-14970:
--
+1 on the change. Should not format DNs if the {{StartupOption.REGULAR}}. Will
commit in a bit.
> MiniHadoopClusterManager doesn't respect lack of format option
> --
>
> Key: HADOOP-14970
> URL: https://issues.apache.org/jira/browse/HADOOP-14970
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Minor
> Attachments: HADOOP-14970.000.patch
>
>
> The CLI MiniCluster, {{MiniHadoopClusterManager}}, says that by default it
> does not format its directories, and provides the {{-format}} option to
> specify that it should do so. However, it builds its {{MiniDFSCluster}} like:
> {code}
> dfs = new MiniDFSCluster.Builder(conf).nameNodePort(nnPort)
> .nameNodeHttpPort(nnHttpPort).numDataNodes(numDataNodes)
> .startupOption(dfsOpts).build();
> {code}
> {{MiniDFSCluster.Builder}}, by default, sets {{format}} to true, so even
> though the {{startupOption}} is {{REGULAR}}, it will still format regardless
> of whether or not the flag is supplied.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15383) wasb FS init hangs for minutes if the container name is wrong (e.g wasb://example.azure/)
[
https://issues.apache.org/jira/browse/HADOOP-15383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435848#comment-16435848
]
Steve Loughran commented on HADOOP-15383:
-
Stack
{code}
"main" #1 prio=5 os_prio=31 tid=0x7fb187006800 nid=0x2403 waiting on
condition [0x72ddf000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(ExecutionEngine.java:255)
at
com.microsoft.azure.storage.blob.CloudBlobContainer.exists(CloudBlobContainer.java:769)
at
com.microsoft.azure.storage.blob.CloudBlobContainer.exists(CloudBlobContainer.java:756)
at
org.apache.hadoop.fs.azure.StorageInterfaceImpl$CloudBlobContainerWrapperImpl.exists(StorageInterfaceImpl.java:233)
at
org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.connectUsingAnonymousCredentials(AzureNativeFileSystemStore.java:856)
at
org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.createAzureStorageSession(AzureNativeFileSystemStore.java:1081)
at
org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.initialize(AzureNativeFileSystemStore.java:538)
at
org.apache.hadoop.fs.azure.NativeAzureFileSystem.initialize(NativeAzureFileSystem.java:1363)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3354)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:124)
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3403)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3371)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:477)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:361)
at
org.apache.hadoop.fs.store.diag.StoreDiag.executeFileSystemOperations(StoreDiag.java:450)
at org.apache.hadoop.fs.store.diag.StoreDiag.run(StoreDiag.java:219)
at org.apache.hadoop.fs.store.diag.StoreDiag.run(StoreDiag.java:195)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at
org.apache.hadoop.fs.store.diag.StoreDiag.diagnostics(StoreDiag.java:655)
{code}
> wasb FS init hangs for minutes if the container name is wrong (e.g
> wasb://example.azure/)
> -
>
> Key: HADOOP-15383
> URL: https://issues.apache.org/jira/browse/HADOOP-15383
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Affects Versions: 3.1.0
>Reporter: Steve Loughran
>Priority: Minor
>
> Happened during some testing: an attempt to create a filesytem with the path
> " wasb://example.azure/" hangs for at least 7 minutes, nothing logged at all.
> Expected: fail fast with UnknownHost exception
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15383) wasb FS init hangs for minutes if the container name is wrong (e.g wasb://example.azure/)
[ https://issues.apache.org/jira/browse/HADOOP-15383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15383: Priority: Minor (was: Major) > wasb FS init hangs for minutes if the container name is wrong (e.g > wasb://example.azure/) > - > > Key: HADOOP-15383 > URL: https://issues.apache.org/jira/browse/HADOOP-15383 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.1.0 >Reporter: Steve Loughran >Priority: Minor > > Happened during some testing: an attempt to create a filesytem with the path > " wasb://example.azure/" hangs for at least 7 minutes, nothing logged at all. > Expected: fail fast with UnknownHost exception -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15383) wasb FS init hangs for minutes if the container name is wrong (e.g wasb://example.azure/)
Steve Loughran created HADOOP-15383: --- Summary: wasb FS init hangs for minutes if the container name is wrong (e.g wasb://example.azure/) Key: HADOOP-15383 URL: https://issues.apache.org/jira/browse/HADOOP-15383 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.1.0 Reporter: Steve Loughran Happened during some testing: an attempt to create a filesytem with the path " wasb://example.azure/" hangs for at least 7 minutes, nothing logged at all. Expected: fail fast with UnknownHost exception -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15350) [JDK10] Update maven plugin tools to fix compile error in hadoop-maven-plugins module
[
https://issues.apache.org/jira/browse/HADOOP-15350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435449#comment-16435449
]
Takanobu Asanuma commented on HADOOP-15350:
---
Thanks, [~ajisakaa]!
> [JDK10] Update maven plugin tools to fix compile error in
> hadoop-maven-plugins module
> -
>
> Key: HADOOP-15350
> URL: https://issues.apache.org/jira/browse/HADOOP-15350
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: build
> Environment: Java 10
>Reporter: Akira Ajisaka
>Assignee: Takanobu Asanuma
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: HADOOP-15350.1.patch
>
>
> {{mvn install -DskipTests}} fails with Java 10.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-plugin-plugin:3.4:descriptor
> (default-descriptor) on project hadoop-maven-plugins: Execution
> default-descriptor of goal
> org.apache.maven.plugins:maven-plugin-plugin:3.4:descriptor failed.
> IllegalArgumentException -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal org.apache.maven.plugins:maven-plugin-plugin:3.4:descriptor
> (default-descriptor) on project hadoop-maven-plugins: Execution
> default-descriptor of goal
> org.apache.maven.plugins:maven-plugin-plugin:3.4:descriptor failed.
> {noformat}
> Upgrading maven plugin tools to 3.5.1 fixes this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15304) [JDK10] Migrate from com.sun.tools.doclets to the replacement
[
https://issues.apache.org/jira/browse/HADOOP-15304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435422#comment-16435422
]
Takanobu Asanuma commented on HADOOP-15304:
---
Thanks for updating it, [~ajisakaa]! I confirmed that {{mvn javadoc:javadoc}}
succeeded with the latest patch through jdk10.
One minor comment is about commons-lang3. Instead of upgrading to the specific
plugin dependency setting, how about upgrading the overall version of
commons-lang3 from 3.4 to 3.7 first? (It may be better to create another jira.)
> [JDK10] Migrate from com.sun.tools.doclets to the replacement
> -
>
> Key: HADOOP-15304
> URL: https://issues.apache.org/jira/browse/HADOOP-15304
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Attachments: HADOOP-15304.01.patch, HADOOP-15304.02.patch
>
>
> com.sun.tools.doclets.* packages were removed in Java 10.
> [https://bugs.openjdk.java.net/browse/JDK-8177511]
> This causes hadoop-annotations module to fail.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile)
> on project hadoop-annotations: Compilation failure: Compilation failure:
> [ERROR]
> /Users/ajisaka/git/hadoop/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/tools/IncludePublicAnnotationsStandardDoclet.java:[61,20]
> cannot find symbol
> [ERROR] symbol: method
> validOptions(java.lang.String[][],com.sun.javadoc.DocErrorReporter)
> [ERROR] location: class com.sun.tools.doclets.standard.Standard
> [ERROR]
> /Users/ajisaka/git/hadoop/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/tools/ExcludePrivateAnnotationsStandardDoclet.java:[56,20]
> cannot find symbol
> [ERROR] symbol: method
> validOptions(java.lang.String[][],com.sun.javadoc.DocErrorReporter)
> [ERROR] location: class com.sun.tools.doclets.standard.Standard
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15304) [JDK10] Migrate from com.sun.tools.doclets to the replacement
[
https://issues.apache.org/jira/browse/HADOOP-15304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435338#comment-16435338
]
genericqa commented on HADOOP-15304:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
28s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 33m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 34m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
81m 50s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 27m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 27m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 58s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
20s{color} | {color:green} hadoop-project in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
22s{color} | {color:green} hadoop-annotations in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}126m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8620d2b |
| JIRA Issue | HADOOP-15304 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12918696/HADOOP-15304.02.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient xml |
| uname | Linux 8a16776f60a6 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4
19:09:19 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 832852c |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_162 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14484/testReport/ |
| Max. process+thread count | 302 (vs. ulimit of 1) |
| modules | C: hadoop-project hadoop-common-project/hadoop-annotations U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14484/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> [JDK10] Migrate from
[jira] [Assigned] (HADOOP-15382) Log kinit output in credential renewal thread
[
https://issues.apache.org/jira/browse/HADOOP-15382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Bota reassigned HADOOP-15382:
---
Assignee: Gabor Bota
> Log kinit output in credential renewal thread
> -
>
> Key: HADOOP-15382
> URL: https://issues.apache.org/jira/browse/HADOOP-15382
> Project: Hadoop Common
> Issue Type: Improvement
> Components: security
>Reporter: Wei-Chiu Chuang
>Assignee: Gabor Bota
>Priority: Minor
>
> We currently run kinit command in a thread to renew kerberos credentials
> periodically.
> {code:java}
> Shell.execCommand(cmd, "-R");
> if (LOG.isDebugEnabled()) {
> LOG.debug("renewed ticket");
> }
> {code}
> It seems useful to log the output of the kinit too.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15350) [JDK10] Update maven plugin tools to fix compile error in hadoop-maven-plugins module
[
https://issues.apache.org/jira/browse/HADOOP-15350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435164#comment-16435164
]
Hudson commented on HADOOP-15350:
-
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13978 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/13978/])
HADOOP-15350. [JDK10] Update maven plugin tools to fix compile error in
(aajisaka: rev 832852ce4ff00d4aa698e89e1df39e5bf0df78b9)
* (edit) hadoop-maven-plugins/pom.xml
> [JDK10] Update maven plugin tools to fix compile error in
> hadoop-maven-plugins module
> -
>
> Key: HADOOP-15350
> URL: https://issues.apache.org/jira/browse/HADOOP-15350
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: build
> Environment: Java 10
>Reporter: Akira Ajisaka
>Assignee: Takanobu Asanuma
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: HADOOP-15350.1.patch
>
>
> {{mvn install -DskipTests}} fails with Java 10.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-plugin-plugin:3.4:descriptor
> (default-descriptor) on project hadoop-maven-plugins: Execution
> default-descriptor of goal
> org.apache.maven.plugins:maven-plugin-plugin:3.4:descriptor failed.
> IllegalArgumentException -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal org.apache.maven.plugins:maven-plugin-plugin:3.4:descriptor
> (default-descriptor) on project hadoop-maven-plugins: Execution
> default-descriptor of goal
> org.apache.maven.plugins:maven-plugin-plugin:3.4:descriptor failed.
> {noformat}
> Upgrading maven plugin tools to 3.5.1 fixes this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15304) [JDK10] Migrate from com.sun.tools.doclets to the replacement
[
https://issues.apache.org/jira/browse/HADOOP-15304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435161#comment-16435161
]
Akira Ajisaka commented on HADOOP-15304:
02 patch:
* Set commons-lang3 version to 3.7 in maven-javadoc-plugin
* Ignore org.apache.hadoop.classification.tools package in maven-javadoc-plugin
I can run {{mvn javadoc:javadoc}} successfully in hadoop-annotation directory.
> [JDK10] Migrate from com.sun.tools.doclets to the replacement
> -
>
> Key: HADOOP-15304
> URL: https://issues.apache.org/jira/browse/HADOOP-15304
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Attachments: HADOOP-15304.01.patch, HADOOP-15304.02.patch
>
>
> com.sun.tools.doclets.* packages were removed in Java 10.
> [https://bugs.openjdk.java.net/browse/JDK-8177511]
> This causes hadoop-annotations module to fail.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile)
> on project hadoop-annotations: Compilation failure: Compilation failure:
> [ERROR]
> /Users/ajisaka/git/hadoop/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/tools/IncludePublicAnnotationsStandardDoclet.java:[61,20]
> cannot find symbol
> [ERROR] symbol: method
> validOptions(java.lang.String[][],com.sun.javadoc.DocErrorReporter)
> [ERROR] location: class com.sun.tools.doclets.standard.Standard
> [ERROR]
> /Users/ajisaka/git/hadoop/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/tools/ExcludePrivateAnnotationsStandardDoclet.java:[56,20]
> cannot find symbol
> [ERROR] symbol: method
> validOptions(java.lang.String[][],com.sun.javadoc.DocErrorReporter)
> [ERROR] location: class com.sun.tools.doclets.standard.Standard
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15304) [JDK10] Migrate from com.sun.tools.doclets to the replacement
[
https://issues.apache.org/jira/browse/HADOOP-15304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-15304:
---
Attachment: HADOOP-15304.02.patch
> [JDK10] Migrate from com.sun.tools.doclets to the replacement
> -
>
> Key: HADOOP-15304
> URL: https://issues.apache.org/jira/browse/HADOOP-15304
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Attachments: HADOOP-15304.01.patch, HADOOP-15304.02.patch
>
>
> com.sun.tools.doclets.* packages were removed in Java 10.
> [https://bugs.openjdk.java.net/browse/JDK-8177511]
> This causes hadoop-annotations module to fail.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile)
> on project hadoop-annotations: Compilation failure: Compilation failure:
> [ERROR]
> /Users/ajisaka/git/hadoop/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/tools/IncludePublicAnnotationsStandardDoclet.java:[61,20]
> cannot find symbol
> [ERROR] symbol: method
> validOptions(java.lang.String[][],com.sun.javadoc.DocErrorReporter)
> [ERROR] location: class com.sun.tools.doclets.standard.Standard
> [ERROR]
> /Users/ajisaka/git/hadoop/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/tools/ExcludePrivateAnnotationsStandardDoclet.java:[56,20]
> cannot find symbol
> [ERROR] symbol: method
> validOptions(java.lang.String[][],com.sun.javadoc.DocErrorReporter)
> [ERROR] location: class com.sun.tools.doclets.standard.Standard
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15304) [JDK10] Migrate from com.sun.tools.doclets to the replacement
[
https://issues.apache.org/jira/browse/HADOOP-15304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435159#comment-16435159
]
Akira Ajisaka commented on HADOOP-15304:
Thanks [~tasanuma0829] for trying the patch!
I tried {{mvn javadoc:javadoc}} and failed due to MJAVADOC-517. We need to
update the maven javadoc plugin to 3.0.1 but the version is not released now.
We can avoid this error by specifying commons-lang3 version to the plugin
dependency setting.
As [~tasanuma0829] pointed out, we need to exclude the package in
maven-javadoc-plugin.
> [JDK10] Migrate from com.sun.tools.doclets to the replacement
> -
>
> Key: HADOOP-15304
> URL: https://issues.apache.org/jira/browse/HADOOP-15304
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Attachments: HADOOP-15304.01.patch
>
>
> com.sun.tools.doclets.* packages were removed in Java 10.
> [https://bugs.openjdk.java.net/browse/JDK-8177511]
> This causes hadoop-annotations module to fail.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile)
> on project hadoop-annotations: Compilation failure: Compilation failure:
> [ERROR]
> /Users/ajisaka/git/hadoop/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/tools/IncludePublicAnnotationsStandardDoclet.java:[61,20]
> cannot find symbol
> [ERROR] symbol: method
> validOptions(java.lang.String[][],com.sun.javadoc.DocErrorReporter)
> [ERROR] location: class com.sun.tools.doclets.standard.Standard
> [ERROR]
> /Users/ajisaka/git/hadoop/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/tools/ExcludePrivateAnnotationsStandardDoclet.java:[56,20]
> cannot find symbol
> [ERROR] symbol: method
> validOptions(java.lang.String[][],com.sun.javadoc.DocErrorReporter)
> [ERROR] location: class com.sun.tools.doclets.standard.Standard
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15350) [JDK10] Update maven plugin tools to fix compile error in hadoop-maven-plugins module
[
https://issues.apache.org/jira/browse/HADOOP-15350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-15350:
---
Summary: [JDK10] Update maven plugin tools to fix compile error in
hadoop-maven-plugins module (was: Update maven plugin tools to fix compile
error in hadoop-maven-plugins module)
> [JDK10] Update maven plugin tools to fix compile error in
> hadoop-maven-plugins module
> -
>
> Key: HADOOP-15350
> URL: https://issues.apache.org/jira/browse/HADOOP-15350
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: build
> Environment: Java 10
>Reporter: Akira Ajisaka
>Assignee: Takanobu Asanuma
>Priority: Major
> Attachments: HADOOP-15350.1.patch
>
>
> {{mvn install -DskipTests}} fails with Java 10.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-plugin-plugin:3.4:descriptor
> (default-descriptor) on project hadoop-maven-plugins: Execution
> default-descriptor of goal
> org.apache.maven.plugins:maven-plugin-plugin:3.4:descriptor failed.
> IllegalArgumentException -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal org.apache.maven.plugins:maven-plugin-plugin:3.4:descriptor
> (default-descriptor) on project hadoop-maven-plugins: Execution
> default-descriptor of goal
> org.apache.maven.plugins:maven-plugin-plugin:3.4:descriptor failed.
> {noformat}
> Upgrading maven plugin tools to 3.5.1 fixes this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15350) [JDK10] Update maven plugin tools to fix compile error in hadoop-maven-plugins module
[
https://issues.apache.org/jira/browse/HADOOP-15350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-15350:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 3.2.0
Status: Resolved (was: Patch Available)
Committed this to trunk. Thanks [~tasanuma0829] for the contribution.
> [JDK10] Update maven plugin tools to fix compile error in
> hadoop-maven-plugins module
> -
>
> Key: HADOOP-15350
> URL: https://issues.apache.org/jira/browse/HADOOP-15350
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: build
> Environment: Java 10
>Reporter: Akira Ajisaka
>Assignee: Takanobu Asanuma
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: HADOOP-15350.1.patch
>
>
> {{mvn install -DskipTests}} fails with Java 10.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-plugin-plugin:3.4:descriptor
> (default-descriptor) on project hadoop-maven-plugins: Execution
> default-descriptor of goal
> org.apache.maven.plugins:maven-plugin-plugin:3.4:descriptor failed.
> IllegalArgumentException -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal org.apache.maven.plugins:maven-plugin-plugin:3.4:descriptor
> (default-descriptor) on project hadoop-maven-plugins: Execution
> default-descriptor of goal
> org.apache.maven.plugins:maven-plugin-plugin:3.4:descriptor failed.
> {noformat}
> Upgrading maven plugin tools to 3.5.1 fixes this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15350) Update maven plugin tools to fix compile error in hadoop-maven-plugins module
[
https://issues.apache.org/jira/browse/HADOOP-15350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435135#comment-16435135
]
Akira Ajisaka commented on HADOOP-15350:
+1
> Update maven plugin tools to fix compile error in hadoop-maven-plugins module
> -
>
> Key: HADOOP-15350
> URL: https://issues.apache.org/jira/browse/HADOOP-15350
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: build
> Environment: Java 10
>Reporter: Akira Ajisaka
>Assignee: Takanobu Asanuma
>Priority: Major
> Attachments: HADOOP-15350.1.patch
>
>
> {{mvn install -DskipTests}} fails with Java 10.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-plugin-plugin:3.4:descriptor
> (default-descriptor) on project hadoop-maven-plugins: Execution
> default-descriptor of goal
> org.apache.maven.plugins:maven-plugin-plugin:3.4:descriptor failed.
> IllegalArgumentException -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal org.apache.maven.plugins:maven-plugin-plugin:3.4:descriptor
> (default-descriptor) on project hadoop-maven-plugins: Execution
> default-descriptor of goal
> org.apache.maven.plugins:maven-plugin-plugin:3.4:descriptor failed.
> {noformat}
> Upgrading maven plugin tools to 3.5.1 fixes this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15304) [JDK10] Migrate from com.sun.tools.doclets to the replacement
[
https://issues.apache.org/jira/browse/HADOOP-15304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435076#comment-16435076
]
Takanobu Asanuma commented on HADOOP-15304:
---
Thanks for working on this, [~ajisakaa]. I confirmed that {{mvn clean install
-DskipTests}} succeeded with jdk10. But {{mvn javadoc:javadoc}} still fails due
to {{com.sun.tools.doclets}}. Let's exclude it in {{maven-javadoc-plugin}} too.
> [JDK10] Migrate from com.sun.tools.doclets to the replacement
> -
>
> Key: HADOOP-15304
> URL: https://issues.apache.org/jira/browse/HADOOP-15304
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Attachments: HADOOP-15304.01.patch
>
>
> com.sun.tools.doclets.* packages were removed in Java 10.
> [https://bugs.openjdk.java.net/browse/JDK-8177511]
> This causes hadoop-annotations module to fail.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile)
> on project hadoop-annotations: Compilation failure: Compilation failure:
> [ERROR]
> /Users/ajisaka/git/hadoop/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/tools/IncludePublicAnnotationsStandardDoclet.java:[61,20]
> cannot find symbol
> [ERROR] symbol: method
> validOptions(java.lang.String[][],com.sun.javadoc.DocErrorReporter)
> [ERROR] location: class com.sun.tools.doclets.standard.Standard
> [ERROR]
> /Users/ajisaka/git/hadoop/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/tools/ExcludePrivateAnnotationsStandardDoclet.java:[56,20]
> cannot find symbol
> [ERROR] symbol: method
> validOptions(java.lang.String[][],com.sun.javadoc.DocErrorReporter)
> [ERROR] location: class com.sun.tools.doclets.standard.Standard
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
