[jira] [Updated] (HADOOP-12098) Unnecessary dependencies in Hadoop Archives
[ https://issues.apache.org/jira/browse/HADOOP-12098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated HADOOP-12098: -- Attachment: (was: HADOOP-12098.01.patch) Unnecessary dependencies in Hadoop Archives --- Key: HADOOP-12098 URL: https://issues.apache.org/jira/browse/HADOOP-12098 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.7.0 Reporter: Varun Saxena Assignee: Varun Saxena Priority: Minor Attachments: HADOOP-12098.001.patch hadoop-mapreduce-client-hs and hadoop-mapreduce-client-jobclient dependencies are not required in Hadoop archives -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12098) Unnecessary dependencies in Hadoop Archives
[
https://issues.apache.org/jira/browse/HADOOP-12098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604021#comment-14604021
]
Varun Saxena commented on HADOOP-12098:
---
Tests fail due to runtime dependence on jobclient main jar. The test jar and hs
jars are still unnecessary.
The runtime dependence comes due to subclasses of {{ClientProtocolProvider}}
classes being loaded at runtime
{{ClientProtocolProvider}} exists in {{hadoop-mapreduce-core}}.
{{LocalClientProtocolProvider}} and associated classes such as
{{LocalJobRunner}} exist in {{hadoop-mapreduce-client-common}}
{{YarnClientProtocolProvider}} and associated classes such as {{YarnRunner}}
exist in {{hadoop-mapreduce-jobclient}}.
Any reason why these classes have been kept this way ? All can move to
{{hadoop-mapreduce-client-core}} IMHO. This creates unnecessary dependencies.
{{hadoop-mapreduce-client-jobclient}} seems to be having hardly any main
classes now. Is it even required as a separate submodule ?
Unnecessary dependencies in Hadoop Archives
---
Key: HADOOP-12098
URL: https://issues.apache.org/jira/browse/HADOOP-12098
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.7.0
Reporter: Varun Saxena
Assignee: Varun Saxena
Priority: Minor
Attachments: HADOOP-12098.001.patch
hadoop-mapreduce-client-hs and hadoop-mapreduce-client-jobclient dependencies
are not required in Hadoop archives
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HADOOP-12129) rework test-patch bug system support
Allen Wittenauer created HADOOP-12129: - Summary: rework test-patch bug system support Key: HADOOP-12129 URL: https://issues.apache.org/jira/browse/HADOOP-12129 Project: Hadoop Common Issue Type: Sub-task Reporter: Allen Wittenauer WARNING: this is a fairly big project. See first comment for a brain dump on the issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HADOOP-12127) some personalities are still using releaseaudit instead of asflicense
[ https://issues.apache.org/jira/browse/HADOOP-12127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer reassigned HADOOP-12127: - Assignee: Allen Wittenauer some personalities are still using releaseaudit instead of asflicense - Key: HADOOP-12127 URL: https://issues.apache.org/jira/browse/HADOOP-12127 Project: Hadoop Common Issue Type: Sub-task Components: yetus Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer Assignee: Allen Wittenauer Priority: Trivial Simple bug: releaseaudit test was renamed to be asflicense. Some personalities are still using the old name and therefore doing the wrong thing. Just need to rename them in the personality files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12098) Unnecessary dependencies in Hadoop Archives
[ https://issues.apache.org/jira/browse/HADOOP-12098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604050#comment-14604050 ] Varun Saxena commented on HADOOP-12098: --- Anyways, there is quite a bit of cross dependency in classes used so not going to touch these mapreduce modules. Removing dependencies from archives for now. Unnecessary dependencies in Hadoop Archives --- Key: HADOOP-12098 URL: https://issues.apache.org/jira/browse/HADOOP-12098 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.7.0 Reporter: Varun Saxena Assignee: Varun Saxena Priority: Minor Attachments: HADOOP-12098.001.patch, HADOOP-12098.02.patch hadoop-mapreduce-client-hs and hadoop-mapreduce-client-jobclient dependencies are not required in Hadoop archives -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12098) Unnecessary dependencies in Hadoop Archives
[ https://issues.apache.org/jira/browse/HADOOP-12098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated HADOOP-12098: -- Description: hadoop-mapreduce-client-hs and hadoop-mapreduce-client-jobclient(test jar) dependencies are not required in Hadoop archives (was: hadoop-mapreduce-client-hs and hadoop-mapreduce-client-jobclient dependencies are not required in Hadoop archives) Unnecessary dependencies in Hadoop Archives --- Key: HADOOP-12098 URL: https://issues.apache.org/jira/browse/HADOOP-12098 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.7.0 Reporter: Varun Saxena Assignee: Varun Saxena Priority: Minor Attachments: HADOOP-12098.001.patch, HADOOP-12098.02.patch hadoop-mapreduce-client-hs and hadoop-mapreduce-client-jobclient(test jar) dependencies are not required in Hadoop archives -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12098) Unnecessary dependencies in Hadoop Archives
[ https://issues.apache.org/jira/browse/HADOOP-12098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated HADOOP-12098: -- Attachment: HADOOP-12098.02.patch Unnecessary dependencies in Hadoop Archives --- Key: HADOOP-12098 URL: https://issues.apache.org/jira/browse/HADOOP-12098 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.7.0 Reporter: Varun Saxena Assignee: Varun Saxena Priority: Minor Attachments: HADOOP-12098.001.patch, HADOOP-12098.02.patch hadoop-mapreduce-client-hs and hadoop-mapreduce-client-jobclient dependencies are not required in Hadoop archives -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HADOOP-12121) smarter branch detection
[ https://issues.apache.org/jira/browse/HADOOP-12121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer reassigned HADOOP-12121: - Assignee: Allen Wittenauer smarter branch detection Key: HADOOP-12121 URL: https://issues.apache.org/jira/browse/HADOOP-12121 Project: Hadoop Common Issue Type: Sub-task Components: yetus Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer Assignee: Allen Wittenauer We should make branch detection smarter so that it works on micro versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12127) some personalities are still using releaseaudit instead of asflicense
[ https://issues.apache.org/jira/browse/HADOOP-12127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-12127: -- Affects Version/s: HADOOP-12111 some personalities are still using releaseaudit instead of asflicense - Key: HADOOP-12127 URL: https://issues.apache.org/jira/browse/HADOOP-12127 Project: Hadoop Common Issue Type: Sub-task Components: yetus Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer Priority: Trivial Simple bug: releaseaudit test was renamed to be asflicense. Some personalities are still using the old name and therefore doing the wrong thing. Just need to rename them in the personality files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12121) smarter branch detection
[ https://issues.apache.org/jira/browse/HADOOP-12121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604033#comment-14604033 ] Allen Wittenauer commented on HADOOP-12121: --- bq. I was wondering if it makes sense to a case insensitive branch name compare ? or just doc it somewhere ? I'm tempted to doc it only because I could see in the (far?) future supporting other SCMs that are case sensitive. I'll play around to see how painful it is. Right now, this chunk of code is already relatively slow for code bases with lots of branches... like Hadoop. :D smarter branch detection Key: HADOOP-12121 URL: https://issues.apache.org/jira/browse/HADOOP-12121 Project: Hadoop Common Issue Type: Sub-task Components: yetus Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer Assignee: Allen Wittenauer We should make branch detection smarter so that it works on micro versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12123) Add example test-patch plugin for commit message format
[ https://issues.apache.org/jira/browse/HADOOP-12123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-12123: -- Affects Version/s: HADOOP-12111 Add example test-patch plugin for commit message format --- Key: HADOOP-12123 URL: https://issues.apache.org/jira/browse/HADOOP-12123 Project: Hadoop Common Issue Type: Sub-task Components: yetus Affects Versions: HADOOP-12111 Reporter: Sean Busbey add an example test plugin to ensures when we get a diff that includes a commit message that that commit message starts with a jira ID. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12129) rework test-patch bug system support
[ https://issues.apache.org/jira/browse/HADOOP-12129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-12129: -- Affects Version/s: HADOOP-12111 rework test-patch bug system support Key: HADOOP-12129 URL: https://issues.apache.org/jira/browse/HADOOP-12129 Project: Hadoop Common Issue Type: Sub-task Components: yetus Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer WARNING: this is a fairly big project. See first comment for a brain dump on the issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12128) test-patch docker mode doesn't support non-project personalities
[ https://issues.apache.org/jira/browse/HADOOP-12128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-12128: -- Affects Version/s: HADOOP-12111 test-patch docker mode doesn't support non-project personalities Key: HADOOP-12128 URL: https://issues.apache.org/jira/browse/HADOOP-12128 Project: Hadoop Common Issue Type: Sub-task Components: yetus Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer If you provide a personality file that doesn't match the project name, docker mode won't use it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-12127) some personalities are still using releaseaudit instead of asflicense
Allen Wittenauer created HADOOP-12127: - Summary: some personalities are still using releaseaudit instead of asflicense Key: HADOOP-12127 URL: https://issues.apache.org/jira/browse/HADOOP-12127 Project: Hadoop Common Issue Type: Sub-task Reporter: Allen Wittenauer Priority: Trivial Simple bug: releaseaudit test was renamed to be asflicense. Some personalities are still using the old name and therefore doing the wrong thing. Just need to rename them in the personality files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12113) update test-patch branch to latest code
[ https://issues.apache.org/jira/browse/HADOOP-12113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-12113: -- Resolution: Fixed Status: Resolved (was: Patch Available) Awesome. Thanks! Committed. Now to file a bunch of JIRAs... :) update test-patch branch to latest code --- Key: HADOOP-12113 URL: https://issues.apache.org/jira/browse/HADOOP-12113 Project: Hadoop Common Issue Type: Sub-task Components: yetus Reporter: Allen Wittenauer Assignee: Allen Wittenauer Fix For: HADOOP-12111 Attachments: HADOOP-12113-HADOOP-12111.patch, HADOOP-12113.HADOOP-12111.01.patch [~sekikn] and I have been working on github. We should update the codebase to reflect all of those changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-12128) test-patch docker mode doesn't support non-project personalities
Allen Wittenauer created HADOOP-12128: - Summary: test-patch docker mode doesn't support non-project personalities Key: HADOOP-12128 URL: https://issues.apache.org/jira/browse/HADOOP-12128 Project: Hadoop Common Issue Type: Sub-task Reporter: Allen Wittenauer If you provide a personality file that doesn't match the project name, docker mode won't use it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12111) [Umbrella] Split test-patch off into its own TLP
[ https://issues.apache.org/jira/browse/HADOOP-12111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604035#comment-14604035 ] Allen Wittenauer commented on HADOOP-12111: --- Do we want to use CHANGES.txt or suck in releasedocmaker early so that we don't have to deal with it? At a minimum, we should probably put an excludes line in the pom.xml to stop triggering rat checks. [Umbrella] Split test-patch off into its own TLP Key: HADOOP-12111 URL: https://issues.apache.org/jira/browse/HADOOP-12111 Project: Hadoop Common Issue Type: New Feature Components: yetus Reporter: Allen Wittenauer Given test-patch's tendency to get forked into a variety of different projects, it makes a lot of sense to make an Apache TLP so that everyone can benefit from a common code base. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12098) Unnecessary dependencies in Hadoop Archives
[ https://issues.apache.org/jira/browse/HADOOP-12098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated HADOOP-12098: -- Status: Open (was: Patch Available) Unnecessary dependencies in Hadoop Archives --- Key: HADOOP-12098 URL: https://issues.apache.org/jira/browse/HADOOP-12098 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.7.0 Reporter: Varun Saxena Assignee: Varun Saxena Priority: Minor Attachments: HADOOP-12098.001.patch, HADOOP-12098.02.patch hadoop-mapreduce-client-hs and hadoop-mapreduce-client-jobclient(test jar) dependencies are not required in Hadoop archives -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12098) Unnecessary dependencies in Hadoop Archives
[ https://issues.apache.org/jira/browse/HADOOP-12098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated HADOOP-12098: -- Status: Patch Available (was: Open) Unnecessary dependencies in Hadoop Archives --- Key: HADOOP-12098 URL: https://issues.apache.org/jira/browse/HADOOP-12098 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.7.0 Reporter: Varun Saxena Assignee: Varun Saxena Priority: Minor Attachments: HADOOP-12098.001.patch, HADOOP-12098.02.patch hadoop-mapreduce-client-hs and hadoop-mapreduce-client-jobclient(test jar) dependencies are not required in Hadoop archives -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12098) Unnecessary dependencies in Hadoop Archives
[
https://issues.apache.org/jira/browse/HADOOP-12098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604059#comment-14604059
]
Hadoop QA commented on HADOOP-12098:
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 44s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 30s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 35s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 37s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | tools/hadoop tests | 0m 50s | Tests passed in
hadoop-archives. |
| | | 35m 15s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12742306/HADOOP-12098.02.patch |
| Optional Tests | javadoc javac unit |
| git revision | trunk / fe6c1bd |
| hadoop-archives test log |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7070/artifact/patchprocess/testrun_hadoop-archives.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7070/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7070/console |
This message was automatically generated.
Unnecessary dependencies in Hadoop Archives
---
Key: HADOOP-12098
URL: https://issues.apache.org/jira/browse/HADOOP-12098
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.7.0
Reporter: Varun Saxena
Assignee: Varun Saxena
Priority: Minor
Attachments: HADOOP-12098.001.patch, HADOOP-12098.02.patch
hadoop-mapreduce-client-hs and hadoop-mapreduce-client-jobclient(test jar)
dependencies are not required in Hadoop archives
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12113) update test-patch branch to latest code
[ https://issues.apache.org/jira/browse/HADOOP-12113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-12113: -- Fix Version/s: HADOOP-12111 update test-patch branch to latest code --- Key: HADOOP-12113 URL: https://issues.apache.org/jira/browse/HADOOP-12113 Project: Hadoop Common Issue Type: Sub-task Components: yetus Reporter: Allen Wittenauer Assignee: Allen Wittenauer Fix For: HADOOP-12111 Attachments: HADOOP-12113-HADOOP-12111.patch, HADOOP-12113.HADOOP-12111.01.patch [~sekikn] and I have been working on github. We should update the codebase to reflect all of those changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12129) rework test-patch bug system support
[ https://issues.apache.org/jira/browse/HADOOP-12129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604026#comment-14604026 ] Allen Wittenauer commented on HADOOP-12129: --- There are a bunch of interleaved issues in play here. This is a summary of some of the discussions that [~sekikn] and I had around these topics. * Some projects use github PRs as their patch repository, with comments going to both github and JIRA. In these projects (e.g., tajo), JIRA status is never patch available and a comment is made by an ASF infra bot in the JIRA that points to the github PR. * It's not uncommon to have JIRA point to another system for code reviews and patch repository via JIRA APIs. * In the future, we should be able to support more than one bug system simultaneously. e.g., github:project/# jira:project/# bugzilla:project/# would push a precommit message to all three. * In the far future, it'd be nice to be able to support multiples of the same comment system. * Bonus points: How does reviewboard, gerrit, etc fit into this? At a minimum, I think we need to do this list sooner rather than later: a) pull more of the JIRA-specific code out of locate_patch and into the jira.sh plugin b) support the not-patch-available-but-referenced-in-JIRA-comment use case rework test-patch bug system support Key: HADOOP-12129 URL: https://issues.apache.org/jira/browse/HADOOP-12129 Project: Hadoop Common Issue Type: Sub-task Components: yetus Reporter: Allen Wittenauer WARNING: this is a fairly big project. See first comment for a brain dump on the issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-12130) document features added in 12113
Allen Wittenauer created HADOOP-12130: - Summary: document features added in 12113 Key: HADOOP-12130 URL: https://issues.apache.org/jira/browse/HADOOP-12130 Project: Hadoop Common Issue Type: Sub-task Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer We need to write up more documentation, especially around the new stuff added in 12113, since a lot of people are looking for information on those bits. (e.g., How do I use multiJDK mode?) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12111) [Umbrella] Split test-patch off into its own TLP
[ https://issues.apache.org/jira/browse/HADOOP-12111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-12111: -- Affects Version/s: HADOOP-12111 [Umbrella] Split test-patch off into its own TLP Key: HADOOP-12111 URL: https://issues.apache.org/jira/browse/HADOOP-12111 Project: Hadoop Common Issue Type: New Feature Components: yetus Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer Given test-patch's tendency to get forked into a variety of different projects, it makes a lot of sense to make an Apache TLP so that everyone can benefit from a common code base. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12036) Consolidate all of the cmake extensions in one directory
[ https://issues.apache.org/jira/browse/HADOOP-12036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604175#comment-14604175 ] Hudson commented on HADOOP-12036: - SUCCESS: Integrated in Hadoop-Hdfs-trunk #2169 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2169/]) HADOOP-12036. Consolidate all of the cmake extensions in one directory (alanburlison via cmccabe) (cmccabe: rev aa07dea3577158b92a17651d10da20df73f54561) * hadoop-common-project/hadoop-common/HadoopJNI.cmake * hadoop-common-project/hadoop-common/src/CMakeLists.txt * hadoop-common-project/hadoop-common/HadoopCommon.cmake * hadoop-common-project/hadoop-common/CHANGES.txt Consolidate all of the cmake extensions in one directory Key: HADOOP-12036 URL: https://issues.apache.org/jira/browse/HADOOP-12036 Project: Hadoop Common Issue Type: Sub-task Reporter: Allen Wittenauer Assignee: Alan Burlison Fix For: 2.8.0 Attachments: HADOOP-12036.001.patch, HADOOP-12036.002.patch, HADOOP-12036.004.patch, HADOOP-12036.005.patch Rather than have a half-dozen redefinitions, custom extensions, etc, we should move them all to one location so that the cmake environment is consistent between the various native components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12036) Consolidate all of the cmake extensions in one directory
[ https://issues.apache.org/jira/browse/HADOOP-12036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604113#comment-14604113 ] Hudson commented on HADOOP-12036: - FAILURE: Integrated in Hadoop-Yarn-trunk #971 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/971/]) HADOOP-12036. Consolidate all of the cmake extensions in one directory (alanburlison via cmccabe) (cmccabe: rev aa07dea3577158b92a17651d10da20df73f54561) * hadoop-common-project/hadoop-common/HadoopJNI.cmake * hadoop-common-project/hadoop-common/HadoopCommon.cmake * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-common-project/hadoop-common/src/CMakeLists.txt Consolidate all of the cmake extensions in one directory Key: HADOOP-12036 URL: https://issues.apache.org/jira/browse/HADOOP-12036 Project: Hadoop Common Issue Type: Sub-task Reporter: Allen Wittenauer Assignee: Alan Burlison Fix For: 2.8.0 Attachments: HADOOP-12036.001.patch, HADOOP-12036.002.patch, HADOOP-12036.004.patch, HADOOP-12036.005.patch Rather than have a half-dozen redefinitions, custom extensions, etc, we should move them all to one location so that the cmake environment is consistent between the various native components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12118) Validate xml configuration files with XML Schema
[
https://issues.apache.org/jira/browse/HADOOP-12118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604182#comment-14604182
]
Kengo Seki commented on HADOOP-12118:
-
Thanks [~gliptak] for the patch. Because HADOOP-7947 depends on HADOOP-6964,
would you backport that as well?
HADOOP-12071 should be backported also, but it is not committed yet.
Thanks [~ctubbsii] for the comment. I agree that XSD is clearer and more
portable. I chose current design because it seemed difficult (maybe
impossible?) to represent the rule to detect [these
patterns|https://issues.apache.org/jira/browse/HADOOP-7947?focusedCommentId=14343946page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14343946]
as an XSD. If someone provides the equivalent XSD, replacement is worth
considering.
Validate xml configuration files with XML Schema
Key: HADOOP-12118
URL: https://issues.apache.org/jira/browse/HADOOP-12118
Project: Hadoop Common
Issue Type: Improvement
Reporter: Christopher Tubbs
Attachments: HADOOP-7947.branch-2.1.patch
I spent an embarrassingly long time today trying to figure out why the
following wouldn't work.
{code}
property
keyfs.defaultFS/key
valuehdfs://localhost:9000/value
/property
{code}
I just kept getting an error about no authority for {{fs.defaultFS}}, with a
value of {{file:///}}, which made no sense... because I knew it was there.
The problem was that the {{core-site.xml}} was parsed entirely without any
validation. This seems incorrect. The very least that could be done is a
simple XML Schema validation against an XSD, before parsing. That way, users
will get immediate failures on common typos and other problems in the xml
configuration files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12118) Validate xml configuration files with XML Schema
[
https://issues.apache.org/jira/browse/HADOOP-12118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604185#comment-14604185
]
Gabor Liptak commented on HADOOP-12118:
---
Considering that current config XML files do not have a schema, manual
parsing seems to be a good backward compatibility solution.
Maven publishes their schemas at:
http://maven.apache.org/xsd/
like
http://maven.apache.org/xsd/maven-4.0.0.xsd
Could a similar setup be used?
http://hadoop.apache.org/xsd/
While most of the config files use the
configuration/property/[name/value/description/final]
hierarchy, there is also fair-scheduler.xml
https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/fair-scheduler.xml
with its own schema.
Validate xml configuration files with XML Schema
Key: HADOOP-12118
URL: https://issues.apache.org/jira/browse/HADOOP-12118
Project: Hadoop Common
Issue Type: Improvement
Reporter: Christopher Tubbs
Attachments: HADOOP-7947.branch-2.1.patch
I spent an embarrassingly long time today trying to figure out why the
following wouldn't work.
{code}
property
keyfs.defaultFS/key
valuehdfs://localhost:9000/value
/property
{code}
I just kept getting an error about no authority for {{fs.defaultFS}}, with a
value of {{file:///}}, which made no sense... because I knew it was there.
The problem was that the {{core-site.xml}} was parsed entirely without any
validation. This seems incorrect. The very least that could be done is a
simple XML Schema validation against an XSD, before parsing. That way, users
will get immediate failures on common typos and other problems in the xml
configuration files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12036) Consolidate all of the cmake extensions in one directory
[ https://issues.apache.org/jira/browse/HADOOP-12036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604167#comment-14604167 ] Hudson commented on HADOOP-12036: - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #230 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/230/]) HADOOP-12036. Consolidate all of the cmake extensions in one directory (alanburlison via cmccabe) (cmccabe: rev aa07dea3577158b92a17651d10da20df73f54561) * hadoop-common-project/hadoop-common/src/CMakeLists.txt * hadoop-common-project/hadoop-common/HadoopJNI.cmake * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-common-project/hadoop-common/HadoopCommon.cmake Consolidate all of the cmake extensions in one directory Key: HADOOP-12036 URL: https://issues.apache.org/jira/browse/HADOOP-12036 Project: Hadoop Common Issue Type: Sub-task Reporter: Allen Wittenauer Assignee: Alan Burlison Fix For: 2.8.0 Attachments: HADOOP-12036.001.patch, HADOOP-12036.002.patch, HADOOP-12036.004.patch, HADOOP-12036.005.patch Rather than have a half-dozen redefinitions, custom extensions, etc, we should move them all to one location so that the cmake environment is consistent between the various native components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12111) [Umbrella] Split test-patch off into its own TLP
[ https://issues.apache.org/jira/browse/HADOOP-12111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604181#comment-14604181 ] Sean Busbey commented on HADOOP-12111: -- I have hated maintaining CHANGES.text in every project I've used it in. Unless someone else is strongly in favor, I say cut it out. If we want to include a record of what has changed in our release artifacts, the releasedocmaker output will do just fine. [Umbrella] Split test-patch off into its own TLP Key: HADOOP-12111 URL: https://issues.apache.org/jira/browse/HADOOP-12111 Project: Hadoop Common Issue Type: New Feature Components: yetus Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer Given test-patch's tendency to get forked into a variety of different projects, it makes a lot of sense to make an Apache TLP so that everyone can benefit from a common code base. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-12131) The different JDKs which versions are the same aren't distiguished on multi-JDK mode
Kengo Seki created HADOOP-12131:
---
Summary: The different JDKs which versions are the same aren't
distiguished on multi-JDK mode
Key: HADOOP-12131
URL: https://issues.apache.org/jira/browse/HADOOP-12131
Project: Hadoop Common
Issue Type: Sub-task
Components: yetus
Affects Versions: HADOOP-12111
Reporter: Kengo Seki
If one installs the same version of Oracle JDK and OpenJDK and enables the
multi-JDK option, output files are overwritten by the last one because the same
value is assigned to $jdk and $fn. Example:
{code}
Pre-patch trunk javac compilation
mvn --batch-mode clean compile -DskipTests -Pnative -Drequire.libwebhdfs
-Drequire.snappy -Drequire.openssl -Drequire.fuse -Drequire.test.libhadoop
-Ptest-patch
/tmp/test-patch-hadoop/25579/branch-javac-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt
21
Elapsed: 1m 0s
mvn --batch-mode clean compile -DskipTests -Pnative -Drequire.libwebhdfs
-Drequire.snappy -Drequire.openssl -Drequire.fuse -Drequire.test.libhadoop
-Ptest-patch
/tmp/test-patch-hadoop/25579/branch-javac-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt
21
Elapsed: 0m 54s
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12036) Consolidate all of the cmake extensions in one directory
[ https://issues.apache.org/jira/browse/HADOOP-12036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604106#comment-14604106 ] Hudson commented on HADOOP-12036: - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #241 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/241/]) HADOOP-12036. Consolidate all of the cmake extensions in one directory (alanburlison via cmccabe) (cmccabe: rev aa07dea3577158b92a17651d10da20df73f54561) * hadoop-common-project/hadoop-common/src/CMakeLists.txt * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-common-project/hadoop-common/HadoopJNI.cmake * hadoop-common-project/hadoop-common/HadoopCommon.cmake Consolidate all of the cmake extensions in one directory Key: HADOOP-12036 URL: https://issues.apache.org/jira/browse/HADOOP-12036 Project: Hadoop Common Issue Type: Sub-task Reporter: Allen Wittenauer Assignee: Alan Burlison Fix For: 2.8.0 Attachments: HADOOP-12036.001.patch, HADOOP-12036.002.patch, HADOOP-12036.004.patch, HADOOP-12036.005.patch Rather than have a half-dozen redefinitions, custom extensions, etc, we should move them all to one location so that the cmake environment is consistent between the various native components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-12138) hadoop personality needs to know about shellprofile.d
Allen Wittenauer created HADOOP-12138: - Summary: hadoop personality needs to know about shellprofile.d Key: HADOOP-12138 URL: https://issues.apache.org/jira/browse/HADOOP-12138 Project: Hadoop Common Issue Type: Sub-task Reporter: Allen Wittenauer Now that test-patch.sh allows projects to override the test determination code, hadoop's personality needs to have some logic for shellprofile.d added to fix the issue noticed in HADOOP-12023. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HADOOP-12023) test-patch.sh triggered full hdfs unit tests when it shouldn't
[ https://issues.apache.org/jira/browse/HADOOP-12023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HADOOP-12023. --- Resolution: Duplicate I'm going to close this as a dupe of HADOOP-12113 since the logic problem that caused this issue is now replaceable by the projects. I've opened HADOOP-12138 to fix the Hadoop-specific logic case. test-patch.sh triggered full hdfs unit tests when it shouldn't -- Key: HADOOP-12023 URL: https://issues.apache.org/jira/browse/HADOOP-12023 Project: Hadoop Common Issue Type: Test Components: test Reporter: Allen Wittenauer On HDFS-7991, the HDFS-7991-shellpart.patch triggered a full hdfs unit test run despite not having any java code. The determine test rules need to be verified to work correctly for patches like that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11977) test-patch.sh smarter javadoc
[ https://issues.apache.org/jira/browse/HADOOP-11977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-11977: -- Description: I suspect an optimization could be made that when a patch to java code doesn't touch either a method definition or a comment, that javadoc might not need to fire off. Additionally (was: I suspect an optimization could be made that when a patch to java code only touches a comment, the javac and unit test code shouldn't fire off.) test-patch.sh smarter javadoc - Key: HADOOP-11977 URL: https://issues.apache.org/jira/browse/HADOOP-11977 Project: Hadoop Common Issue Type: Test Reporter: Allen Wittenauer I suspect an optimization could be made that when a patch to java code doesn't touch either a method definition or a comment, that javadoc might not need to fire off. Additionally -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11977) test-patch.sh smarter javadoc
[ https://issues.apache.org/jira/browse/HADOOP-11977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-11977: -- Summary: test-patch.sh smarter javadoc (was: test-patch.sh should only run javadoc if comments are touched) test-patch.sh smarter javadoc - Key: HADOOP-11977 URL: https://issues.apache.org/jira/browse/HADOOP-11977 Project: Hadoop Common Issue Type: Test Reporter: Allen Wittenauer I suspect an optimization could be made that when a patch to java code only touches a comment, the javac and unit test code shouldn't fire off. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11977) test-patch.sh smarter java
[ https://issues.apache.org/jira/browse/HADOOP-11977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-11977: -- Description: I suspect an optimization could be made that when a patch to java code doesn't touch either a method definition or a comment, that javadoc might not need to fire off. Additionally, if a patch only touches a comment, then it should only do javadoc tests. (was: I suspect an optimization could be made that when a patch to java code doesn't touch either a method definition or a comment, that javadoc might not need to fire off. Additionally) test-patch.sh smarter java -- Key: HADOOP-11977 URL: https://issues.apache.org/jira/browse/HADOOP-11977 Project: Hadoop Common Issue Type: Test Reporter: Allen Wittenauer I suspect an optimization could be made that when a patch to java code doesn't touch either a method definition or a comment, that javadoc might not need to fire off. Additionally, if a patch only touches a comment, then it should only do javadoc tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11977) test-patch.sh smarter java
[ https://issues.apache.org/jira/browse/HADOOP-11977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-11977: -- Labels: yetus (was: ) test-patch.sh smarter java -- Key: HADOOP-11977 URL: https://issues.apache.org/jira/browse/HADOOP-11977 Project: Hadoop Common Issue Type: Sub-task Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer Labels: yetus I suspect an optimization could be made that when a patch to java code doesn't touch either a method definition or a comment, that javadoc might not need to fire off. Additionally, if a patch only touches a comment, then it should only do javadoc tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11977) test-patch.sh smarter java
[ https://issues.apache.org/jira/browse/HADOOP-11977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-11977: -- Issue Type: Sub-task (was: Test) Parent: HADOOP-12111 test-patch.sh smarter java -- Key: HADOOP-11977 URL: https://issues.apache.org/jira/browse/HADOOP-11977 Project: Hadoop Common Issue Type: Sub-task Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer Labels: yetus I suspect an optimization could be made that when a patch to java code doesn't touch either a method definition or a comment, that javadoc might not need to fire off. Additionally, if a patch only touches a comment, then it should only do javadoc tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11977) test-patch.sh smarter java
[ https://issues.apache.org/jira/browse/HADOOP-11977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-11977: -- Affects Version/s: HADOOP-12111 test-patch.sh smarter java -- Key: HADOOP-11977 URL: https://issues.apache.org/jira/browse/HADOOP-11977 Project: Hadoop Common Issue Type: Sub-task Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer Labels: yetus I suspect an optimization could be made that when a patch to java code doesn't touch either a method definition or a comment, that javadoc might not need to fire off. Additionally, if a patch only touches a comment, then it should only do javadoc tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11977) test-patch.sh smarter java
[ https://issues.apache.org/jira/browse/HADOOP-11977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-11977: -- Summary: test-patch.sh smarter java (was: test-patch.sh smarter javadoc) test-patch.sh smarter java -- Key: HADOOP-11977 URL: https://issues.apache.org/jira/browse/HADOOP-11977 Project: Hadoop Common Issue Type: Test Reporter: Allen Wittenauer I suspect an optimization could be made that when a patch to java code doesn't touch either a method definition or a comment, that javadoc might not need to fire off. Additionally -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12044) test-patch should handle module movement
[ https://issues.apache.org/jira/browse/HADOOP-12044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-12044: -- Affects Version/s: HADOOP-12111 test-patch should handle module movement Key: HADOOP-12044 URL: https://issues.apache.org/jira/browse/HADOOP-12044 Project: Hadoop Common Issue Type: Sub-task Components: test, yetus Affects Versions: HADOOP-12111 Reporter: Sean Busbey test-patch reported a variety of errors against HADOOP-11804 caused by the fact that a module moved -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12138) hadoop personality needs to know about shellprofile.d
[ https://issues.apache.org/jira/browse/HADOOP-12138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-12138: -- Affects Version/s: HADOOP-12111 hadoop personality needs to know about shellprofile.d - Key: HADOOP-12138 URL: https://issues.apache.org/jira/browse/HADOOP-12138 Project: Hadoop Common Issue Type: Sub-task Components: yetus Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer Now that test-patch.sh allows projects to override the test determination code, hadoop's personality needs to have some logic for shellprofile.d added to fix the issue noticed in HADOOP-12023. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12044) test-patch should handle module movement
[ https://issues.apache.org/jira/browse/HADOOP-12044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-12044: -- Component/s: yetus test-patch should handle module movement Key: HADOOP-12044 URL: https://issues.apache.org/jira/browse/HADOOP-12044 Project: Hadoop Common Issue Type: Sub-task Components: test, yetus Affects Versions: HADOOP-12111 Reporter: Sean Busbey test-patch reported a variety of errors against HADOOP-11804 caused by the fact that a module moved -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12137) build environment and unit tests
[ https://issues.apache.org/jira/browse/HADOOP-12137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-12137: -- Affects Version/s: HADOOP-12111 build environment and unit tests Key: HADOOP-12137 URL: https://issues.apache.org/jira/browse/HADOOP-12137 Project: Hadoop Common Issue Type: Sub-task Components: yetus Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer Priority: Critical We need to have some way to build (esp the documentation!) and unit tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-12139) patch to a unit test should only trigger that unit test
Allen Wittenauer created HADOOP-12139: - Summary: patch to a unit test should only trigger that unit test Key: HADOOP-12139 URL: https://issues.apache.org/jira/browse/HADOOP-12139 Project: Hadoop Common Issue Type: Sub-task Reporter: Allen Wittenauer A great optimization would be to determine if only a unit test was patched and run just that unit test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11909) test-patch javac javadoc output clarity
[ https://issues.apache.org/jira/browse/HADOOP-11909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-11909: -- Component/s: yetus test-patch javac javadoc output clarity - Key: HADOOP-11909 URL: https://issues.apache.org/jira/browse/HADOOP-11909 Project: Hadoop Common Issue Type: Sub-task Components: yetus Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer Assignee: Jayaradha Priority: Minor It should be possible to use similar code as is in the checkstyle plugin to just print the errors/warnings of relevant lines rather than the full diff. Additionally, when javac (and maybe javadoc too?) fails, it doesn't provide a link to the log file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11909) test-patch javac javadoc output clarity
[ https://issues.apache.org/jira/browse/HADOOP-11909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-11909: -- Affects Version/s: HADOOP-12111 test-patch javac javadoc output clarity - Key: HADOOP-11909 URL: https://issues.apache.org/jira/browse/HADOOP-11909 Project: Hadoop Common Issue Type: Sub-task Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer Assignee: Jayaradha Priority: Minor It should be possible to use similar code as is in the checkstyle plugin to just print the errors/warnings of relevant lines rather than the full diff. Additionally, when javac (and maybe javadoc too?) fails, it doesn't provide a link to the log file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11909) test-patch javac javadoc output clarity
[ https://issues.apache.org/jira/browse/HADOOP-11909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-11909: -- Issue Type: Sub-task (was: Test) Parent: HADOOP-12111 test-patch javac javadoc output clarity - Key: HADOOP-11909 URL: https://issues.apache.org/jira/browse/HADOOP-11909 Project: Hadoop Common Issue Type: Sub-task Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer Assignee: Jayaradha Priority: Minor It should be possible to use similar code as is in the checkstyle plugin to just print the errors/warnings of relevant lines rather than the full diff. Additionally, when javac (and maybe javadoc too?) fails, it doesn't provide a link to the log file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11914) test-patch.sh confused by certain patch formats
[
https://issues.apache.org/jira/browse/HADOOP-11914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer updated HADOOP-11914:
--
Issue Type: Sub-task (was: Bug)
Parent: HADOOP-12111
test-patch.sh confused by certain patch formats
---
Key: HADOOP-11914
URL: https://issues.apache.org/jira/browse/HADOOP-11914
Project: Hadoop Common
Issue Type: Sub-task
Components: yetus
Affects Versions: HADOOP-12111
Reporter: Allen Wittenauer
Assignee: Kengo Seki
Priority: Critical
Attachments: HADOOP-11914.001.patch, HADOOP-11914.002.patch
A simple patch example:
{code}
diff --git a/start-build-env.sh b/start-build-env.sh
old mode 100644
new mode 100755
{code}
start-build-env.sh will not show up in the changed files list and therefore
will not get run by shellcheck.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11914) test-patch.sh confused by certain patch formats
[
https://issues.apache.org/jira/browse/HADOOP-11914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer updated HADOOP-11914:
--
Affects Version/s: HADOOP-12111
test-patch.sh confused by certain patch formats
---
Key: HADOOP-11914
URL: https://issues.apache.org/jira/browse/HADOOP-11914
Project: Hadoop Common
Issue Type: Sub-task
Components: yetus
Affects Versions: HADOOP-12111
Reporter: Allen Wittenauer
Assignee: Kengo Seki
Priority: Critical
Attachments: HADOOP-11914.001.patch, HADOOP-11914.002.patch
A simple patch example:
{code}
diff --git a/start-build-env.sh b/start-build-env.sh
old mode 100644
new mode 100755
{code}
start-build-env.sh will not show up in the changed files list and therefore
will not get run by shellcheck.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11914) test-patch.sh confused by certain patch formats
[
https://issues.apache.org/jira/browse/HADOOP-11914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer updated HADOOP-11914:
--
Component/s: yetus
test-patch.sh confused by certain patch formats
---
Key: HADOOP-11914
URL: https://issues.apache.org/jira/browse/HADOOP-11914
Project: Hadoop Common
Issue Type: Sub-task
Components: yetus
Affects Versions: HADOOP-12111
Reporter: Allen Wittenauer
Assignee: Kengo Seki
Priority: Critical
Attachments: HADOOP-11914.001.patch, HADOOP-11914.002.patch
A simple patch example:
{code}
diff --git a/start-build-env.sh b/start-build-env.sh
old mode 100644
new mode 100755
{code}
start-build-env.sh will not show up in the changed files list and therefore
will not get run by shellcheck.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HADOOP-11956) test-patch.sh's git ref support is fragile
[ https://issues.apache.org/jira/browse/HADOOP-11956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HADOOP-11956. --- Resolution: Incomplete test-patch.sh's git ref support is fragile -- Key: HADOOP-11956 URL: https://issues.apache.org/jira/browse/HADOOP-11956 Project: Hadoop Common Issue Type: Improvement Reporter: Allen Wittenauer Assignee: Allen Wittenauer Using a git ref breaks in various places. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11903) test-patch should fail any new classes called Default-foo
[ https://issues.apache.org/jira/browse/HADOOP-11903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-11903: -- Issue Type: Sub-task (was: Test) Parent: HADOOP-12111 test-patch should fail any new classes called Default-foo - Key: HADOOP-11903 URL: https://issues.apache.org/jira/browse/HADOOP-11903 Project: Hadoop Common Issue Type: Sub-task Components: yetus Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer In the past, we've named things like DefaultResourceCalculator, DefaultContainerExecutor, and DefaultCodec that do nothing but cause problems down the road since they are effectively version and functionality locked forever. If these examples had been named what they truly were (e.g., MemoryResourceCalculator, SimpleContainerExecutor, and GZipCodec), the defaults could then be changed in the future in a compatible way. One way to enforce this is to prevent the creation of new classes called Default-anything. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11903) test-patch should fail any new classes called Default-foo
[ https://issues.apache.org/jira/browse/HADOOP-11903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-11903: -- Affects Version/s: HADOOP-12111 test-patch should fail any new classes called Default-foo - Key: HADOOP-11903 URL: https://issues.apache.org/jira/browse/HADOOP-11903 Project: Hadoop Common Issue Type: Sub-task Components: yetus Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer In the past, we've named things like DefaultResourceCalculator, DefaultContainerExecutor, and DefaultCodec that do nothing but cause problems down the road since they are effectively version and functionality locked forever. If these examples had been named what they truly were (e.g., MemoryResourceCalculator, SimpleContainerExecutor, and GZipCodec), the defaults could then be changed in the future in a compatible way. One way to enforce this is to prevent the creation of new classes called Default-anything. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HADOOP-7334) test-patch should check for hard tabs
[ https://issues.apache.org/jira/browse/HADOOP-7334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HADOOP-7334. -- Resolution: Implemented Fix Version/s: HADOOP-12111 tab support now in yetus. closing as implemented. test-patch should check for hard tabs - Key: HADOOP-7334 URL: https://issues.apache.org/jira/browse/HADOOP-7334 Project: Hadoop Common Issue Type: Test Components: build, test Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: HADOOP-12111 Attachments: hadoop-7334.txt, hadoop-7334.txt Our coding guidelines say that hard tabs are disallowed in the Hadoop code, but they sometimes sneak in (there are about 280 in the common codebase at the moment). We should run a simple check for this in the test-patch process so it's harder for them to sneak in. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-9112) test-patch should -1 for @Tests without a timeout
[ https://issues.apache.org/jira/browse/HADOOP-9112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-9112: - Assignee: (was: Surenkumar Nihalani) test-patch should -1 for @Tests without a timeout - Key: HADOOP-9112 URL: https://issues.apache.org/jira/browse/HADOOP-9112 Project: Hadoop Common Issue Type: Sub-task Components: build Reporter: Todd Lipcon Attachments: HADOOP-9112-1.patch, HADOOP-9112-2.patch, HADOOP-9112-3.patch, HADOOP-9112-4.patch, HADOOP-9112-5.patch, HADOOP-9112-6.patch, HADOOP-9112-7.patch With our current test running infrastructure, if a test with no timeout set runs too long, it triggers a surefire-wide timeout, which for some reason doesn't show up as a failed test in the test-patch output. Given that, we should require that all tests have a timeout set, and have test-patch enforce this with a simple check -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-9112) test-patch should -1 for @Tests without a timeout
[ https://issues.apache.org/jira/browse/HADOOP-9112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-9112: - Issue Type: Sub-task (was: Test) Parent: HADOOP-12111 test-patch should -1 for @Tests without a timeout - Key: HADOOP-9112 URL: https://issues.apache.org/jira/browse/HADOOP-9112 Project: Hadoop Common Issue Type: Sub-task Components: build Reporter: Todd Lipcon Assignee: Surenkumar Nihalani Attachments: HADOOP-9112-1.patch, HADOOP-9112-2.patch, HADOOP-9112-3.patch, HADOOP-9112-4.patch, HADOOP-9112-5.patch, HADOOP-9112-6.patch, HADOOP-9112-7.patch With our current test running infrastructure, if a test with no timeout set runs too long, it triggers a surefire-wide timeout, which for some reason doesn't show up as a failed test in the test-patch output. Given that, we should require that all tests have a timeout set, and have test-patch enforce this with a simple check -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-9112) test-patch should -1 for @Tests without a timeout
[ https://issues.apache.org/jira/browse/HADOOP-9112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604280#comment-14604280 ] Allen Wittenauer commented on HADOOP-9112: -- Moving this to be a subtask under yetus. This is something we should build a plug-in for as part of the standard bits. test-patch should -1 for @Tests without a timeout - Key: HADOOP-9112 URL: https://issues.apache.org/jira/browse/HADOOP-9112 Project: Hadoop Common Issue Type: Sub-task Components: build Reporter: Todd Lipcon Attachments: HADOOP-9112-1.patch, HADOOP-9112-2.patch, HADOOP-9112-3.patch, HADOOP-9112-4.patch, HADOOP-9112-5.patch, HADOOP-9112-6.patch, HADOOP-9112-7.patch With our current test running infrastructure, if a test with no timeout set runs too long, it triggers a surefire-wide timeout, which for some reason doesn't show up as a failed test in the test-patch output. Given that, we should require that all tests have a timeout set, and have test-patch enforce this with a simple check -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HADOOP-8964) test-patch.sh doesn't run all tests
[ https://issues.apache.org/jira/browse/HADOOP-8964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HADOOP-8964. -- Resolution: Won't Fix at this point, i'm closing this as won't fix. the nightly runs pretty much for the purpose of doing a full test run and should really be used for this type of intense testing. (for example, hadoop-tools is particularly problematic since it would require everything to get built!) test-patch.sh doesn't run all tests --- Key: HADOOP-8964 URL: https://issues.apache.org/jira/browse/HADOOP-8964 Project: Hadoop Common Issue Type: Test Components: build Reporter: Vinod Kumar Vavilapalli Assignee: Vinod Kumar Vavilapalli test-patch.sh tries to be smart but with bad results. When it runs tests, say on each commit, it only runs tests only from modules which have corresponding changes. This is a cause of concern for downstream modules. We ran into multiple issues in the past because of this, the most recent of it being YARN-179. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12133) Add schemas to Maven Assembly XMLs
[
https://issues.apache.org/jira/browse/HADOOP-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604283#comment-14604283
]
Hadoop QA commented on HADOOP-12133:
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 39s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 29s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 39s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 21s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 34s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 36s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | assemblies tests | 0m 11s | Tests passed in
hadoop-assemblies. |
| {color:green}+1{color} | tools/hadoop tests | 0m 52s | Tests passed in
hadoop-sls. |
| | | 35m 25s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12742336/HADOOP-12133.2.patch |
| Optional Tests | javadoc javac unit |
| git revision | trunk / 79ed0f9 |
| hadoop-assemblies test log |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7074/artifact/patchprocess/testrun_hadoop-assemblies.txt
|
| hadoop-sls test log |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7074/artifact/patchprocess/testrun_hadoop-sls.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7074/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7074/console |
This message was automatically generated.
Add schemas to Maven Assembly XMLs
--
Key: HADOOP-12133
URL: https://issues.apache.org/jira/browse/HADOOP-12133
Project: Hadoop Common
Issue Type: Improvement
Reporter: Gabor Liptak
Assignee: Gabor Liptak
Priority: Minor
Attachments: HADOOP-12133.1.patch, HADOOP-12133.2.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12118) Validate xml configuration files with XML Schema
[
https://issues.apache.org/jira/browse/HADOOP-12118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604292#comment-14604292
]
Gabor Liptak commented on HADOOP-12118:
---
[~sekikn] Attached hadoop-configuration.xsd validates (most of) configuration
xml-s in current tree:
$ xmllint --schema hadoop-configuration.xsd *.xml
The one thing unclear if value/ is required ...
Validate xml configuration files with XML Schema
Key: HADOOP-12118
URL: https://issues.apache.org/jira/browse/HADOOP-12118
Project: Hadoop Common
Issue Type: Improvement
Reporter: Christopher Tubbs
Attachments: HADOOP-7947.branch-2.1.patch
I spent an embarrassingly long time today trying to figure out why the
following wouldn't work.
{code}
property
keyfs.defaultFS/key
valuehdfs://localhost:9000/value
/property
{code}
I just kept getting an error about no authority for {{fs.defaultFS}}, with a
value of {{file:///}}, which made no sense... because I knew it was there.
The problem was that the {{core-site.xml}} was parsed entirely without any
validation. This seems incorrect. The very least that could be done is a
simple XML Schema validation against an XSD, before parsing. That way, users
will get immediate failures on common typos and other problems in the xml
configuration files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12118) Validate xml configuration files with XML Schema
[
https://issues.apache.org/jira/browse/HADOOP-12118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Liptak updated HADOOP-12118:
--
Attachment: hadoop-configuration.xsd
Validate xml configuration files with XML Schema
Key: HADOOP-12118
URL: https://issues.apache.org/jira/browse/HADOOP-12118
Project: Hadoop Common
Issue Type: Improvement
Reporter: Christopher Tubbs
Attachments: HADOOP-7947.branch-2.1.patch, hadoop-configuration.xsd
I spent an embarrassingly long time today trying to figure out why the
following wouldn't work.
{code}
property
keyfs.defaultFS/key
valuehdfs://localhost:9000/value
/property
{code}
I just kept getting an error about no authority for {{fs.defaultFS}}, with a
value of {{file:///}}, which made no sense... because I knew it was there.
The problem was that the {{core-site.xml}} was parsed entirely without any
validation. This seems incorrect. The very least that could be done is a
simple XML Schema validation against an XSD, before parsing. That way, users
will get immediate failures on common typos and other problems in the xml
configuration files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HADOOP-12132) test-patch needs Git 1.7.3+
Kengo Seki created HADOOP-12132: --- Summary: test-patch needs Git 1.7.3+ Key: HADOOP-12132 URL: https://issues.apache.org/jira/browse/HADOOP-12132 Project: Hadoop Common Issue Type: Sub-task Components: yetus Affects Versions: HADOOP-12111 Reporter: Kengo Seki test-patch can run `git clean -xdf -e ...`, but -e is supported only 1.7.3+. Version checking and documentation are needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12133) Add schemas to Maven Assembly XMLs
[ https://issues.apache.org/jira/browse/HADOOP-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Liptak updated HADOOP-12133: -- Release Note: HADOOP-12133 Add schemas to Maven Assembly XMLs Status: Patch Available (was: Open) Add schemas to Maven Assembly XMLs -- Key: HADOOP-12133 URL: https://issues.apache.org/jira/browse/HADOOP-12133 Project: Hadoop Common Issue Type: Improvement Reporter: Gabor Liptak Assignee: Gabor Liptak Priority: Minor Attachments: HADOOP-12133.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12133) Add schemas to Maven Assembly XMLs
[ https://issues.apache.org/jira/browse/HADOOP-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Liptak updated HADOOP-12133: -- Status: Open (was: Patch Available) Add schemas to Maven Assembly XMLs -- Key: HADOOP-12133 URL: https://issues.apache.org/jira/browse/HADOOP-12133 Project: Hadoop Common Issue Type: Improvement Reporter: Gabor Liptak Assignee: Gabor Liptak Priority: Minor Attachments: HADOOP-12133.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12036) Consolidate all of the cmake extensions in one directory
[ https://issues.apache.org/jira/browse/HADOOP-12036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604192#comment-14604192 ] Hudson commented on HADOOP-12036: - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #239 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/239/]) HADOOP-12036. Consolidate all of the cmake extensions in one directory (alanburlison via cmccabe) (cmccabe: rev aa07dea3577158b92a17651d10da20df73f54561) * hadoop-common-project/hadoop-common/src/CMakeLists.txt * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-common-project/hadoop-common/HadoopJNI.cmake * hadoop-common-project/hadoop-common/HadoopCommon.cmake Consolidate all of the cmake extensions in one directory Key: HADOOP-12036 URL: https://issues.apache.org/jira/browse/HADOOP-12036 Project: Hadoop Common Issue Type: Sub-task Reporter: Allen Wittenauer Assignee: Alan Burlison Fix For: 2.8.0 Attachments: HADOOP-12036.001.patch, HADOOP-12036.002.patch, HADOOP-12036.004.patch, HADOOP-12036.005.patch Rather than have a half-dozen redefinitions, custom extensions, etc, we should move them all to one location so that the cmake environment is consistent between the various native components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11807) add a lint mode to releasedocmaker
[ https://issues.apache.org/jira/browse/HADOOP-11807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-11807: -- Resolution: Fixed Fix Version/s: HADOOP-12111 3.0.0 Status: Resolved (was: Patch Available) +1 committed to trunk and to HADOOP-12111 branch. Thanks! add a lint mode to releasedocmaker -- Key: HADOOP-11807 URL: https://issues.apache.org/jira/browse/HADOOP-11807 Project: Hadoop Common Issue Type: Improvement Components: build, documentation, yetus Affects Versions: 3.0.0 Reporter: Allen Wittenauer Assignee: ramtin Priority: Minor Fix For: 3.0.0, HADOOP-12111 Attachments: HADOOP-11807.001.patch, HADOOP-11807.002.patch, HADOOP-11807.003.patch, HADOOP-11807.004.patch, HADOOP-11807.005.patch * check for missing components (error) * check for missing assignee (error) * check for common version problems (warning) * add an error message for missing release notes -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12135) releasedocmaker needs to be de-hadoop-ified
[ https://issues.apache.org/jira/browse/HADOOP-12135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-12135: -- Assignee: (was: Allen Wittenauer) releasedocmaker needs to be de-hadoop-ified --- Key: HADOOP-12135 URL: https://issues.apache.org/jira/browse/HADOOP-12135 Project: Hadoop Common Issue Type: Sub-task Components: yetus Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer Currently, releasedocmaker is hard-coded to Hadoop. It should be made more generic to support other projects. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12118) Validate xml configuration files with XML Schema
[
https://issues.apache.org/jira/browse/HADOOP-12118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604218#comment-14604218
]
Kengo Seki commented on HADOOP-12118:
-
Thanks [~gliptak]. In more detail, I couldn't represent the following
conditions as XSD:
bq. * same attributes and/or elements are duplicately defined in property
bq. * duplicated propertys with the same name value exist
I also couldn't represent the rule of property's children, say:
* name and value are needed and occurs only once.
* Other elements (like source or description) can exist.
* Their order is arbitrary.
Validate xml configuration files with XML Schema
Key: HADOOP-12118
URL: https://issues.apache.org/jira/browse/HADOOP-12118
Project: Hadoop Common
Issue Type: Improvement
Reporter: Christopher Tubbs
Attachments: HADOOP-7947.branch-2.1.patch
I spent an embarrassingly long time today trying to figure out why the
following wouldn't work.
{code}
property
keyfs.defaultFS/key
valuehdfs://localhost:9000/value
/property
{code}
I just kept getting an error about no authority for {{fs.defaultFS}}, with a
value of {{file:///}}, which made no sense... because I knew it was there.
The problem was that the {{core-site.xml}} was parsed entirely without any
validation. This seems incorrect. The very least that could be done is a
simple XML Schema validation against an XSD, before parsing. That way, users
will get immediate failures on common typos and other problems in the xml
configuration files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HADOOP-12136) Add schemas to site.xml-s
Gabor Liptak created HADOOP-12136: - Summary: Add schemas to site.xml-s Key: HADOOP-12136 URL: https://issues.apache.org/jira/browse/HADOOP-12136 Project: Hadoop Common Issue Type: Improvement Reporter: Gabor Liptak Assignee: Gabor Liptak Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12135) cleanup releasedocmaker
[ https://issues.apache.org/jira/browse/HADOOP-12135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-12135: -- Summary: cleanup releasedocmaker (was: releasedocmaker needs to be de-hadoop-ified) cleanup releasedocmaker --- Key: HADOOP-12135 URL: https://issues.apache.org/jira/browse/HADOOP-12135 Project: Hadoop Common Issue Type: Sub-task Components: yetus Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer Currently, releasedocmaker is hard-coded to Hadoop. It should be made more generic to support other projects. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12131) The different JDKs which versions are the same aren't distiguished on multi-JDK mode
[
https://issues.apache.org/jira/browse/HADOOP-12131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604225#comment-14604225
]
Kengo Seki commented on HADOOP-12131:
-
A possible solution is simply numbering each JDK uniquely. Output image is:
{code}
| Vote | Subsystem | Runtime | Comment
| +1 |mvninstall | 12m 25s | trunk passed
| +1 | javac | 1m 46s| trunk passed with JDK v1.7.0_79 (1)
| +1 | javac | 1m 44s| trunk passed with JDK v1.7.0_80 (2)
| +1 | javac | 1m 46s| trunk passed with JDK v1.7.0_79 (3)
| +1 | javadoc | 2m 17s| trunk passed with JDK v1.7.0_79 (1)
| +1 | javadoc | 2m 32s| trunk passed with JDK v1.7.0_80 (2)
| +1 | javadoc | 2m 17s| trunk passed with JDK v1.7.0_79 (3)
| +1 | javac | 1m 41s| the patch passed with JDK v1.7.0_79 (1)
| +1 | javac | 1m 43s| the patch passed with JDK v1.7.0_80 (2)
| +1 | javac | 1m 41s| the patch passed with JDK v1.7.0_79 (3)
| +1 | javadoc | 2m 10s| the patch passed with JDK v1.7.0_79 (1)
| +1 | javadoc | 2m 14s| the patch passed with JDK v1.7.0_80 (2)
| +1 | javadoc | 2m 10s| the patch passed with JDK v1.7.0_79 (3)
| -1 | unit | 19m 58s | hadoop-common in the patch failed with
| ||| JDK v1.7.0_79 (1).
| -1 | unit | 27m 14s | hadoop-common in the patch failed with
| ||| JDK v1.7.0_80 (2).
| -1 | unit | 19m 58s | hadoop-common in the patch failed with
| ||| JDK v1.7.0_79 (3).
Reason | Tests
JDK v1.7.0_79 (1) Failed unit tests | hadoop.crypto.TestCryptoCodec
JDK v1.7.0_80 (2) Failed unit tests | hadoop.crypto.TestCryptoCodec
JDK v1.7.0_79 (3) Failed unit tests | hadoop.crypto.TestCryptoCodec
|| Subsystem || Report/Notes ||
| Default Java | (3) 1.7.0_79 |
| Multi-JDK versions | (1) /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.79.x86_64 |
|| (2)
/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.45-28.b13.el6_6.x86 |
|| (3) /usr/java/jdk1.7.0_79 |
| unit |
/private/tmp/test-patch-hadoop/43110/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_79_1.txt
|
| unit |
/private/tmp/test-patch-hadoop/43110/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_80_2.txt
|
| unit |
/private/tmp/test-patch-hadoop/43110/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_79_3.txt
|
{code}
The different JDKs which versions are the same aren't distiguished on
multi-JDK mode
Key: HADOOP-12131
URL: https://issues.apache.org/jira/browse/HADOOP-12131
Project: Hadoop Common
Issue Type: Sub-task
Components: yetus
Affects Versions: HADOOP-12111
Reporter: Kengo Seki
If one installs the same version of Oracle JDK and OpenJDK and enables the
multi-JDK option, output files are overwritten by the last one because the
same value is assigned to $jdk and $fn. Example:
{code}
Pre-patch trunk javac compilation
mvn --batch-mode clean compile -DskipTests -Pnative -Drequire.libwebhdfs
-Drequire.snappy -Drequire.openssl -Drequire.fuse -Drequire.test.libhadoop
-Ptest-patch
/tmp/test-patch-hadoop/25579/branch-javac-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt
21
Elapsed: 1m 0s
mvn --batch-mode clean compile -DskipTests -Pnative -Drequire.libwebhdfs
-Drequire.snappy -Drequire.openssl -Drequire.fuse -Drequire.test.libhadoop
-Ptest-patch
/tmp/test-patch-hadoop/25579/branch-javac-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt
21
Elapsed: 0m 54s
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HADOOP-12137) build environment and unit tests
Allen Wittenauer created HADOOP-12137: - Summary: build environment and unit tests Key: HADOOP-12137 URL: https://issues.apache.org/jira/browse/HADOOP-12137 Project: Hadoop Common Issue Type: Sub-task Reporter: Allen Wittenauer Priority: Critical We need to have some way to build (esp the documentation!) and unit tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11791) Update src/site/markdown/releases to include old versions of Hadoop
[ https://issues.apache.org/jira/browse/HADOOP-11791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-11791: -- Labels: (was: BB2015-05-TBR) Update src/site/markdown/releases to include old versions of Hadoop --- Key: HADOOP-11791 URL: https://issues.apache.org/jira/browse/HADOOP-11791 Project: Hadoop Common Issue Type: Task Components: build, documentation Affects Versions: 3.0.0 Reporter: Allen Wittenauer Attachments: HADOOP-11791.001.patch With the commit of HADOOP-11731, we need to include the new format of release information in trunk. This JIRA is about including those old versions in the tree. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12136) Add schemas to site.xml-s
[
https://issues.apache.org/jira/browse/HADOOP-12136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604241#comment-14604241
]
Hadoop QA commented on HADOOP-12136:
\\
\\
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 2m 56s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | release audit | 0m 20s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | site | 2m 59s | Site still builds. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| | | 6m 18s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12742333/HADOOP-12136.1.patch |
| Optional Tests | site |
| git revision | trunk / 79ed0f9 |
| Java | 1.7.0_55 |
| uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7072/console |
This message was automatically generated.
Add schemas to site.xml-s
-
Key: HADOOP-12136
URL: https://issues.apache.org/jira/browse/HADOOP-12136
Project: Hadoop Common
Issue Type: Improvement
Reporter: Gabor Liptak
Assignee: Gabor Liptak
Priority: Minor
Attachments: HADOOP-12136.1.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11791) Update src/site/markdown/releases to include old versions of Hadoop
[
https://issues.apache.org/jira/browse/HADOOP-11791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604245#comment-14604245
]
Hadoop QA commented on HADOOP-11791:
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 2m 58s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 1s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | release audit | 0m 20s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | site | 0m 54s | Site compilation is broken. |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| | | 4m 20s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12709071/HADOOP-11791.001.patch
|
| Optional Tests | site |
| git revision | trunk / 79ed0f9 |
| site |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7073/artifact/patchprocess/patchSiteWarnings.txt
|
| Java | 1.7.0_55 |
| uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7073/console |
This message was automatically generated.
Update src/site/markdown/releases to include old versions of Hadoop
---
Key: HADOOP-11791
URL: https://issues.apache.org/jira/browse/HADOOP-11791
Project: Hadoop Common
Issue Type: Task
Components: build, documentation
Affects Versions: 3.0.0
Reporter: Allen Wittenauer
Attachments: HADOOP-11791.001.patch
With the commit of HADOOP-11731, we need to include the new format of release
information in trunk. This JIRA is about including those old versions in the
tree.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12044) test-patch should handle module movement
[ https://issues.apache.org/jira/browse/HADOOP-12044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-12044: -- Issue Type: Sub-task (was: Test) Parent: HADOOP-12111 test-patch should handle module movement Key: HADOOP-12044 URL: https://issues.apache.org/jira/browse/HADOOP-12044 Project: Hadoop Common Issue Type: Sub-task Components: test Reporter: Sean Busbey test-patch reported a variety of errors against HADOOP-11804 caused by the fact that a module moved -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12044) test-patch should handle module movement
[ https://issues.apache.org/jira/browse/HADOOP-12044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604249#comment-14604249 ] Allen Wittenauer commented on HADOOP-12044: --- bq. Also makes me wonder if we're handling the addition of new modules correctly. This, BTW, has been dealt with as of the rewrite. test-patch should handle module movement Key: HADOOP-12044 URL: https://issues.apache.org/jira/browse/HADOOP-12044 Project: Hadoop Common Issue Type: Sub-task Components: test Reporter: Sean Busbey test-patch reported a variety of errors against HADOOP-11804 caused by the fact that a module moved -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12132) test-patch needs Git 1.7.3+
[ https://issues.apache.org/jira/browse/HADOOP-12132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kengo Seki updated HADOOP-12132: Priority: Minor (was: Major) test-patch needs Git 1.7.3+ --- Key: HADOOP-12132 URL: https://issues.apache.org/jira/browse/HADOOP-12132 Project: Hadoop Common Issue Type: Sub-task Components: yetus Affects Versions: HADOOP-12111 Reporter: Kengo Seki Priority: Minor test-patch can run `git clean -xdf -e ...`, but -e is supported only 1.7.3+. Version checking and documentation are needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-12133) Add schemas to Maven Assembly XMLs
Gabor Liptak created HADOOP-12133: - Summary: Add schemas to Maven Assembly XMLs Key: HADOOP-12133 URL: https://issues.apache.org/jira/browse/HADOOP-12133 Project: Hadoop Common Issue Type: Improvement Reporter: Gabor Liptak Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12133) Add schemas to Maven Assembly XMLs
[ https://issues.apache.org/jira/browse/HADOOP-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Liptak updated HADOOP-12133: -- Assignee: Gabor Liptak Status: Patch Available (was: Open) Add schemas to Maven Assembly XMLs -- Key: HADOOP-12133 URL: https://issues.apache.org/jira/browse/HADOOP-12133 Project: Hadoop Common Issue Type: Improvement Reporter: Gabor Liptak Assignee: Gabor Liptak Priority: Minor Attachments: HADOOP-12133.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12133) Add schemas to Maven Assembly XMLs
[ https://issues.apache.org/jira/browse/HADOOP-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Liptak updated HADOOP-12133: -- Attachment: HADOOP-12133.1.patch Add schemas to Maven Assembly XMLs -- Key: HADOOP-12133 URL: https://issues.apache.org/jira/browse/HADOOP-12133 Project: Hadoop Common Issue Type: Improvement Reporter: Gabor Liptak Priority: Minor Attachments: HADOOP-12133.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-12134) Pig personality always fails at precheck_javac and check_patch_javac
Kengo Seki created HADOOP-12134:
---
Summary: Pig personality always fails at precheck_javac and
check_patch_javac
Key: HADOOP-12134
URL: https://issues.apache.org/jira/browse/HADOOP-12134
Project: Hadoop Common
Issue Type: Sub-task
Components: yetus
Affects Versions: HADOOP-12111
Reporter: Kengo Seki
Currently, pig personality always fails at precheck_javac and check_patch_javac
by the following error:
{code}
forrest.check:
BUILD FAILED
/Users/sekikn/pig/build.xml:648: 'forrest.home' is not defined. Please
pass -Dforrest.home=base of Apache Forrest installation to Ant on the
command-line.
{code}
This is because tar target depends on docs via package. But publishing
documents isn't needed in javac phase. Probably piggybank target is suitable
for the purpose of this phase. It kicks jar target also.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12134) Pig personality always fails at precheck_javac and check_patch_javac
[
https://issues.apache.org/jira/browse/HADOOP-12134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604198#comment-14604198
]
Kengo Seki commented on HADOOP-12134:
-
Though I filed this JIRA, the personality files should be modified by the
respective projects in principle.
Pig personality always fails at precheck_javac and check_patch_javac
Key: HADOOP-12134
URL: https://issues.apache.org/jira/browse/HADOOP-12134
Project: Hadoop Common
Issue Type: Sub-task
Components: yetus
Affects Versions: HADOOP-12111
Reporter: Kengo Seki
Currently, pig personality always fails at precheck_javac and
check_patch_javac by the following error:
{code}
forrest.check:
BUILD FAILED
/Users/sekikn/pig/build.xml:648: 'forrest.home' is not defined. Please
pass -Dforrest.home=base of Apache Forrest installation to Ant on the
command-line.
{code}
This is because tar target depends on docs via package. But publishing
documents isn't needed in javac phase. Probably piggybank target is
suitable for the purpose of this phase. It kicks jar target also.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12036) Consolidate all of the cmake extensions in one directory
[ https://issues.apache.org/jira/browse/HADOOP-12036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604201#comment-14604201 ] Hudson commented on HADOOP-12036: - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2187 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2187/]) HADOOP-12036. Consolidate all of the cmake extensions in one directory (alanburlison via cmccabe) (cmccabe: rev aa07dea3577158b92a17651d10da20df73f54561) * hadoop-common-project/hadoop-common/HadoopCommon.cmake * hadoop-common-project/hadoop-common/HadoopJNI.cmake * hadoop-common-project/hadoop-common/src/CMakeLists.txt * hadoop-common-project/hadoop-common/CHANGES.txt Consolidate all of the cmake extensions in one directory Key: HADOOP-12036 URL: https://issues.apache.org/jira/browse/HADOOP-12036 Project: Hadoop Common Issue Type: Sub-task Reporter: Allen Wittenauer Assignee: Alan Burlison Fix For: 2.8.0 Attachments: HADOOP-12036.001.patch, HADOOP-12036.002.patch, HADOOP-12036.004.patch, HADOOP-12036.005.patch Rather than have a half-dozen redefinitions, custom extensions, etc, we should move them all to one location so that the cmake environment is consistent between the various native components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12133) Add schemas to Maven Assembly XMLs
[
https://issues.apache.org/jira/browse/HADOOP-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604211#comment-14604211
]
Hadoop QA commented on HADOOP-12133:
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 36s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 31s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 42s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | assemblies tests | 0m 11s | Tests passed in
hadoop-assemblies. |
| {color:green}+1{color} | tools/hadoop tests | 0m 51s | Tests passed in
hadoop-sls. |
| | | 35m 24s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12742325/HADOOP-12133.1.patch |
| Optional Tests | javadoc javac unit |
| git revision | trunk / fe6c1bd |
| hadoop-assemblies test log |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7071/artifact/patchprocess/testrun_hadoop-assemblies.txt
|
| hadoop-sls test log |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7071/artifact/patchprocess/testrun_hadoop-sls.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7071/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7071/console |
This message was automatically generated.
Add schemas to Maven Assembly XMLs
--
Key: HADOOP-12133
URL: https://issues.apache.org/jira/browse/HADOOP-12133
Project: Hadoop Common
Issue Type: Improvement
Reporter: Gabor Liptak
Assignee: Gabor Liptak
Priority: Minor
Attachments: HADOOP-12133.1.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11807) add a lint mode to releasedocmaker
[ https://issues.apache.org/jira/browse/HADOOP-11807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604216#comment-14604216 ] Hudson commented on HADOOP-11807: - FAILURE: Integrated in Hadoop-trunk-Commit #8077 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8077/]) HADOOP-11807. add a lint mode to releasedocmaker (ramtin via aw) (aw: rev 79ed0f959ffc490414ca56a73e026500c24e7078) * hadoop-common-project/hadoop-common/CHANGES.txt * dev-support/releasedocmaker.py add a lint mode to releasedocmaker -- Key: HADOOP-11807 URL: https://issues.apache.org/jira/browse/HADOOP-11807 Project: Hadoop Common Issue Type: Improvement Components: build, documentation, yetus Affects Versions: 3.0.0 Reporter: Allen Wittenauer Assignee: ramtin Priority: Minor Fix For: 3.0.0, HADOOP-12111 Attachments: HADOOP-11807.001.patch, HADOOP-11807.002.patch, HADOOP-11807.003.patch, HADOOP-11807.004.patch, HADOOP-11807.005.patch * check for missing components (error) * check for missing assignee (error) * check for common version problems (warning) * add an error message for missing release notes -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-12135) releasedocmaker needs to be de-hadoop-ified
Allen Wittenauer created HADOOP-12135: - Summary: releasedocmaker needs to be de-hadoop-ified Key: HADOOP-12135 URL: https://issues.apache.org/jira/browse/HADOOP-12135 Project: Hadoop Common Issue Type: Sub-task Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer Currently, releasedocmaker is hard-coded to Hadoop. It should be made more generic to support other projects. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HADOOP-12135) releasedocmaker needs to be de-hadoop-ified
[ https://issues.apache.org/jira/browse/HADOOP-12135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer reassigned HADOOP-12135: - Assignee: Allen Wittenauer releasedocmaker needs to be de-hadoop-ified --- Key: HADOOP-12135 URL: https://issues.apache.org/jira/browse/HADOOP-12135 Project: Hadoop Common Issue Type: Sub-task Components: yetus Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer Assignee: Allen Wittenauer Currently, releasedocmaker is hard-coded to Hadoop. It should be made more generic to support other projects. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12135) cleanup releasedocmaker
[ https://issues.apache.org/jira/browse/HADOOP-12135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604222#comment-14604222 ] Allen Wittenauer commented on HADOOP-12135: --- Most important would be to fix my inability to write python. ;) cleanup releasedocmaker --- Key: HADOOP-12135 URL: https://issues.apache.org/jira/browse/HADOOP-12135 Project: Hadoop Common Issue Type: Sub-task Components: yetus Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer For Yetus, releasedocmaker needs some work: * de-hadoop-ify it * still allow it to work w/4+ different projects simultaneously * just running it w/out any options should provide more help I think probably other things too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12135) cleanup releasedocmaker
[ https://issues.apache.org/jira/browse/HADOOP-12135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-12135: -- Description: For Yetus, releasedocmaker needs some work: * de-hadoop-ify it * still allow it to work w/4+ different projects simultaneously * just running it w/out any options should provide more help I think probably other things too. was:Currently, releasedocmaker is hard-coded to Hadoop. It should be made more generic to support other projects. cleanup releasedocmaker --- Key: HADOOP-12135 URL: https://issues.apache.org/jira/browse/HADOOP-12135 Project: Hadoop Common Issue Type: Sub-task Components: yetus Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer For Yetus, releasedocmaker needs some work: * de-hadoop-ify it * still allow it to work w/4+ different projects simultaneously * just running it w/out any options should provide more help I think probably other things too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12137) build environment and unit tests
[ https://issues.apache.org/jira/browse/HADOOP-12137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604228#comment-14604228 ] Allen Wittenauer commented on HADOOP-12137: --- For the short term, it might be useful to: * keep using maven * move yetus out of dev-support/ and into a different module * figure out how to use bats and tap4j+junit Longer term, we might want me to: * switch to a different build system * add support for that build system into test-patch :) We'll most definitely want to nuke all of the Hadoop bits out of the branch at some point, but for now it's useful for testing test-patch.sh itself. build environment and unit tests Key: HADOOP-12137 URL: https://issues.apache.org/jira/browse/HADOOP-12137 Project: Hadoop Common Issue Type: Sub-task Components: yetus Reporter: Allen Wittenauer Priority: Critical We need to have some way to build (esp the documentation!) and unit tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12136) Add schemas to site.xml-s
[ https://issues.apache.org/jira/browse/HADOOP-12136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Liptak updated HADOOP-12136: -- Release Note: HADOOP-12136 Add schemas to site.xml-s Status: Patch Available (was: Open) Add schemas to site.xml-s - Key: HADOOP-12136 URL: https://issues.apache.org/jira/browse/HADOOP-12136 Project: Hadoop Common Issue Type: Improvement Reporter: Gabor Liptak Assignee: Gabor Liptak Priority: Minor Attachments: HADOOP-12136.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12136) Add schemas to site.xml-s
[ https://issues.apache.org/jira/browse/HADOOP-12136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Liptak updated HADOOP-12136: -- Attachment: HADOOP-12136.1.patch Add schemas to site.xml-s - Key: HADOOP-12136 URL: https://issues.apache.org/jira/browse/HADOOP-12136 Project: Hadoop Common Issue Type: Improvement Reporter: Gabor Liptak Assignee: Gabor Liptak Priority: Minor Attachments: HADOOP-12136.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12133) Add schemas to Maven Assembly XMLs
[ https://issues.apache.org/jira/browse/HADOOP-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Liptak updated HADOOP-12133: -- Attachment: HADOOP-12133.2.patch Add schemas to Maven Assembly XMLs -- Key: HADOOP-12133 URL: https://issues.apache.org/jira/browse/HADOOP-12133 Project: Hadoop Common Issue Type: Improvement Reporter: Gabor Liptak Assignee: Gabor Liptak Priority: Minor Attachments: HADOOP-12133.1.patch, HADOOP-12133.2.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-12140) releasedocmaker better versions support
Allen Wittenauer created HADOOP-12140: - Summary: releasedocmaker better versions support Key: HADOOP-12140 URL: https://issues.apache.org/jira/browse/HADOOP-12140 Project: Hadoop Common Issue Type: Sub-task Components: yetus Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer It would be better if releasedocmaker didn't merge versions; it should create a separate file for each version. Additionally, it would be great if it took a range of versions, querying jira to discover which versions are available between release A and release B. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12126) Configuration might use ApplicationClassLoader to create XML parser
[
https://issues.apache.org/jira/browse/HADOOP-12126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604410#comment-14604410
]
Gera Shegalov commented on HADOOP-12126:
I agree that loading {{DocumentBuilderFactory}} has to be fixed, however, it
should respect the {{classloader}} instance variable of {{Configuration}}
instead of blindly resorting to {{Configuration.class.getClassloader()}}. By
default, the {{classloader}} instance variable will point to
ApplicationClassLoader. Thus we still have a problem that has to be resolved
outside the configuration class in the {{ApplicationClassLoader}} of the app.
Configuration might use ApplicationClassLoader to create XML parser
---
Key: HADOOP-12126
URL: https://issues.apache.org/jira/browse/HADOOP-12126
Project: Hadoop Common
Issue Type: Bug
Components: conf
Reporter: Laurent Goujon
Assignee: Laurent Goujon
Attachments: HADOOP-12126.001.patch, HADOOP-12126.002.patch,
HADOOP-12126.003.patch
{{org.apache.hadoop.conf.Configuration}} creates a new DocumentBuilder to
parse the XML config files, but it doesn't specify which classloader to use
to discover and instantiate the XML parser.
Because DocumentBuilderFactory relies on ServiceProvider, whose by default,
uses the context classloader. If classpath isolation is turned on, one might
expect that that Configuration will only load classes from the system
classloader, but it turns out that the context classloader is set to
ApplicationClassLoader, and that a XML parser might be loaded from the user
classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12126) Configuration might use ApplicationClassLoader to create XML parser
[
https://issues.apache.org/jira/browse/HADOOP-12126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gera Shegalov updated HADOOP-12126:
---
Status: Open (was: Patch Available)
Configuration might use ApplicationClassLoader to create XML parser
---
Key: HADOOP-12126
URL: https://issues.apache.org/jira/browse/HADOOP-12126
Project: Hadoop Common
Issue Type: Bug
Components: conf
Reporter: Laurent Goujon
Assignee: Laurent Goujon
Attachments: HADOOP-12126.001.patch, HADOOP-12126.002.patch,
HADOOP-12126.003.patch
{{org.apache.hadoop.conf.Configuration}} creates a new DocumentBuilder to
parse the XML config files, but it doesn't specify which classloader to use
to discover and instantiate the XML parser.
Because DocumentBuilderFactory relies on ServiceProvider, whose by default,
uses the context classloader. If classpath isolation is turned on, one might
expect that that Configuration will only load classes from the system
classloader, but it turns out that the context classloader is set to
ApplicationClassLoader, and that a XML parser might be loaded from the user
classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12126) Configuration might use ApplicationClassLoader to create XML parser
[
https://issues.apache.org/jira/browse/HADOOP-12126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gera Shegalov updated HADOOP-12126:
---
Affects Version/s: 2.7.0
Configuration might use ApplicationClassLoader to create XML parser
---
Key: HADOOP-12126
URL: https://issues.apache.org/jira/browse/HADOOP-12126
Project: Hadoop Common
Issue Type: Bug
Components: conf
Affects Versions: 2.7.0
Reporter: Laurent Goujon
Assignee: Laurent Goujon
Attachments: HADOOP-12126.001.patch, HADOOP-12126.002.patch,
HADOOP-12126.003.patch
{{org.apache.hadoop.conf.Configuration}} creates a new DocumentBuilder to
parse the XML config files, but it doesn't specify which classloader to use
to discover and instantiate the XML parser.
Because DocumentBuilderFactory relies on ServiceProvider, whose by default,
uses the context classloader. If classpath isolation is turned on, one might
expect that that Configuration will only load classes from the system
classloader, but it turns out that the context classloader is set to
ApplicationClassLoader, and that a XML parser might be loaded from the user
classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
