[jira] [Moved] (HADOOP-11842) Common side change for follow-on work for erasure coding phase I
[ https://issues.apache.org/jira/browse/HADOOP-11842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze moved HDFS-8168 to HADOOP-11842: Key: HADOOP-11842 (was: HDFS-8168) Project: Hadoop Common (was: Hadoop HDFS) > Common side change for follow-on work for erasure coding phase I > > > Key: HADOOP-11842 > URL: https://issues.apache.org/jira/browse/HADOOP-11842 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > > This is the common side work for HDFS-8031. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11540) Raw Reed-Solomon coder using Intel ISA-L library
[ https://issues.apache.org/jira/browse/HADOOP-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499032#comment-14499032 ] Tsz Wo Nicholas Sze commented on HADOOP-11540: -- Do you like this in Phase 1 or the follow-on work for phase 1? > Raw Reed-Solomon coder using Intel ISA-L library > > > Key: HADOOP-11540 > URL: https://issues.apache.org/jira/browse/HADOOP-11540 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: HDFS-7285 >Reporter: Zhe Zhang >Assignee: Kai Zheng > > This is to provide RS codec implementation using Intel ISA-L library for > encoding and decoding. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11837) After HADOOP-11754, oozie fails to stop cleanly
[
https://issues.apache.org/jira/browse/HADOOP-11837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498990#comment-14498990
]
Hadoop QA commented on HADOOP-11837:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12725969/HADOOP-11837.patch
against trunk revision 75bbcc8.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 javac{color}. The applied patch generated 1147 javac
compiler warnings (more than the trunk's current 209 warnings).
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
43 warning messages.
See
https://builds.apache.org/job/PreCommit-HADOOP-Build/6113//artifact/patchprocess/diffJavadocWarnings.txt
for details.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-client:
org.apache.hadoop.hdfs.TestPread
org.apache.hadoop.hdfs.TestParallelShortCircuitRead
org.apache.hadoop.hdfs.TestDatanodeLayoutUpgrade
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6113//testReport/
Javac warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6113//artifact/patchprocess/diffJavacWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6113//console
This message is automatically generated.
> After HADOOP-11754, oozie fails to stop cleanly
> ---
>
> Key: HADOOP-11837
> URL: https://issues.apache.org/jira/browse/HADOOP-11837
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Affects Versions: 2.7.0
>Reporter: Venkat Ranganathan
>Assignee: Bowen Zhang
>Priority: Blocker
> Fix For: 2.7.0
>
> Attachments: HADOOP-11837.patch, HADOOP-11837.patch,
> HADOOP-11837.patch
>
>
> After HADOOP-11754, AuthenticationFilter has to be enhanced to destroy to
> secret provider. Else, products like Oozie which extend the
> AuthenticationFilter fail to stop
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11766) Generic token authentication support for Hadoop
[
https://issues.apache.org/jira/browse/HADOOP-11766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liu updated HADOOP-11766:
Assignee: Kai Zheng
> Generic token authentication support for Hadoop
> ---
>
> Key: HADOOP-11766
> URL: https://issues.apache.org/jira/browse/HADOOP-11766
> Project: Hadoop Common
> Issue Type: New Feature
> Components: security
>Reporter: Kai Zheng
>Assignee: Kai Zheng
>
> As a major goal of Rhino project, we proposed *TokenAuth* effort in
> HADOOP-9392, where it's to provide a common token authentication framework to
> integrate multiple authentication mechanisms, by adding a new
> {{AuthenticationMethod}} in lieu of {{KERBEROS}} and {{SIMPLE}}. To minimize
> the required changes and risk, we thought of another approach to achieve the
> general goals based on Kerberos as Kerberos itself supports a
> pre-authentication framework in both spec and implementation, which was
> discussed in HADOOP-10959 as *TokenPreauth*. In both approaches, we had
> performed workable prototypes covering both command line console and Hadoop
> web UI.
> As HADOOP-9392 is rather lengthy and heavy, HADOOP-10959 is mostly focused on
> the concrete implementation approach based on Kerberos, we open this for more
> general and updated discussions about requirement, use cases, and concerns
> for the generic token authentication support for Hadoop. We distinguish this
> token from existing Hadoop tokens as the token in this discussion is majorly
> for the initial and primary authentication. We will refine our existing codes
> in HADOOP-9392 and HADOOP-10959, break them down into smaller patches based
> on latest trunk.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11837) After HADOOP-11754, oozie fails to stop cleanly
[
https://issues.apache.org/jira/browse/HADOOP-11837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499346#comment-14499346
]
Hadoop QA commented on HADOOP-11837:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12725969/HADOOP-11837.patch
against trunk revision 76e7264.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-auth.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6114//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6114//console
This message is automatically generated.
> After HADOOP-11754, oozie fails to stop cleanly
> ---
>
> Key: HADOOP-11837
> URL: https://issues.apache.org/jira/browse/HADOOP-11837
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Affects Versions: 2.7.0
>Reporter: Venkat Ranganathan
>Assignee: Bowen Zhang
>Priority: Blocker
> Fix For: 2.7.0
>
> Attachments: HADOOP-11837.patch, HADOOP-11837.patch,
> HADOOP-11837.patch
>
>
> After HADOOP-11754, AuthenticationFilter has to be enhanced to destroy to
> secret provider. Else, products like Oozie which extend the
> AuthenticationFilter fail to stop
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11540) Raw Reed-Solomon coder using Intel ISA-L library
[ https://issues.apache.org/jira/browse/HADOOP-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499243#comment-14499243 ] Kai Zheng commented on HADOOP-11540: I have the workable codes already and would commit them in HDFS-7285. Thanks. > Raw Reed-Solomon coder using Intel ISA-L library > > > Key: HADOOP-11540 > URL: https://issues.apache.org/jira/browse/HADOOP-11540 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: HDFS-7285 >Reporter: Zhe Zhang >Assignee: Kai Zheng > > This is to provide RS codec implementation using Intel ISA-L library for > encoding and decoding. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11746) rewrite test-patch.sh
[ https://issues.apache.org/jira/browse/HADOOP-11746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-11746: -- Attachment: HADOOP-11746-17.patch After sleeping on it, I think it's better to be safe than sorry in case of problems. A few extra lines for extra paranoia... -17: * During re-exec mode, make sure jenkins reports the console in the "switching" message in case something goes wrong. * In reset repo mode, always force checkout to trunk after cleaning the repo area. This might not actually matter in the grand scheme of things, but if something goes haywire in the middle this should provide some level of safety to get us back to a clean slate. * Don't switch to any branches that match: branch-0, branch-1, branch-2.[0-7]. It should be noted that currently the code to find a branch won't properly detect minor or micro version branches anyway, so the branch-2 minor version lockout is just in case someone fixes that bug later. ;) It should be safe to commit this to branch-2 and have the system come back to trunk for 'real' patches. > rewrite test-patch.sh > - > > Key: HADOOP-11746 > URL: https://issues.apache.org/jira/browse/HADOOP-11746 > Project: Hadoop Common > Issue Type: Test > Components: build, test >Affects Versions: 3.0.0 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Attachments: HADOOP-11746-00.patch, HADOOP-11746-01.patch, > HADOOP-11746-02.patch, HADOOP-11746-03.patch, HADOOP-11746-04.patch, > HADOOP-11746-05.patch, HADOOP-11746-06.patch, HADOOP-11746-07.patch, > HADOOP-11746-09.patch, HADOOP-11746-10.patch, HADOOP-11746-11.patch, > HADOOP-11746-12.patch, HADOOP-11746-13.patch, HADOOP-11746-14.patch, > HADOOP-11746-15.patch, HADOOP-11746-16.patch, HADOOP-11746-17.patch > > > This code is bad and you should feel bad. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-10101) Update guava dependency to the latest version
[ https://issues.apache.org/jira/browse/HADOOP-10101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499461#comment-14499461 ] Karel Vervaeke commented on HADOOP-10101: - I think [~busbey] made a good point. I'm sure there are some practical concerns, but as more libraries start to depend on more recent guava versions it becomes more of a problem. For now I've taken the reverse approach for our integration tests (shading dropwizard 0.8.1 w. relocated guava classes) to allow both hadoop and dropwizard services to run in one jvm. > Update guava dependency to the latest version > - > > Key: HADOOP-10101 > URL: https://issues.apache.org/jira/browse/HADOOP-10101 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 2.2.0, 2.6.0 >Reporter: Rakesh R >Assignee: Vinayakumar B > Attachments: HADOOP-10101-002.patch, HADOOP-10101-004.patch, > HADOOP-10101-005.patch, HADOOP-10101-006.patch, HADOOP-10101-007.patch, > HADOOP-10101-008.patch, HADOOP-10101-009.patch, HADOOP-10101-009.patch, > HADOOP-10101-010.patch, HADOOP-10101-010.patch, HADOOP-10101-011.patch, > HADOOP-10101.patch, HADOOP-10101.patch > > > The existing guava version is 11.0.2 which is quite old. This -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11746) rewrite test-patch.sh
[
https://issues.apache.org/jira/browse/HADOOP-11746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499479#comment-14499479
]
Hadoop QA commented on HADOOP-11746:
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 23s | The patch command could not apply
the patch. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12726104/HADOOP-11746-17.patch |
| Optional Tests | shellcheck |
| git revision | HADOOP-11746 / 4d7c867 |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/fake//console |
This message was automatically generated.
> rewrite test-patch.sh
> -
>
> Key: HADOOP-11746
> URL: https://issues.apache.org/jira/browse/HADOOP-11746
> Project: Hadoop Common
> Issue Type: Test
> Components: build, test
>Affects Versions: 3.0.0
>Reporter: Allen Wittenauer
>Assignee: Allen Wittenauer
> Attachments: HADOOP-11746-00.patch, HADOOP-11746-01.patch,
> HADOOP-11746-02.patch, HADOOP-11746-03.patch, HADOOP-11746-04.patch,
> HADOOP-11746-05.patch, HADOOP-11746-06.patch, HADOOP-11746-07.patch,
> HADOOP-11746-09.patch, HADOOP-11746-10.patch, HADOOP-11746-11.patch,
> HADOOP-11746-12.patch, HADOOP-11746-13.patch, HADOOP-11746-14.patch,
> HADOOP-11746-15.patch, HADOOP-11746-16.patch, HADOOP-11746-17.patch
>
>
> This code is bad and you should feel bad.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Issue Comment Deleted] (HADOOP-11746) rewrite test-patch.sh
[
https://issues.apache.org/jira/browse/HADOOP-11746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer updated HADOOP-11746:
--
Comment: was deleted
(was: \\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 23s | The patch command could not apply
the patch. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12726104/HADOOP-11746-17.patch |
| Optional Tests | shellcheck |
| git revision | HADOOP-11746 / 4d7c867 |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/fake//console |
This message was automatically generated.)
> rewrite test-patch.sh
> -
>
> Key: HADOOP-11746
> URL: https://issues.apache.org/jira/browse/HADOOP-11746
> Project: Hadoop Common
> Issue Type: Test
> Components: build, test
>Affects Versions: 3.0.0
>Reporter: Allen Wittenauer
>Assignee: Allen Wittenauer
> Attachments: HADOOP-11746-00.patch, HADOOP-11746-01.patch,
> HADOOP-11746-02.patch, HADOOP-11746-03.patch, HADOOP-11746-04.patch,
> HADOOP-11746-05.patch, HADOOP-11746-06.patch, HADOOP-11746-07.patch,
> HADOOP-11746-09.patch, HADOOP-11746-10.patch, HADOOP-11746-11.patch,
> HADOOP-11746-12.patch, HADOOP-11746-13.patch, HADOOP-11746-14.patch,
> HADOOP-11746-15.patch, HADOOP-11746-16.patch, HADOOP-11746-17.patch
>
>
> This code is bad and you should feel bad.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11820) aw jira testing, ignore
[ https://issues.apache.org/jira/browse/HADOOP-11820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-11820: -- Attachment: HADOOP-11746-17.patch > aw jira testing, ignore > --- > > Key: HADOOP-11820 > URL: https://issues.apache.org/jira/browse/HADOOP-11820 > Project: Hadoop Common > Issue Type: Task >Reporter: Allen Wittenauer > Attachments: HADOOP-11746-14.patch, HADOOP-11746-16.patch, > HADOOP-11746-17.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11820) aw jira testing, ignore
[ https://issues.apache.org/jira/browse/HADOOP-11820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499482#comment-14499482 ] Hadoop QA commented on HADOOP-11820: (!) A patch to test-patch or smart-apply-patch has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/PreCommit-HADOOP-Build/fake//console in case of problems. > aw jira testing, ignore > --- > > Key: HADOOP-11820 > URL: https://issues.apache.org/jira/browse/HADOOP-11820 > Project: Hadoop Common > Issue Type: Task >Reporter: Allen Wittenauer > Attachments: HADOOP-11746-14.patch, HADOOP-11746-16.patch, > HADOOP-11746-17.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11820) aw jira testing, ignore
[
https://issues.apache.org/jira/browse/HADOOP-11820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499483#comment-14499483
]
Hadoop QA commented on HADOOP-11820:
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | reexec | 0m 00s | dev-support patch detected. |
| {color:blue}0{color} | pre-patch | 0m 00s | Pre-patch trunk compilation is
healthy. |
| {color:red}-1{color} | @author | 0m 00s | The patch appears to contain 13
@author tags which the Hadoop community has agreed to not allow in code
contributions. |
| {color:green}+1{color} | whitespace | 0m 00s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | release audit | 0m 09s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | shellcheck | 0m 03s | The applied patch generated 2
new shellcheck issues (total was 169, now 27). |
| | | 0m 30s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12726106/HADOOP-11746-17.patch |
| Optional Tests | shellcheck |
| git revision | trunk / b9b832a |
| shellcheck |
https://builds.apache.org/job/PreCommit-HADOOP-Build/fake/artifact/patchprocess/diffpatchshellcheck.txt
|
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/fake//console |
This message was automatically generated.
> aw jira testing, ignore
> ---
>
> Key: HADOOP-11820
> URL: https://issues.apache.org/jira/browse/HADOOP-11820
> Project: Hadoop Common
> Issue Type: Task
>Reporter: Allen Wittenauer
> Attachments: HADOOP-11746-14.patch, HADOOP-11746-16.patch,
> HADOOP-11746-17.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11746) rewrite test-patch.sh
[
https://issues.apache.org/jira/browse/HADOOP-11746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499508#comment-14499508
]
Hadoop QA commented on HADOOP-11746:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12726104/HADOOP-11746-17.patch
against trunk revision 76e7264.
{color:red}-1 @author{color}. The patch appears to contain 13 @author tags
which the Hadoop community has agreed to not allow in code contributions.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6115//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6115//console
This message is automatically generated.
> rewrite test-patch.sh
> -
>
> Key: HADOOP-11746
> URL: https://issues.apache.org/jira/browse/HADOOP-11746
> Project: Hadoop Common
> Issue Type: Test
> Components: build, test
>Affects Versions: 3.0.0
>Reporter: Allen Wittenauer
>Assignee: Allen Wittenauer
> Attachments: HADOOP-11746-00.patch, HADOOP-11746-01.patch,
> HADOOP-11746-02.patch, HADOOP-11746-03.patch, HADOOP-11746-04.patch,
> HADOOP-11746-05.patch, HADOOP-11746-06.patch, HADOOP-11746-07.patch,
> HADOOP-11746-09.patch, HADOOP-11746-10.patch, HADOOP-11746-11.patch,
> HADOOP-11746-12.patch, HADOOP-11746-13.patch, HADOOP-11746-14.patch,
> HADOOP-11746-15.patch, HADOOP-11746-16.patch, HADOOP-11746-17.patch
>
>
> This code is bad and you should feel bad.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11820) aw jira testing, ignore
[
https://issues.apache.org/jira/browse/HADOOP-11820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499516#comment-14499516
]
Hadoop QA commented on HADOOP-11820:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12726106/HADOOP-11746-17.patch
against trunk revision 76e7264.
{color:red}-1 @author{color}. The patch appears to contain 13 @author tags
which the Hadoop community has agreed to not allow in code contributions.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6116//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6116//console
This message is automatically generated.
> aw jira testing, ignore
> ---
>
> Key: HADOOP-11820
> URL: https://issues.apache.org/jira/browse/HADOOP-11820
> Project: Hadoop Common
> Issue Type: Task
>Reporter: Allen Wittenauer
> Attachments: HADOOP-11746-14.patch, HADOOP-11746-16.patch,
> HADOOP-11746-17.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HADOOP-11843) Make setting up the build environment easier
Niels Basjes created HADOOP-11843: - Summary: Make setting up the build environment easier Key: HADOOP-11843 URL: https://issues.apache.org/jira/browse/HADOOP-11843 Project: Hadoop Common Issue Type: New Feature Reporter: Niels Basjes Assignee: Niels Basjes ( As discussed with [~aw] ) In AVRO-1537 a docker based solution was created to setup all the tools for doing a full build. This enables much easier reproduction of any issues and getting up and running for new developers. This issue is to 'copy/port' that setup into the hadoop project in preparation for the bug squash. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11649) Erasure Coding: Load multiple erasure codecs
[ https://issues.apache.org/jira/browse/HADOOP-11649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HADOOP-11649: --- Summary: Erasure Coding: Load multiple erasure codecs (was: Allow to configure multiple erasure codecs) > Erasure Coding: Load multiple erasure codecs > > > Key: HADOOP-11649 > URL: https://issues.apache.org/jira/browse/HADOOP-11649 > Project: Hadoop Common > Issue Type: Sub-task > Components: io >Reporter: Kai Zheng >Assignee: Kai Zheng > Attachments: HADOOP-11649-v1.patch > > > This is to allow to configure erasure codec and coder in core-site > configuration file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HADOOP-11834) Add ErasureCodecFactory to create ErasureCodec using codec's short name.
[
https://issues.apache.org/jira/browse/HADOOP-11834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinayakumar B resolved HADOOP-11834.
Resolution: Duplicate
> Add ErasureCodecFactory to create ErasureCodec using codec's short name.
>
>
> Key: HADOOP-11834
> URL: https://issues.apache.org/jira/browse/HADOOP-11834
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: io
>Reporter: Vinayakumar B
>Assignee: Vinayakumar B
>
> Using Codec's shortname Codec instance should be create.
> such as, using name "rs" {{RSErasureCodec}} should be created.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11843) Make setting up the build environment easier
[ https://issues.apache.org/jira/browse/HADOOP-11843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niels Basjes updated HADOOP-11843: -- Attachment: HADOOP-11843-2015-04-17-1612.patch This patch offers an start-build-env.sh that creates a fully functional environment with (as far as I've been able to test) all the tools needed to build the Hadoop source tree. Jenkins won't be able to test this. > Make setting up the build environment easier > > > Key: HADOOP-11843 > URL: https://issues.apache.org/jira/browse/HADOOP-11843 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Niels Basjes >Assignee: Niels Basjes > Attachments: HADOOP-11843-2015-04-17-1612.patch > > > ( As discussed with [~aw] ) > In AVRO-1537 a docker based solution was created to setup all the tools for > doing a full build. This enables much easier reproduction of any issues and > getting up and running for new developers. > This issue is to 'copy/port' that setup into the hadoop project in > preparation for the bug squash. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11843) Make setting up the build environment easier
[ https://issues.apache.org/jira/browse/HADOOP-11843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499897#comment-14499897 ] Niels Basjes commented on HADOOP-11843: --- Because this patch only has new files I expect this patch can be applied to any branch without modification. > Make setting up the build environment easier > > > Key: HADOOP-11843 > URL: https://issues.apache.org/jira/browse/HADOOP-11843 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Niels Basjes >Assignee: Niels Basjes > Attachments: HADOOP-11843-2015-04-17-1612.patch > > > ( As discussed with [~aw] ) > In AVRO-1537 a docker based solution was created to setup all the tools for > doing a full build. This enables much easier reproduction of any issues and > getting up and running for new developers. > This issue is to 'copy/port' that setup into the hadoop project in > preparation for the bug squash. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11843) Make setting up the build environment easier
[ https://issues.apache.org/jira/browse/HADOOP-11843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niels Basjes updated HADOOP-11843: -- Status: Patch Available (was: Open) > Make setting up the build environment easier > > > Key: HADOOP-11843 > URL: https://issues.apache.org/jira/browse/HADOOP-11843 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Niels Basjes >Assignee: Niels Basjes > Attachments: HADOOP-11843-2015-04-17-1612.patch > > > ( As discussed with [~aw] ) > In AVRO-1537 a docker based solution was created to setup all the tools for > doing a full build. This enables much easier reproduction of any issues and > getting up and running for new developers. > This issue is to 'copy/port' that setup into the hadoop project in > preparation for the bug squash. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11844) SLS docs point to invalid rumen link
[ https://issues.apache.org/jira/browse/HADOOP-11844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-11844: -- Affects Version/s: 2.6.0 > SLS docs point to invalid rumen link > > > Key: HADOOP-11844 > URL: https://issues.apache.org/jira/browse/HADOOP-11844 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 2.6.0 >Reporter: Allen Wittenauer >Priority: Trivial > Labels: newbie > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-11844) SLS docs point to invalid rumen link
Allen Wittenauer created HADOOP-11844: - Summary: SLS docs point to invalid rumen link Key: HADOOP-11844 URL: https://issues.apache.org/jira/browse/HADOOP-11844 Project: Hadoop Common Issue Type: Bug Components: documentation Reporter: Allen Wittenauer Priority: Trivial -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11844) SLS docs point to invalid rumen link
[ https://issues.apache.org/jira/browse/HADOOP-11844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-11844: -- Description: SchedulerLoadSimulator at least on 2.6.0 points to an invalid link to rumen. Need to verify and potentially fix this link in newer releases. > SLS docs point to invalid rumen link > > > Key: HADOOP-11844 > URL: https://issues.apache.org/jira/browse/HADOOP-11844 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 2.6.0 >Reporter: Allen Wittenauer >Priority: Trivial > Labels: newbie > > SchedulerLoadSimulator at least on 2.6.0 points to an invalid link to rumen. > Need to verify and potentially fix this link in newer releases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11746) rewrite test-patch.sh
[ https://issues.apache.org/jira/browse/HADOOP-11746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499925#comment-14499925 ] Sean Busbey commented on HADOOP-11746: -- The jenkins job should be restoring a clean workspace at job start, so we shouldn't need to worry about changing branches back after. > rewrite test-patch.sh > - > > Key: HADOOP-11746 > URL: https://issues.apache.org/jira/browse/HADOOP-11746 > Project: Hadoop Common > Issue Type: Test > Components: build, test >Affects Versions: 3.0.0 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Attachments: HADOOP-11746-00.patch, HADOOP-11746-01.patch, > HADOOP-11746-02.patch, HADOOP-11746-03.patch, HADOOP-11746-04.patch, > HADOOP-11746-05.patch, HADOOP-11746-06.patch, HADOOP-11746-07.patch, > HADOOP-11746-09.patch, HADOOP-11746-10.patch, HADOOP-11746-11.patch, > HADOOP-11746-12.patch, HADOOP-11746-13.patch, HADOOP-11746-14.patch, > HADOOP-11746-15.patch, HADOOP-11746-16.patch, HADOOP-11746-17.patch > > > This code is bad and you should feel bad. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11843) Make setting up the build environment easier
[
https://issues.apache.org/jira/browse/HADOOP-11843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499924#comment-14499924
]
Hadoop QA commented on HADOOP-11843:
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 0m 00s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 00s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | whitespace | 0m 00s | The patch has 7 line(s) that
end in whitespace. |
| {color:red}-1{color} | release audit | 0m 04s | The applied patch generated
1 release audit warnings. |
| {color:red}-1{color} | shellcheck | 0m 02s | The applied patch generated
12 new shellcheck issues (total was 25, now 37). |
| | | 0m 14s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12726165/HADOOP-11843-2015-04-17-1612.patch
|
| Optional Tests | shellcheck |
| git revision | trunk / b9b832a |
| whitespace |
https://builds.apache.org/job/PreCommit-HADOOP-Build/fake/artifact/patchprocess/whitespace.txt
|
| Release Audit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/fake/artifact/patchprocess/patchReleaseAuditProblems.txt
|
| shellcheck |
https://builds.apache.org/job/PreCommit-HADOOP-Build/fake/artifact/patchprocess/diffpatchshellcheck.txt
|
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/fake//console |
This message was automatically generated.
> Make setting up the build environment easier
>
>
> Key: HADOOP-11843
> URL: https://issues.apache.org/jira/browse/HADOOP-11843
> Project: Hadoop Common
> Issue Type: New Feature
>Reporter: Niels Basjes
>Assignee: Niels Basjes
> Attachments: HADOOP-11843-2015-04-17-1612.patch
>
>
> ( As discussed with [~aw] )
> In AVRO-1537 a docker based solution was created to setup all the tools for
> doing a full build. This enables much easier reproduction of any issues and
> getting up and running for new developers.
> This issue is to 'copy/port' that setup into the hadoop project in
> preparation for the bug squash.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11843) Make setting up the build environment easier
[ https://issues.apache.org/jira/browse/HADOOP-11843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499926#comment-14499926 ] Allen Wittenauer commented on HADOOP-11843: --- just me testing the new test-patch.sh script against your patch :) > Make setting up the build environment easier > > > Key: HADOOP-11843 > URL: https://issues.apache.org/jira/browse/HADOOP-11843 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Niels Basjes >Assignee: Niels Basjes > Attachments: HADOOP-11843-2015-04-17-1612.patch > > > ( As discussed with [~aw] ) > In AVRO-1537 a docker based solution was created to setup all the tools for > doing a full build. This enables much easier reproduction of any issues and > getting up and running for new developers. > This issue is to 'copy/port' that setup into the hadoop project in > preparation for the bug squash. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11843) Make setting up the build environment easier
[ https://issues.apache.org/jira/browse/HADOOP-11843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499943#comment-14499943 ] Allen Wittenauer commented on HADOOP-11843: --- This is awesome. I'll start playing with it here in a bit! A quick view of the patch does seem to indicate a few things to fix already though: * motd should have an apache license on it. * Dockerfile says "for Avro" still ;) * Hadoop's dependencies are actually smaller. off the top, no autotools, subversion, boost, doxygen, forrest, or ant. * Add shellcheck and findbugs, if possible, for the new test-patch.sh code that's about to get committed... * Let's clean up the extra whitespace on the end of lines. This makes git happy. Much thanks! > Make setting up the build environment easier > > > Key: HADOOP-11843 > URL: https://issues.apache.org/jira/browse/HADOOP-11843 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Niels Basjes >Assignee: Niels Basjes > Attachments: HADOOP-11843-2015-04-17-1612.patch > > > ( As discussed with [~aw] ) > In AVRO-1537 a docker based solution was created to setup all the tools for > doing a full build. This enables much easier reproduction of any issues and > getting up and running for new developers. > This issue is to 'copy/port' that setup into the hadoop project in > preparation for the bug squash. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11843) Make setting up the build environment easier
[
https://issues.apache.org/jira/browse/HADOOP-11843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499973#comment-14499973
]
Hadoop QA commented on HADOOP-11843:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12726165/HADOOP-11843-2015-04-17-1612.patch
against trunk revision 76e7264.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1
release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6117//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6117//artifact/patchprocess/patchReleaseAuditProblems.txt
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6117//console
This message is automatically generated.
> Make setting up the build environment easier
>
>
> Key: HADOOP-11843
> URL: https://issues.apache.org/jira/browse/HADOOP-11843
> Project: Hadoop Common
> Issue Type: New Feature
>Reporter: Niels Basjes
>Assignee: Niels Basjes
> Attachments: HADOOP-11843-2015-04-17-1612.patch
>
>
> ( As discussed with [~aw] )
> In AVRO-1537 a docker based solution was created to setup all the tools for
> doing a full build. This enables much easier reproduction of any issues and
> getting up and running for new developers.
> This issue is to 'copy/port' that setup into the hadoop project in
> preparation for the bug squash.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11746) rewrite test-patch.sh
[
https://issues.apache.org/jira/browse/HADOOP-11746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499986#comment-14499986
]
Sean Busbey commented on HADOOP-11746:
--
+1 after non-nit issues below. Adding audience / stability comments is a really
nice touch.
{code}
+## @description Some crazy people like Eclipse. Make sure Maven's eclipse
generation
+## @description works so they don't freak out.
{code}
Please leave out the ableist pejorative.
{code}
+function check_unittests
+{
+ verify_needed_test unit
+
+ if [[ $? == 0 ]]; then
+echo "Existing unit tests do not test patched files. Skipping."
+return 0
+ fi
+ big_console_header "Running unit tests"
{code}
Nit: could we make this (and maybe the module choosing) optional so that if I
know tests end up covering my changes indirectly I can set a flag to run
through things? (Although maybe I missed how I could do this myself)
This will mostly be useful when I try to merge this set of changes with things
over in HBase, since we run through all the unit tests on pre-patch.
{code}
+if [[ -d "${mypwd}/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs"
]]; then
+ echo "Changing permission on
${mypwd}/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs to avoid broken
builds"
+ chmod +x -R
"${mypwd}/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs"
+fi
{code}
Nit: so long as we're cleaning things up, could we replace this with a general
solution? The root cause of the issue that introduced this change originally
was that we need to ensure any directories that exist under a maven module
target directory are executable. We know tests can break that one, but there's
no reason some other module might break a different one later.
> rewrite test-patch.sh
> -
>
> Key: HADOOP-11746
> URL: https://issues.apache.org/jira/browse/HADOOP-11746
> Project: Hadoop Common
> Issue Type: Test
> Components: build, test
>Affects Versions: 3.0.0
>Reporter: Allen Wittenauer
>Assignee: Allen Wittenauer
> Attachments: HADOOP-11746-00.patch, HADOOP-11746-01.patch,
> HADOOP-11746-02.patch, HADOOP-11746-03.patch, HADOOP-11746-04.patch,
> HADOOP-11746-05.patch, HADOOP-11746-06.patch, HADOOP-11746-07.patch,
> HADOOP-11746-09.patch, HADOOP-11746-10.patch, HADOOP-11746-11.patch,
> HADOOP-11746-12.patch, HADOOP-11746-13.patch, HADOOP-11746-14.patch,
> HADOOP-11746-15.patch, HADOOP-11746-16.patch, HADOOP-11746-17.patch
>
>
> This code is bad and you should feel bad.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HADOOP-11845) small bug in skipprnorep option handling in shelldocs.py
Allen Wittenauer created HADOOP-11845: - Summary: small bug in skipprnorep option handling in shelldocs.py Key: HADOOP-11845 URL: https://issues.apache.org/jira/browse/HADOOP-11845 Project: Hadoop Common Issue Type: Bug Components: documentation, scripts Affects Versions: 3.0.0 Reporter: Allen Wittenauer shelldocs.py doesn't work with test-patch.sh due to a bug in an if construction -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11845) small bug in skipprnorep option handling in shelldocs.py
[ https://issues.apache.org/jira/browse/HADOOP-11845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-11845: -- Attachment: HADOOP-11845-00.patch > small bug in skipprnorep option handling in shelldocs.py > > > Key: HADOOP-11845 > URL: https://issues.apache.org/jira/browse/HADOOP-11845 > Project: Hadoop Common > Issue Type: Bug > Components: documentation, scripts >Affects Versions: 3.0.0 >Reporter: Allen Wittenauer > Attachments: HADOOP-11845-00.patch > > > shelldocs.py doesn't work with test-patch.sh due to a bug in an if > construction -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11843) Make setting up the build environment easier
[ https://issues.apache.org/jira/browse/HADOOP-11843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14500090#comment-14500090 ] Chris Nauroth commented on HADOOP-11843: Great idea! Could we please update BUILDING.txt so that people know this is here as an option for a quick dev environment? > Make setting up the build environment easier > > > Key: HADOOP-11843 > URL: https://issues.apache.org/jira/browse/HADOOP-11843 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Niels Basjes >Assignee: Niels Basjes > Attachments: HADOOP-11843-2015-04-17-1612.patch > > > ( As discussed with [~aw] ) > In AVRO-1537 a docker based solution was created to setup all the tools for > doing a full build. This enables much easier reproduction of any issues and > getting up and running for new developers. > This issue is to 'copy/port' that setup into the hadoop project in > preparation for the bug squash. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11843) Make setting up the build environment easier
[ https://issues.apache.org/jira/browse/HADOOP-11843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14500139#comment-14500139 ] Chris Nauroth commented on HADOOP-11843: I recommend adding libcurl4-openssl-dev. libwebhdfs needs the libcurl headers. BTW, I think I'm seeing a build failure in libwebhdfs, but I'll follow up separately on that. I think we can drop libprotobuf-dev. We just need the protoc binary provided by protobuf-compiler. I don't believe we need asciidoc, source-highlight, flex, or bison. (I have no objection though if someone wants to keep these for some reason.) This is in addition to what Allen already mentioned in his earlier comment. > Make setting up the build environment easier > > > Key: HADOOP-11843 > URL: https://issues.apache.org/jira/browse/HADOOP-11843 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Niels Basjes >Assignee: Niels Basjes > Attachments: HADOOP-11843-2015-04-17-1612.patch > > > ( As discussed with [~aw] ) > In AVRO-1537 a docker based solution was created to setup all the tools for > doing a full build. This enables much easier reproduction of any issues and > getting up and running for new developers. > This issue is to 'copy/port' that setup into the hadoop project in > preparation for the bug squash. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11746) rewrite test-patch.sh
[ https://issues.apache.org/jira/browse/HADOOP-11746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14500159#comment-14500159 ] Chris Nauroth commented on HADOOP-11746: +1 from me also after resolution of Sean's comments. (Thanks, Sean!) > rewrite test-patch.sh > - > > Key: HADOOP-11746 > URL: https://issues.apache.org/jira/browse/HADOOP-11746 > Project: Hadoop Common > Issue Type: Test > Components: build, test >Affects Versions: 3.0.0 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Attachments: HADOOP-11746-00.patch, HADOOP-11746-01.patch, > HADOOP-11746-02.patch, HADOOP-11746-03.patch, HADOOP-11746-04.patch, > HADOOP-11746-05.patch, HADOOP-11746-06.patch, HADOOP-11746-07.patch, > HADOOP-11746-09.patch, HADOOP-11746-10.patch, HADOOP-11746-11.patch, > HADOOP-11746-12.patch, HADOOP-11746-13.patch, HADOOP-11746-14.patch, > HADOOP-11746-15.patch, HADOOP-11746-16.patch, HADOOP-11746-17.patch > > > This code is bad and you should feel bad. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11627) Remove io.native.lib.available from trunk
[
https://issues.apache.org/jira/browse/HADOOP-11627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14500194#comment-14500194
]
Akira AJISAKA commented on HADOOP-11627:
Mostly looks good. One comment:
{code}
+| io.native.lib.available | NONE - By Default native libs will be used for bz2
and zlib compression codecs if available. |
{code}
Not only bz2 and zlib but also other native libraries will be always used if
available. (Please see HADOOP-8642) Would you change the description such as
"NONE - Always use native libraries if available."?
I'm +1 if that is addressed.
> Remove io.native.lib.available from trunk
> -
>
> Key: HADOOP-11627
> URL: https://issues.apache.org/jira/browse/HADOOP-11627
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Akira AJISAKA
>Assignee: Brahma Reddy Battula
> Attachments: HADOOP-11627-002.patch, HADOOP-11627-003.patch,
> HADOOP-11627-004.patch, HADOOP-11627-005.patch, HADOOP-11627-006.patch,
> HADOOP-11627-007.patch, HADOOP-11627-008.patch, HADOOP-11627.patch
>
>
> According to the discussion in HADOOP-8642, we should remove
> {{io.native.lib.available}} from trunk, and always use native libraries if
> they exist.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11837) After HADOOP-11754, oozie fails to stop cleanly
[ https://issues.apache.org/jira/browse/HADOOP-11837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14500310#comment-14500310 ] Haohui Mai commented on HADOOP-11837: - I'll commit it shortly. > After HADOOP-11754, oozie fails to stop cleanly > --- > > Key: HADOOP-11837 > URL: https://issues.apache.org/jira/browse/HADOOP-11837 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.0 >Reporter: Venkat Ranganathan >Assignee: Bowen Zhang >Priority: Blocker > Fix For: 2.7.0 > > Attachments: HADOOP-11837.patch, HADOOP-11837.patch, > HADOOP-11837.patch > > > After HADOOP-11754, AuthenticationFilter has to be enhanced to destroy to > secret provider. Else, products like Oozie which extend the > AuthenticationFilter fail to stop -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11837) AuthenticationFilter should destroy SignerSecretProvider in Tomcat deployments
[ https://issues.apache.org/jira/browse/HADOOP-11837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HADOOP-11837: Description: AuthenticationFilter creates SignerSecretProvider if the filter is initialized through Tomcat. The SignerSecretProvider needs to be properly destroyed. (was: After HADOOP-11754, AuthenticationFilter has to be enhanced to destroy to secret provider. Else, products like Oozie which extend the AuthenticationFilter fail to stop ) > AuthenticationFilter should destroy SignerSecretProvider in Tomcat deployments > -- > > Key: HADOOP-11837 > URL: https://issues.apache.org/jira/browse/HADOOP-11837 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.0 >Reporter: Venkat Ranganathan >Assignee: Bowen Zhang >Priority: Blocker > Fix For: 2.7.0 > > Attachments: HADOOP-11837.patch, HADOOP-11837.patch, > HADOOP-11837.patch > > > AuthenticationFilter creates SignerSecretProvider if the filter is > initialized through Tomcat. The SignerSecretProvider needs to be properly > destroyed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11837) AuthenticationFilter should destroy SignerSecretProvider in Tomcat deployments
[ https://issues.apache.org/jira/browse/HADOOP-11837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HADOOP-11837: Summary: AuthenticationFilter should destroy SignerSecretProvider in Tomcat deployments (was: After HADOOP-11754, oozie fails to stop cleanly) > AuthenticationFilter should destroy SignerSecretProvider in Tomcat deployments > -- > > Key: HADOOP-11837 > URL: https://issues.apache.org/jira/browse/HADOOP-11837 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.0 >Reporter: Venkat Ranganathan >Assignee: Bowen Zhang >Priority: Blocker > Fix For: 2.7.0 > > Attachments: HADOOP-11837.patch, HADOOP-11837.patch, > HADOOP-11837.patch > > > After HADOOP-11754, AuthenticationFilter has to be enhanced to destroy to > secret provider. Else, products like Oozie which extend the > AuthenticationFilter fail to stop -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11837) AuthenticationFilter should destroy SignerSecretProvider in Tomcat deployments
[ https://issues.apache.org/jira/browse/HADOOP-11837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HADOOP-11837: Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I've committed the patch to trunk, branch-2 and branch-2.7. Thanks [~bowenzhangusa] for the contribution. > AuthenticationFilter should destroy SignerSecretProvider in Tomcat deployments > -- > > Key: HADOOP-11837 > URL: https://issues.apache.org/jira/browse/HADOOP-11837 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.0 >Reporter: Venkat Ranganathan >Assignee: Bowen Zhang >Priority: Blocker > Fix For: 2.7.0, 2.7.0 > > Attachments: HADOOP-11837.patch, HADOOP-11837.patch, > HADOOP-11837.patch > > > AuthenticationFilter creates SignerSecretProvider if the filter is > initialized through Tomcat. The SignerSecretProvider needs to be properly > destroyed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11837) AuthenticationFilter should destroy SignerSecretProvider in Tomcat deployments
[ https://issues.apache.org/jira/browse/HADOOP-11837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14500326#comment-14500326 ] Hudson commented on HADOOP-11837: - FAILURE: Integrated in Hadoop-trunk-Commit #7605 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7605/]) HADOOP-11837. AuthenticationFilter should destroy SignerSecretProvider in Tomcat deployments. Contributed by Bowen Zhang. (wheat9: rev c6b5203cfdfccfa22ad5379b7fee75fed850d95e) * hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationFilter.java * hadoop-common-project/hadoop-common/CHANGES.txt > AuthenticationFilter should destroy SignerSecretProvider in Tomcat deployments > -- > > Key: HADOOP-11837 > URL: https://issues.apache.org/jira/browse/HADOOP-11837 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.0 >Reporter: Venkat Ranganathan >Assignee: Bowen Zhang >Priority: Blocker > Fix For: 2.7.0 > > Attachments: HADOOP-11837.patch, HADOOP-11837.patch, > HADOOP-11837.patch > > > AuthenticationFilter creates SignerSecretProvider if the filter is > initialized through Tomcat. The SignerSecretProvider needs to be properly > destroyed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11127) Improve versioning and compatibility support in native library for downstream hadoop-common users.
[ https://issues.apache.org/jira/browse/HADOOP-11127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14500555#comment-14500555 ] William Watson commented on HADOOP-11127: - Just curious, is there an update on any discussions that took place on dev? > Improve versioning and compatibility support in native library for downstream > hadoop-common users. > -- > > Key: HADOOP-11127 > URL: https://issues.apache.org/jira/browse/HADOOP-11127 > Project: Hadoop Common > Issue Type: Bug > Components: native >Reporter: Chris Nauroth > Attachments: HADOOP-11064.003.patch > > > There is no compatibility policy enforced on the JNI function signatures > implemented in the native library. This library typically is deployed to all > nodes in a cluster, built from a specific source code version. However, > downstream applications that want to run in that cluster might choose to > bundle a hadoop-common jar at a different version. Since there is no > compatibility policy, this can cause link errors at runtime when the native > function signatures expected by hadoop-common.jar do not exist in > libhadoop.so/hadoop.dll. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11127) Improve versioning and compatibility support in native library for downstream hadoop-common users.
[ https://issues.apache.org/jira/browse/HADOOP-11127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14500566#comment-14500566 ] Chris Nauroth commented on HADOOP-11127: [~wattsinabox], no, there has been no further progress since the discussion that's already documented here. I've been meaning to revisit this, hopefully by May. > Improve versioning and compatibility support in native library for downstream > hadoop-common users. > -- > > Key: HADOOP-11127 > URL: https://issues.apache.org/jira/browse/HADOOP-11127 > Project: Hadoop Common > Issue Type: Bug > Components: native >Reporter: Chris Nauroth > Attachments: HADOOP-11064.003.patch > > > There is no compatibility policy enforced on the JNI function signatures > implemented in the native library. This library typically is deployed to all > nodes in a cluster, built from a specific source code version. However, > downstream applications that want to run in that cluster might choose to > bundle a hadoop-common jar at a different version. Since there is no > compatibility policy, this can cause link errors at runtime when the native > function signatures expected by hadoop-common.jar do not exist in > libhadoop.so/hadoop.dll. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11746) rewrite test-patch.sh
[
https://issues.apache.org/jira/browse/HADOOP-11746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer updated HADOOP-11746:
--
Attachment: HADOOP-11746-18.patch
-18:
* add --modulelist and --testlist options to forcibly add modules and test
subsystems to a run. there is no way to delete, but that's probably a good
thing really.
* removed the branch restrictions since this apparently won't impact jenkins
* the check for new tests actually uses the /test/ file path against the list
of changed files in the check for new tests in unit test mode. This prevents
some false positives and should improve the count in certain/rare situations.
Prior to this, it just looked for (something)/test(something) in the actual
patch... which meant test-patch itself would be cleared. (Ironically,
test-patch.sh is now capable of testing itself, so ... yeah. test-ception.)
* in developer mode when --resetrepo is used, the user output reminds that the
code the user chose to run in a destructive mode. (--resetrepo is the default
in jenkins mode, obviously)
* Added code to chmod +x all directories in echo_and_redirect. This should
cover every maven run.
* Cleaned up the eclipse description.
* Checkstyle now uses the ${PROJECT_NAME} var instead of the hard coded hadoop
value, making the code more portable.
* Reworded some debug output for test subsystems when they are triggered
* verify_needed_test was missing documentation
bq. Adding audience / stability comments is a really nice touch.
Thanks. If you run shelldoc.py (after HADOOP-11845 is applied... ugh) against
the code, you'll get a markdown document of the function calls. We're currently
using this same functionality in trunk to document hadoop-functions.sh as the
'official' shell API for shell profiles.
> rewrite test-patch.sh
> -
>
> Key: HADOOP-11746
> URL: https://issues.apache.org/jira/browse/HADOOP-11746
> Project: Hadoop Common
> Issue Type: Test
> Components: build, test
>Affects Versions: 3.0.0
>Reporter: Allen Wittenauer
>Assignee: Allen Wittenauer
> Attachments: HADOOP-11746-00.patch, HADOOP-11746-01.patch,
> HADOOP-11746-02.patch, HADOOP-11746-03.patch, HADOOP-11746-04.patch,
> HADOOP-11746-05.patch, HADOOP-11746-06.patch, HADOOP-11746-07.patch,
> HADOOP-11746-09.patch, HADOOP-11746-10.patch, HADOOP-11746-11.patch,
> HADOOP-11746-12.patch, HADOOP-11746-13.patch, HADOOP-11746-14.patch,
> HADOOP-11746-15.patch, HADOOP-11746-16.patch, HADOOP-11746-17.patch,
> HADOOP-11746-18.patch
>
>
> This code is bad and you should feel bad.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HADOOP-11746) rewrite test-patch.sh
[
https://issues.apache.org/jira/browse/HADOOP-11746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14500609#comment-14500609
]
Allen Wittenauer edited comment on HADOOP-11746 at 4/17/15 8:36 PM:
-18:
* add --modulelist and --testlist options to forcibly add modules and test
subsystems to a run. there is no way to delete, but that's probably a good
thing really.
* removed the branch restrictions since this apparently won't impact jenkins
* the check for new tests actually uses the /test/ file path against the list
of changed files in the check for new tests in unit test mode. This prevents
some false positives and should improve the count in certain/rare situations.
Prior to this, it just looked for (something)/test(something) in the actual
patch... which meant test-patch itself would be cleared. (Ironically,
test-patch.sh is now capable of testing itself, so ... yeah. test-ception.)
* in developer mode when \-\-resetrepo is used, the user output reminds that
the code the user chose to run in a destructive mode. (\-\-resetrepo is the
default in jenkins mode, obviously)
* Added code to chmod +x all directories in echo_and_redirect. This should
cover every maven run.
* Cleaned up the eclipse description.
* Checkstyle now uses the ${PROJECT_NAME} var instead of the hard coded hadoop
value, making the code more portable.
* Reworded some debug output for test subsystems when they are triggered
* verify_needed_test was missing documentation
bq. Adding audience / stability comments is a really nice touch.
Thanks. If you run shelldoc.py (after HADOOP-11845 is applied... ugh) against
the code, you'll get a markdown document of the function calls. We're currently
using this same functionality in trunk to document hadoop-functions.sh as the
'official' shell API for shell profiles.
was (Author: aw):
-18:
* add --modulelist and --testlist options to forcibly add modules and test
subsystems to a run. there is no way to delete, but that's probably a good
thing really.
* removed the branch restrictions since this apparently won't impact jenkins
* the check for new tests actually uses the /test/ file path against the list
of changed files in the check for new tests in unit test mode. This prevents
some false positives and should improve the count in certain/rare situations.
Prior to this, it just looked for (something)/test(something) in the actual
patch... which meant test-patch itself would be cleared. (Ironically,
test-patch.sh is now capable of testing itself, so ... yeah. test-ception.)
* in developer mode when --resetrepo is used, the user output reminds that the
code the user chose to run in a destructive mode. (--resetrepo is the default
in jenkins mode, obviously)
* Added code to chmod +x all directories in echo_and_redirect. This should
cover every maven run.
* Cleaned up the eclipse description.
* Checkstyle now uses the ${PROJECT_NAME} var instead of the hard coded hadoop
value, making the code more portable.
* Reworded some debug output for test subsystems when they are triggered
* verify_needed_test was missing documentation
bq. Adding audience / stability comments is a really nice touch.
Thanks. If you run shelldoc.py (after HADOOP-11845 is applied... ugh) against
the code, you'll get a markdown document of the function calls. We're currently
using this same functionality in trunk to document hadoop-functions.sh as the
'official' shell API for shell profiles.
> rewrite test-patch.sh
> -
>
> Key: HADOOP-11746
> URL: https://issues.apache.org/jira/browse/HADOOP-11746
> Project: Hadoop Common
> Issue Type: Test
> Components: build, test
>Affects Versions: 3.0.0
>Reporter: Allen Wittenauer
>Assignee: Allen Wittenauer
> Attachments: HADOOP-11746-00.patch, HADOOP-11746-01.patch,
> HADOOP-11746-02.patch, HADOOP-11746-03.patch, HADOOP-11746-04.patch,
> HADOOP-11746-05.patch, HADOOP-11746-06.patch, HADOOP-11746-07.patch,
> HADOOP-11746-09.patch, HADOOP-11746-10.patch, HADOOP-11746-11.patch,
> HADOOP-11746-12.patch, HADOOP-11746-13.patch, HADOOP-11746-14.patch,
> HADOOP-11746-15.patch, HADOOP-11746-16.patch, HADOOP-11746-17.patch,
> HADOOP-11746-18.patch
>
>
> This code is bad and you should feel bad.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11843) Make setting up the build environment easier
[ https://issues.apache.org/jira/browse/HADOOP-11843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niels Basjes updated HADOOP-11843: -- Attachment: HADOOP-11843-2015-04-17-2226.patch - Fixed shellcheck warnings - Fixed the motd point and added a filter to not show it on the screen. - Cleaned up whitespace - "for Hadoop" ... - Removed a lot of dependencies. I added most of these because they are explicitly mentioned as needed for specific parts of the build (like -Pnative) in BUILDING.TXT. Someone should update that. I do not have enough knowledge at this moment to do that. - Added a section to the BUILDING.TXT to point out this new feature. - Added all requested dependencies (shellcheck, findbugs and libcurl4-openssl-dev). > Make setting up the build environment easier > > > Key: HADOOP-11843 > URL: https://issues.apache.org/jira/browse/HADOOP-11843 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Niels Basjes >Assignee: Niels Basjes > Attachments: HADOOP-11843-2015-04-17-1612.patch, > HADOOP-11843-2015-04-17-2226.patch > > > ( As discussed with [~aw] ) > In AVRO-1537 a docker based solution was created to setup all the tools for > doing a full build. This enables much easier reproduction of any issues and > getting up and running for new developers. > This issue is to 'copy/port' that setup into the hadoop project in > preparation for the bug squash. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11843) Make setting up the build environment easier
[ https://issues.apache.org/jira/browse/HADOOP-11843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niels Basjes updated HADOOP-11843: -- Status: Open (was: Patch Available) > Make setting up the build environment easier > > > Key: HADOOP-11843 > URL: https://issues.apache.org/jira/browse/HADOOP-11843 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Niels Basjes >Assignee: Niels Basjes > Attachments: HADOOP-11843-2015-04-17-1612.patch, > HADOOP-11843-2015-04-17-2226.patch > > > ( As discussed with [~aw] ) > In AVRO-1537 a docker based solution was created to setup all the tools for > doing a full build. This enables much easier reproduction of any issues and > getting up and running for new developers. > This issue is to 'copy/port' that setup into the hadoop project in > preparation for the bug squash. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11843) Make setting up the build environment easier
[
https://issues.apache.org/jira/browse/HADOOP-11843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Niels Basjes updated HADOOP-11843:
--
Status: Patch Available (was: Open)
This patch succesfully ran
{code}mvn package -Pdist -Pdocs -Psrc -Pnative -Dtar -Preleasedocs
-DskipTests=true{code}
So everything except I didn't run the tests because that takes very long.
> Make setting up the build environment easier
>
>
> Key: HADOOP-11843
> URL: https://issues.apache.org/jira/browse/HADOOP-11843
> Project: Hadoop Common
> Issue Type: New Feature
>Reporter: Niels Basjes
>Assignee: Niels Basjes
> Attachments: HADOOP-11843-2015-04-17-1612.patch,
> HADOOP-11843-2015-04-17-2226.patch
>
>
> ( As discussed with [~aw] )
> In AVRO-1537 a docker based solution was created to setup all the tools for
> doing a full build. This enables much easier reproduction of any issues and
> getting up and running for new developers.
> This issue is to 'copy/port' that setup into the hadoop project in
> preparation for the bug squash.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11843) Make setting up the build environment easier
[ https://issues.apache.org/jira/browse/HADOOP-11843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14500644#comment-14500644 ] Niels Basjes commented on HADOOP-11843: --- P.S. I left Forrest in because I found several references in the code base. > Make setting up the build environment easier > > > Key: HADOOP-11843 > URL: https://issues.apache.org/jira/browse/HADOOP-11843 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Niels Basjes >Assignee: Niels Basjes > Attachments: HADOOP-11843-2015-04-17-1612.patch, > HADOOP-11843-2015-04-17-2226.patch > > > ( As discussed with [~aw] ) > In AVRO-1537 a docker based solution was created to setup all the tools for > doing a full build. This enables much easier reproduction of any issues and > getting up and running for new developers. > This issue is to 'copy/port' that setup into the hadoop project in > preparation for the bug squash. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11843) Make setting up the build environment easier
[ https://issues.apache.org/jira/browse/HADOOP-11843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14500660#comment-14500660 ] Chris Nauroth commented on HADOOP-11843: bq. I added most of these because they are explicitly mentioned as needed for specific parts of the build (like -Pnative) in BUILDING.TXT. Thanks for pointing that out. I took a look, and I think libprotobuf-dev is the only extraneous thing mentioned in BUILDING.txt. Would you mind removing that since you're already touching this file? > Make setting up the build environment easier > > > Key: HADOOP-11843 > URL: https://issues.apache.org/jira/browse/HADOOP-11843 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Niels Basjes >Assignee: Niels Basjes > Attachments: HADOOP-11843-2015-04-17-1612.patch, > HADOOP-11843-2015-04-17-2226.patch > > > ( As discussed with [~aw] ) > In AVRO-1537 a docker based solution was created to setup all the tools for > doing a full build. This enables much easier reproduction of any issues and > getting up and running for new developers. > This issue is to 'copy/port' that setup into the hadoop project in > preparation for the bug squash. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11843) Make setting up the build environment easier
[ https://issues.apache.org/jira/browse/HADOOP-11843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niels Basjes updated HADOOP-11843: -- Attachment: HADOOP-11843-2015-04-17-2308.patch Removed reference to libprotobuf-dev in BUILDING.txt > Make setting up the build environment easier > > > Key: HADOOP-11843 > URL: https://issues.apache.org/jira/browse/HADOOP-11843 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Niels Basjes >Assignee: Niels Basjes > Attachments: HADOOP-11843-2015-04-17-1612.patch, > HADOOP-11843-2015-04-17-2226.patch, HADOOP-11843-2015-04-17-2308.patch > > > ( As discussed with [~aw] ) > In AVRO-1537 a docker based solution was created to setup all the tools for > doing a full build. This enables much easier reproduction of any issues and > getting up and running for new developers. > This issue is to 'copy/port' that setup into the hadoop project in > preparation for the bug squash. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11746) rewrite test-patch.sh
[
https://issues.apache.org/jira/browse/HADOOP-11746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14500686#comment-14500686
]
Hadoop QA commented on HADOOP-11746:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12726255/HADOOP-11746-18.patch
against trunk revision d573f09.
{color:red}-1 @author{color}. The patch appears to contain 13 @author tags
which the Hadoop community has agreed to not allow in code contributions.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6118//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6118//console
This message is automatically generated.
> rewrite test-patch.sh
> -
>
> Key: HADOOP-11746
> URL: https://issues.apache.org/jira/browse/HADOOP-11746
> Project: Hadoop Common
> Issue Type: Test
> Components: build, test
>Affects Versions: 3.0.0
>Reporter: Allen Wittenauer
>Assignee: Allen Wittenauer
> Attachments: HADOOP-11746-00.patch, HADOOP-11746-01.patch,
> HADOOP-11746-02.patch, HADOOP-11746-03.patch, HADOOP-11746-04.patch,
> HADOOP-11746-05.patch, HADOOP-11746-06.patch, HADOOP-11746-07.patch,
> HADOOP-11746-09.patch, HADOOP-11746-10.patch, HADOOP-11746-11.patch,
> HADOOP-11746-12.patch, HADOOP-11746-13.patch, HADOOP-11746-14.patch,
> HADOOP-11746-15.patch, HADOOP-11746-16.patch, HADOOP-11746-17.patch,
> HADOOP-11746-18.patch
>
>
> This code is bad and you should feel bad.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11843) Make setting up the build environment easier
[
https://issues.apache.org/jira/browse/HADOOP-11843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14500706#comment-14500706
]
Hadoop QA commented on HADOOP-11843:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12726259/HADOOP-11843-2015-04-17-2226.patch
against trunk revision d573f09.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6119//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6119//console
This message is automatically generated.
> Make setting up the build environment easier
>
>
> Key: HADOOP-11843
> URL: https://issues.apache.org/jira/browse/HADOOP-11843
> Project: Hadoop Common
> Issue Type: New Feature
>Reporter: Niels Basjes
>Assignee: Niels Basjes
> Attachments: HADOOP-11843-2015-04-17-1612.patch,
> HADOOP-11843-2015-04-17-2226.patch, HADOOP-11843-2015-04-17-2308.patch
>
>
> ( As discussed with [~aw] )
> In AVRO-1537 a docker based solution was created to setup all the tools for
> doing a full build. This enables much easier reproduction of any issues and
> getting up and running for new developers.
> This issue is to 'copy/port' that setup into the hadoop project in
> preparation for the bug squash.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11843) Make setting up the build environment easier
[ https://issues.apache.org/jira/browse/HADOOP-11843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niels Basjes updated HADOOP-11843: -- Status: Patch Available (was: Open) > Make setting up the build environment easier > > > Key: HADOOP-11843 > URL: https://issues.apache.org/jira/browse/HADOOP-11843 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Niels Basjes >Assignee: Niels Basjes > Attachments: HADOOP-11843-2015-04-17-1612.patch, > HADOOP-11843-2015-04-17-2226.patch, HADOOP-11843-2015-04-17-2308.patch > > > ( As discussed with [~aw] ) > In AVRO-1537 a docker based solution was created to setup all the tools for > doing a full build. This enables much easier reproduction of any issues and > getting up and running for new developers. > This issue is to 'copy/port' that setup into the hadoop project in > preparation for the bug squash. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11843) Make setting up the build environment easier
[ https://issues.apache.org/jira/browse/HADOOP-11843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niels Basjes updated HADOOP-11843: -- Status: Open (was: Patch Available) > Make setting up the build environment easier > > > Key: HADOOP-11843 > URL: https://issues.apache.org/jira/browse/HADOOP-11843 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Niels Basjes >Assignee: Niels Basjes > Attachments: HADOOP-11843-2015-04-17-1612.patch, > HADOOP-11843-2015-04-17-2226.patch, HADOOP-11843-2015-04-17-2308.patch > > > ( As discussed with [~aw] ) > In AVRO-1537 a docker based solution was created to setup all the tools for > doing a full build. This enables much easier reproduction of any issues and > getting up and running for new developers. > This issue is to 'copy/port' that setup into the hadoop project in > preparation for the bug squash. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11843) Make setting up the build environment easier
[
https://issues.apache.org/jira/browse/HADOOP-11843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14500743#comment-14500743
]
Hadoop QA commented on HADOOP-11843:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12726263/HADOOP-11843-2015-04-17-2308.patch
against trunk revision d573f09.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6120//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6120//console
This message is automatically generated.
> Make setting up the build environment easier
>
>
> Key: HADOOP-11843
> URL: https://issues.apache.org/jira/browse/HADOOP-11843
> Project: Hadoop Common
> Issue Type: New Feature
>Reporter: Niels Basjes
>Assignee: Niels Basjes
> Attachments: HADOOP-11843-2015-04-17-1612.patch,
> HADOOP-11843-2015-04-17-2226.patch, HADOOP-11843-2015-04-17-2308.patch
>
>
> ( As discussed with [~aw] )
> In AVRO-1537 a docker based solution was created to setup all the tools for
> doing a full build. This enables much easier reproduction of any issues and
> getting up and running for new developers.
> This issue is to 'copy/port' that setup into the hadoop project in
> preparation for the bug squash.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11841) Remove ecschema-def.xml
[ https://issues.apache.org/jira/browse/HADOOP-11841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14500823#comment-14500823 ] Jing Zhao commented on HADOOP-11841: +1 > Remove ecschema-def.xml > --- > > Key: HADOOP-11841 > URL: https://issues.apache.org/jira/browse/HADOOP-11841 > Project: Hadoop Common > Issue Type: Sub-task > Components: io >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Attachments: c11841_20150416.patch > > > Currently ecschema-def.xml is only used by a test. It is not clear if the > file is needed and how to use that file. Let's remove it for HDFS-7285. We > may add it back later on if we decide that it is required. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HADOOP-11841) Remove ecschema-def.xml
[ https://issues.apache.org/jira/browse/HADOOP-11841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze resolved HADOOP-11841. -- Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed Thanks Jing for reviewing the patch. I have committed this. > Remove ecschema-def.xml > --- > > Key: HADOOP-11841 > URL: https://issues.apache.org/jira/browse/HADOOP-11841 > Project: Hadoop Common > Issue Type: Sub-task > Components: io >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Fix For: HDFS-7285 > > Attachments: c11841_20150416.patch > > > Currently ecschema-def.xml is only used by a test. It is not clear if the > file is needed and how to use that file. Let's remove it for HDFS-7285. We > may add it back later on if we decide that it is required. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11127) Improve versioning and compatibility support in native library for downstream hadoop-common users.
[ https://issues.apache.org/jira/browse/HADOOP-11127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14500970#comment-14500970 ] Colin Patrick McCabe commented on HADOOP-11127: --- I was thinking we: 1. Add the Hadoop release version to libhadoop.so. It's very, very simple and solves a lot of problems here. 2. Remove libhadoop.so and libhdfs.so from the release tarball, since they are CPU and OS-specific and the tarballs are not 3. Schedule some follow-on work to include the native libraries inside jars, as Chris suggested. This will take longer but ultimately be the best solution. > Improve versioning and compatibility support in native library for downstream > hadoop-common users. > -- > > Key: HADOOP-11127 > URL: https://issues.apache.org/jira/browse/HADOOP-11127 > Project: Hadoop Common > Issue Type: Bug > Components: native >Reporter: Chris Nauroth > Attachments: HADOOP-11064.003.patch > > > There is no compatibility policy enforced on the JNI function signatures > implemented in the native library. This library typically is deployed to all > nodes in a cluster, built from a specific source code version. However, > downstream applications that want to run in that cluster might choose to > bundle a hadoop-common jar at a different version. Since there is no > compatibility policy, this can cause link errors at runtime when the native > function signatures expected by hadoop-common.jar do not exist in > libhadoop.so/hadoop.dll. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
