[jira] [Comment Edited] (HADOOP-11361) Fix a race condition in MetricsSourceAdapter.updateJmxCache
[
https://issues.apache.org/jira/browse/HADOOP-11361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372166#comment-15372166
]
Tsuyoshi Ozawa edited comment on HADOOP-11361 at 7/12/16 4:44 AM:
--
[~yzhangal] +1 for the idea. It avoid the deadlock between MetricsSystemImpl's
lock and MetricsSourceAdapter's lock
{{getMetricsPart1}} should be named as {{getMetrics}}(), and
{{getMetricsPart2}} should be renamed as {{updateAndGetLastRecs}}, since it's
more straightforward. Could you update the patch?
[~brahmareddy] [~vinayrpet] could you review [~yzhangal]'s patch if you have
time? I would like to double-check new patch if possible.
was (Author: ozawa):
[~yzhangal] +1 for the idea. It avoid the deadlock between MetricsSystemImpl's
lock and MetricsSourceAdapter's lock
{{getMetricsPart1}} should be named as {{getMetrics}}(), and
{{getMetricsPart2}} should be renamed as {{updateAndGetLastRecs}}, since it's
more straightforward. Could you update the patch?
> Fix a race condition in MetricsSourceAdapter.updateJmxCache
> ---
>
> Key: HADOOP-11361
> URL: https://issues.apache.org/jira/browse/HADOOP-11361
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.4.1, 2.5.1, 2.6.0
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Attachments: HADOOP-111361-003.patch, HADOOP-11361-002.patch,
> HADOOP-11361-004.patch, HADOOP-11361.patch, HDFS-7487.patch
>
>
> {noformat}
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateAttrCache(MetricsSourceAdapter.java:247)
> at
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:177)
> at
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getAttribute(MetricsSourceAdapter.java:102)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Issue Comment Deleted] (HADOOP-11361) Fix a race condition in MetricsSourceAdapter.updateJmxCache
[
https://issues.apache.org/jira/browse/HADOOP-11361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-11361:
Comment: was deleted
(was: [~brahmareddy] could you work with [~yzhangal]?)
> Fix a race condition in MetricsSourceAdapter.updateJmxCache
> ---
>
> Key: HADOOP-11361
> URL: https://issues.apache.org/jira/browse/HADOOP-11361
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.4.1, 2.5.1, 2.6.0
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Attachments: HADOOP-111361-003.patch, HADOOP-11361-002.patch,
> HADOOP-11361-004.patch, HADOOP-11361.patch, HDFS-7487.patch
>
>
> {noformat}
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateAttrCache(MetricsSourceAdapter.java:247)
> at
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:177)
> at
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getAttribute(MetricsSourceAdapter.java:102)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11361) Fix a race condition in MetricsSourceAdapter.updateJmxCache
[
https://issues.apache.org/jira/browse/HADOOP-11361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372167#comment-15372167
]
Tsuyoshi Ozawa commented on HADOOP-11361:
-
[~brahmareddy] could you work with [~yzhangal]?
> Fix a race condition in MetricsSourceAdapter.updateJmxCache
> ---
>
> Key: HADOOP-11361
> URL: https://issues.apache.org/jira/browse/HADOOP-11361
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.4.1, 2.5.1, 2.6.0
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Attachments: HADOOP-111361-003.patch, HADOOP-11361-002.patch,
> HADOOP-11361-004.patch, HADOOP-11361.patch, HDFS-7487.patch
>
>
> {noformat}
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateAttrCache(MetricsSourceAdapter.java:247)
> at
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:177)
> at
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getAttribute(MetricsSourceAdapter.java:102)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11361) Fix a race condition in MetricsSourceAdapter.updateJmxCache
[
https://issues.apache.org/jira/browse/HADOOP-11361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372166#comment-15372166
]
Tsuyoshi Ozawa commented on HADOOP-11361:
-
[~yzhangal] +1 for the idea. It avoid the deadlock between MetricsSystemImpl's
lock and MetricsSourceAdapter's lock
{{getMetricsPart1}} should be named as {{getMetrics}}(), and
{{getMetricsPart2}} should be renamed as {{updateAndGetLastRecs}}, since it's
more straightforward. Could you update the patch?
> Fix a race condition in MetricsSourceAdapter.updateJmxCache
> ---
>
> Key: HADOOP-11361
> URL: https://issues.apache.org/jira/browse/HADOOP-11361
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.4.1, 2.5.1, 2.6.0
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Attachments: HADOOP-111361-003.patch, HADOOP-11361-002.patch,
> HADOOP-11361-004.patch, HADOOP-11361.patch, HDFS-7487.patch
>
>
> {noformat}
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateAttrCache(MetricsSourceAdapter.java:247)
> at
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:177)
> at
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getAttribute(MetricsSourceAdapter.java:102)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13290) Appropriate use of generics in FairCallQueue
[
https://issues.apache.org/jira/browse/HADOOP-13290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372125#comment-15372125
]
Hadoop QA commented on HADOOP-13290:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
23s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 44m 8s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12817321/HADOOP-13290.002.patch
|
| JIRA Issue | HADOOP-13290 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux fdbbe8aa5bd9 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / f292624 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9964/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9964/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Appropriate use of generics in FairCallQueue
>
>
> Key: HADOOP-13290
> URL: https://issues.apache.org/jira/browse/HADOOP-13290
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Affects Versions: 2.6.0
>Reporter: Konstantin Shvachko
>Assignee: Jonathan Hung
> Labels: newbie++
> Attachments: HADOOP-13290.001.patch, HADOOP-13290.002.patch
>
>
> # {{BlockingQueue}} is intermittently used with and without generic
> parameters in
[jira] [Commented] (HADOOP-13240) TestAclCommands.testSetfaclValidations fail
[
https://issues.apache.org/jira/browse/HADOOP-13240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372105#comment-15372105
]
Hadoop QA commented on HADOOP-13240:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
16s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 23s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 6 new + 39 unchanged - 6 fixed = 45 total (was 45) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
11s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 39m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12817320/HADOOP-13240.001.patch
|
| JIRA Issue | HADOOP-13240 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux e7f3453b523e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / f292624 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9963/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9963/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9963/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> TestAclCommands.testSetfaclValidations fail
> ---
>
> Key: HADOOP-13240
> URL: https://issues.apache.org/jira/browse/HADOOP-13240
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.4.1, 2.7.1
> Environment: hadoop 2.4.1,as6.5
>
[jira] [Commented] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[
https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372096#comment-15372096
]
Tsuyoshi Ozawa commented on HADOOP-13363:
-
{quote}
This merits a discussion on common-dev for visibility.
{quote}
Sure.
{quote}
What is going to happen when code generated by protoc 2.5 tries to run against
2.6 library? Has anyone tested this? Because thats the metric of how traumatic
this is going to be
{quote}
We must test it before merging it, of course. IIUC, protobuf is wire compatible
between 2.5 and 2.6. I found a test suite which seems useful here:
https://chromium.googlesource.com/external/github.com/google/protobuf/+/master/java/compatibility_tests/README.md
We also need to check code-level compatibility.
> Upgrade protobuf from 2.5.0 to something newer
> --
>
> Key: HADOOP-13363
> URL: https://issues.apache.org/jira/browse/HADOOP-13363
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Affects Versions: 3.0.0-alpha1
>Reporter: Allen Wittenauer
> Attachments: HADOOP-13363.001.patch
>
>
> Standard protobuf 2.5.0 does not work properly on many platforms. (See, for
> example, https://gist.github.com/BennettSmith/7111094 ). In order for us to
> avoid crazy work arounds in the build environment and the fact that 2.5.0 is
> starting to slowly disappear as a standard install-able package for even
> Linux/x86, we need to either upgrade or self bundle or something else.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[
https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372096#comment-15372096
]
Tsuyoshi Ozawa edited comment on HADOOP-13363 at 7/12/16 2:50 AM:
--
{quote}
This merits a discussion on common-dev for visibility. There was a discussion
before but no conclusion iirc.
{quote}
Sure.
{quote}
What is going to happen when code generated by protoc 2.5 tries to run against
2.6 library? Has anyone tested this? Because thats the metric of how traumatic
this is going to be
{quote}
We must test it before merging it, of course. IIUC, protobuf is wire compatible
between 2.5 and 2.6. I found a test suite which seems useful here:
https://chromium.googlesource.com/external/github.com/google/protobuf/+/master/java/compatibility_tests/README.md
We also need to check code-level compatibility.
was (Author: ozawa):
{quote}
This merits a discussion on common-dev for visibility.
{quote}
Sure.
{quote}
What is going to happen when code generated by protoc 2.5 tries to run against
2.6 library? Has anyone tested this? Because thats the metric of how traumatic
this is going to be
{quote}
We must test it before merging it, of course. IIUC, protobuf is wire compatible
between 2.5 and 2.6. I found a test suite which seems useful here:
https://chromium.googlesource.com/external/github.com/google/protobuf/+/master/java/compatibility_tests/README.md
We also need to check code-level compatibility.
> Upgrade protobuf from 2.5.0 to something newer
> --
>
> Key: HADOOP-13363
> URL: https://issues.apache.org/jira/browse/HADOOP-13363
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Affects Versions: 3.0.0-alpha1
>Reporter: Allen Wittenauer
> Attachments: HADOOP-13363.001.patch
>
>
> Standard protobuf 2.5.0 does not work properly on many platforms. (See, for
> example, https://gist.github.com/BennettSmith/7111094 ). In order for us to
> avoid crazy work arounds in the build environment and the fact that 2.5.0 is
> starting to slowly disappear as a standard install-able package for even
> Linux/x86, we need to either upgrade or self bundle or something else.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13290) Appropriate use of generics in FairCallQueue
[
https://issues.apache.org/jira/browse/HADOOP-13290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372081#comment-15372081
]
Jonathan Hung commented on HADOOP-13290:
Ah. Thanks. Just uploaded a new patch and triggered jenkins build.
> Appropriate use of generics in FairCallQueue
>
>
> Key: HADOOP-13290
> URL: https://issues.apache.org/jira/browse/HADOOP-13290
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Affects Versions: 2.6.0
>Reporter: Konstantin Shvachko
>Assignee: Jonathan Hung
> Labels: newbie++
> Attachments: HADOOP-13290.001.patch, HADOOP-13290.002.patch
>
>
> # {{BlockingQueue}} is intermittently used with and without generic
> parameters in {{FairCallQueue}} class. Should be parameterized.
> # Same for {{FairCallQueue}}. Should be parameterized. Could be a bit more
> tricky for that one.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13290) Appropriate use of generics in FairCallQueue
[
https://issues.apache.org/jira/browse/HADOOP-13290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hung updated HADOOP-13290:
---
Status: Patch Available (was: Open)
> Appropriate use of generics in FairCallQueue
>
>
> Key: HADOOP-13290
> URL: https://issues.apache.org/jira/browse/HADOOP-13290
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Affects Versions: 2.6.0
>Reporter: Konstantin Shvachko
>Assignee: Jonathan Hung
> Labels: newbie++
> Attachments: HADOOP-13290.001.patch, HADOOP-13290.002.patch
>
>
> # {{BlockingQueue}} is intermittently used with and without generic
> parameters in {{FairCallQueue}} class. Should be parameterized.
> # Same for {{FairCallQueue}}. Should be parameterized. Could be a bit more
> tricky for that one.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13290) Appropriate use of generics in FairCallQueue
[
https://issues.apache.org/jira/browse/HADOOP-13290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hung updated HADOOP-13290:
---
Attachment: HADOOP-13290.002.patch
> Appropriate use of generics in FairCallQueue
>
>
> Key: HADOOP-13290
> URL: https://issues.apache.org/jira/browse/HADOOP-13290
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Affects Versions: 2.6.0
>Reporter: Konstantin Shvachko
>Assignee: Jonathan Hung
> Labels: newbie++
> Attachments: HADOOP-13290.001.patch, HADOOP-13290.002.patch
>
>
> # {{BlockingQueue}} is intermittently used with and without generic
> parameters in {{FairCallQueue}} class. Should be parameterized.
> # Same for {{FairCallQueue}}. Should be parameterized. Could be a bit more
> tricky for that one.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13290) Appropriate use of generics in FairCallQueue
[
https://issues.apache.org/jira/browse/HADOOP-13290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hung updated HADOOP-13290:
---
Status: Open (was: Patch Available)
> Appropriate use of generics in FairCallQueue
>
>
> Key: HADOOP-13290
> URL: https://issues.apache.org/jira/browse/HADOOP-13290
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Affects Versions: 2.6.0
>Reporter: Konstantin Shvachko
>Assignee: Jonathan Hung
> Labels: newbie++
> Attachments: HADOOP-13290.001.patch, HADOOP-13290.002.patch
>
>
> # {{BlockingQueue}} is intermittently used with and without generic
> parameters in {{FairCallQueue}} class. Should be parameterized.
> # Same for {{FairCallQueue}}. Should be parameterized. Could be a bit more
> tricky for that one.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13240) TestAclCommands.testSetfaclValidations fail
[
https://issues.apache.org/jira/browse/HADOOP-13240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Zhuge updated HADOOP-13240:
Target Version/s: 2.8.0
Status: Patch Available (was: Open)
> TestAclCommands.testSetfaclValidations fail
> ---
>
> Key: HADOOP-13240
> URL: https://issues.apache.org/jira/browse/HADOOP-13240
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.7.1, 2.4.1
> Environment: hadoop 2.4.1,as6.5
>Reporter: linbao111
>Assignee: John Zhuge
>Priority: Minor
> Attachments: HADOOP-13240.001.patch
>
>
> mvn test -Djava.net.preferIPv4Stack=true -Dlog4j.rootLogger=DEBUG,console
> -Dtest=TestAclCommands#testSetfaclValidations failed with following message:
> ---
> Test set: org.apache.hadoop.fs.shell.TestAclCommands
> ---
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.599 sec <<<
> FAILURE! - in org.apache.hadoop.fs.shell.TestAclCommands
> testSetfaclValidations(org.apache.hadoop.fs.shell.TestAclCommands) Time
> elapsed: 0.534 sec <<< FAILURE!
> java.lang.AssertionError: setfacl should fail ACL spec missing
> at org.junit.Assert.fail(Assert.java:93)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at org.junit.Assert.assertFalse(Assert.java:68)
> at
> org.apache.hadoop.fs.shell.TestAclCommands.testSetfaclValidations(TestAclCommands.java:81)
> i notice from
> HADOOP-10277,hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/AclEntry.java
> code changed
> should
> hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.javabe
> changed to:
> diff --git
> a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.java
>
> b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.java
> index b14cd37..463bfcd
> ---
> a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.java
> +++
> b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.java
> @@ -80,7 +80,7 @@ public void testSetfaclValidations() throws Exception {
> "/path" }));
> assertFalse("setfacl should fail ACL spec missing",
> 0 == runCommand(new String[] { "-setfacl", "-m",
> -"", "/path" }));
> +":", "/path" }));
>}
>
>@Test
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13240) TestAclCommands.testSetfaclValidations fail
[
https://issues.apache.org/jira/browse/HADOOP-13240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Zhuge updated HADOOP-13240:
Attachment: HADOOP-13240.001.patch
Patch 001:
* Report error when is empty for {{setfacl -m}}
* Fix {{TestAclCommands}} to use existing file path instead of {{/path}}
> TestAclCommands.testSetfaclValidations fail
> ---
>
> Key: HADOOP-13240
> URL: https://issues.apache.org/jira/browse/HADOOP-13240
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.4.1, 2.7.1
> Environment: hadoop 2.4.1,as6.5
>Reporter: linbao111
>Assignee: John Zhuge
>Priority: Minor
> Attachments: HADOOP-13240.001.patch
>
>
> mvn test -Djava.net.preferIPv4Stack=true -Dlog4j.rootLogger=DEBUG,console
> -Dtest=TestAclCommands#testSetfaclValidations failed with following message:
> ---
> Test set: org.apache.hadoop.fs.shell.TestAclCommands
> ---
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.599 sec <<<
> FAILURE! - in org.apache.hadoop.fs.shell.TestAclCommands
> testSetfaclValidations(org.apache.hadoop.fs.shell.TestAclCommands) Time
> elapsed: 0.534 sec <<< FAILURE!
> java.lang.AssertionError: setfacl should fail ACL spec missing
> at org.junit.Assert.fail(Assert.java:93)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at org.junit.Assert.assertFalse(Assert.java:68)
> at
> org.apache.hadoop.fs.shell.TestAclCommands.testSetfaclValidations(TestAclCommands.java:81)
> i notice from
> HADOOP-10277,hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/AclEntry.java
> code changed
> should
> hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.javabe
> changed to:
> diff --git
> a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.java
>
> b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.java
> index b14cd37..463bfcd
> ---
> a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.java
> +++
> b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.java
> @@ -80,7 +80,7 @@ public void testSetfaclValidations() throws Exception {
> "/path" }));
> assertFalse("setfacl should fail ACL spec missing",
> 0 == runCommand(new String[] { "-setfacl", "-m",
> -"", "/path" }));
> +":", "/path" }));
>}
>
>@Test
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13240) TestAclCommands.testSetfaclValidations fail
[
https://issues.apache.org/jira/browse/HADOOP-13240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372062#comment-15372062
]
John Zhuge commented on HADOOP-13240:
-
The {{TestAclCommands.testSetfaclValidations}} tests should use an existing
file in test root instead of {{/path}} because there is no guarantee which
error gets reported when there are 2 errors in command options.
There is bug in {{setfacl -m "" }} validation code. At least one
valid ACL entry is expected for option {{-m or -x or --set}}. Please see
{{FileSystemShell.md}}. And command {{setfacl -m "" }} returns
error on Linux.
> TestAclCommands.testSetfaclValidations fail
> ---
>
> Key: HADOOP-13240
> URL: https://issues.apache.org/jira/browse/HADOOP-13240
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.4.1, 2.7.1
> Environment: hadoop 2.4.1,as6.5
>Reporter: linbao111
>Assignee: John Zhuge
>Priority: Minor
>
> mvn test -Djava.net.preferIPv4Stack=true -Dlog4j.rootLogger=DEBUG,console
> -Dtest=TestAclCommands#testSetfaclValidations failed with following message:
> ---
> Test set: org.apache.hadoop.fs.shell.TestAclCommands
> ---
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.599 sec <<<
> FAILURE! - in org.apache.hadoop.fs.shell.TestAclCommands
> testSetfaclValidations(org.apache.hadoop.fs.shell.TestAclCommands) Time
> elapsed: 0.534 sec <<< FAILURE!
> java.lang.AssertionError: setfacl should fail ACL spec missing
> at org.junit.Assert.fail(Assert.java:93)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at org.junit.Assert.assertFalse(Assert.java:68)
> at
> org.apache.hadoop.fs.shell.TestAclCommands.testSetfaclValidations(TestAclCommands.java:81)
> i notice from
> HADOOP-10277,hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/AclEntry.java
> code changed
> should
> hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.javabe
> changed to:
> diff --git
> a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.java
>
> b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.java
> index b14cd37..463bfcd
> ---
> a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.java
> +++
> b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.java
> @@ -80,7 +80,7 @@ public void testSetfaclValidations() throws Exception {
> "/path" }));
> assertFalse("setfacl should fail ACL spec missing",
> 0 == runCommand(new String[] { "-setfacl", "-m",
> -"", "/path" }));
> +":", "/path" }));
>}
>
>@Test
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12893) Verify LICENSE.txt and NOTICE.txt
[ https://issues.apache.org/jira/browse/HADOOP-12893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372040#comment-15372040 ] Andrew Wang commented on HADOOP-12893: -- LEGAL-262 has been resolved, consensus was to just use the NOTICE files as is (which is what we already did). So I think we're good on that front. > Verify LICENSE.txt and NOTICE.txt > - > > Key: HADOOP-12893 > URL: https://issues.apache.org/jira/browse/HADOOP-12893 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.8.0, 2.7.3, 2.6.5, 3.0.0-alpha1 >Reporter: Allen Wittenauer >Assignee: Xiao Chen >Priority: Blocker > Fix For: 2.7.3, 2.6.5 > > Attachments: HADOOP-12893-addendum-branch-2.7.01.patch, > HADOOP-12893.002.patch, HADOOP-12893.003.patch, HADOOP-12893.004.patch, > HADOOP-12893.005.patch, HADOOP-12893.006.patch, HADOOP-12893.007.patch, > HADOOP-12893.008.patch, HADOOP-12893.009.patch, HADOOP-12893.01.patch, > HADOOP-12893.011.patch, HADOOP-12893.012.patch, HADOOP-12893.10.patch, > HADOOP-12893.addendum-trunk.01.patch, HADOOP-12893.branch-2.01.patch, > HADOOP-12893.branch-2.6.01.patch, HADOOP-12893.branch-2.7.01.patch, > HADOOP-12893.branch-2.7.02.patch, HADOOP-12893.branch-2.7.3.01.patch > > > We have many bundled dependencies in both the source and the binary artifacts > that are not in LICENSE.txt and NOTICE.txt. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13290) Appropriate use of generics in FairCallQueue
[
https://issues.apache.org/jira/browse/HADOOP-13290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372037#comment-15372037
]
Konstantin Shvachko commented on HADOOP-13290:
--
Looks like you made a line longer than 80 symbols, therefore the checkstyle
warning.
> Appropriate use of generics in FairCallQueue
>
>
> Key: HADOOP-13290
> URL: https://issues.apache.org/jira/browse/HADOOP-13290
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Affects Versions: 2.6.0
>Reporter: Konstantin Shvachko
>Assignee: Jonathan Hung
> Labels: newbie++
> Attachments: HADOOP-13290.001.patch
>
>
> # {{BlockingQueue}} is intermittently used with and without generic
> parameters in {{FairCallQueue}} class. Should be parameterized.
> # Same for {{FairCallQueue}}. Should be parameterized. Could be a bit more
> tricky for that one.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13297) Add missing dependency in setting maven-remote-resource-plugin to fix builds
[
https://issues.apache.org/jira/browse/HADOOP-13297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371987#comment-15371987
]
Hudson commented on HADOOP-13297:
-
SUCCESS: Integrated in Hadoop-trunk-Commit #10076 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/10076/])
HADOOP-13297. Add missing dependency in setting (aajisaka: rev
7bd5d4272cd686e06c5d5fcc489b69312dacb47b)
* hadoop-project/pom.xml
> Add missing dependency in setting maven-remote-resource-plugin to fix builds
>
>
> Key: HADOOP-13297
> URL: https://issues.apache.org/jira/browse/HADOOP-13297
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 2.7.3, 2.6.5
>Reporter: Akira Ajisaka
>Assignee: Sean Busbey
> Fix For: 2.8.0, 2.7.3, 2.6.5
>
> Attachments: HADOOP-13297.00.patch, HADOOP-13297.1.patch,
> HADOOP-13297.branch-2.6.00.patch, HADOOP-13297.branch-2.6.v1.patch
>
>
> After HADOOP-12893, we are seeing {{mvn install -DskipTests}} failing in
> branch-2.7, branch-2.7.3, and branch-2.6. This failure is caused by the
> followings
> * hadoop-project module depends on hadoop-build-tools module, but
> hadoop-project module does not declare hadoop-build-tools as its submodule.
> Therefore, hadoop-build-tools is not built before building hadoop-project.
> * hadoop-build-tools pom and jar are not uploaded to the snapshot repository
> (https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-build-tools/)
> The build failure occurs if the *both* of the above conditions are satisfied.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13297) Add missing dependency in setting maven-remote-resource-plugin to fix builds
[
https://issues.apache.org/jira/browse/HADOOP-13297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-13297:
---
Resolution: Fixed
Fix Version/s: 2.6.5
2.7.3
2.8.0
Status: Resolved (was: Patch Available)
Committed this to all the related branches including branch-2.7.3. Thanks
[~busbey] for the very nice fix and thanks [~xiaochen] for the review.
> Add missing dependency in setting maven-remote-resource-plugin to fix builds
>
>
> Key: HADOOP-13297
> URL: https://issues.apache.org/jira/browse/HADOOP-13297
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 2.7.3, 2.6.5
>Reporter: Akira Ajisaka
>Assignee: Sean Busbey
> Fix For: 2.8.0, 2.7.3, 2.6.5
>
> Attachments: HADOOP-13297.00.patch, HADOOP-13297.1.patch,
> HADOOP-13297.branch-2.6.00.patch, HADOOP-13297.branch-2.6.v1.patch
>
>
> After HADOOP-12893, we are seeing {{mvn install -DskipTests}} failing in
> branch-2.7, branch-2.7.3, and branch-2.6. This failure is caused by the
> followings
> * hadoop-project module depends on hadoop-build-tools module, but
> hadoop-project module does not declare hadoop-build-tools as its submodule.
> Therefore, hadoop-build-tools is not built before building hadoop-project.
> * hadoop-build-tools pom and jar are not uploaded to the snapshot repository
> (https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-build-tools/)
> The build failure occurs if the *both* of the above conditions are satisfied.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13297) Add missing dependency in setting maven-remote-resource-plugin to fix builds
[
https://issues.apache.org/jira/browse/HADOOP-13297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-13297:
---
Hadoop Flags: Reviewed
Summary: Add missing dependency in setting
maven-remote-resource-plugin to fix builds (was: hadoop-common module depends
on hadoop-build-tools module, but the modules are not ordered correctly)
> Add missing dependency in setting maven-remote-resource-plugin to fix builds
>
>
> Key: HADOOP-13297
> URL: https://issues.apache.org/jira/browse/HADOOP-13297
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 2.7.3, 2.6.5
>Reporter: Akira Ajisaka
>Assignee: Sean Busbey
> Attachments: HADOOP-13297.00.patch, HADOOP-13297.1.patch,
> HADOOP-13297.branch-2.6.00.patch, HADOOP-13297.branch-2.6.v1.patch
>
>
> After HADOOP-12893, we are seeing {{mvn install -DskipTests}} failing in
> branch-2.7, branch-2.7.3, and branch-2.6. This failure is caused by the
> followings
> * hadoop-project module depends on hadoop-build-tools module, but
> hadoop-project module does not declare hadoop-build-tools as its submodule.
> Therefore, hadoop-build-tools is not built before building hadoop-project.
> * hadoop-build-tools pom and jar are not uploaded to the snapshot repository
> (https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-build-tools/)
> The build failure occurs if the *both* of the above conditions are satisfied.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13254) Create framework for configurable disk checkers
[
https://issues.apache.org/jira/browse/HADOOP-13254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371918#comment-15371918
]
Robert Kanter commented on HADOOP-13254:
I wasn't meaning to overly complicate things here. My concern was that if
someone went and made their own plugin, that we don't want them to cause
problems in the NM. If we're going to restrict the plugins to only
Hadoop-supplied ones, which it looks like we are (re-looking at the code, I see
that we have Private audience and Unstable on the {{DiskValidator}} interface),
then I think it's fine to not worry about Threads, being super defensive, and
those sorts of things for now.
Given that, I'm +1 on the 008 patch. I'll commit this tomorrow if nobody has
any other comments.
> Create framework for configurable disk checkers
> ---
>
> Key: HADOOP-13254
> URL: https://issues.apache.org/jira/browse/HADOOP-13254
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Yufei Gu
>Assignee: Yufei Gu
> Attachments: HADOOP-13254.001.patch, HADOOP-13254.002.patch,
> HADOOP-13254.003.patch, HADOOP-13254.004.patch, HADOOP-13254.005.patch,
> HADOOP-13254.006.patch, HADOOP-13254.007.patch, HADOOP-13254.008.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13043) Add LICENSE.txt entries for bundled javascript dependencies
[ https://issues.apache.org/jira/browse/HADOOP-13043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371848#comment-15371848 ] Andrew Wang commented on HADOOP-13043: -- Seems like I forgot to "git push" just branch-2.7 when I committed this. Sorry. On looking further, it looks like Akira's later backport of HADOOP-12893 to branch-2.7 fixed this up. It looks like this was also carried forward to branch-2.7.3. Should I make a dummy commit so grepping "git log" works? > Add LICENSE.txt entries for bundled javascript dependencies > --- > > Key: HADOOP-13043 > URL: https://issues.apache.org/jira/browse/HADOOP-13043 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.6.4 >Reporter: Andrew Wang >Assignee: Andrew Wang > Fix For: 2.8.0, 2.7.3, 2.6.5 > > Attachments: hadoop-13043.001.patch, hadoop-13043.002.patch > > > None of our bundled javascript dependencies are mentioned in LICENSE.txt. > Let's fix that. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13297) hadoop-common module depends on hadoop-build-tools module, but the modules are not ordered correctly
[
https://issues.apache.org/jira/browse/HADOOP-13297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371830#comment-15371830
]
Xiao Chen commented on HADOOP-13297:
Thanks Sean for the nice fix and educational (to me at least) trouble-shooting
details!
+1 (non-binding), verified this builds good against branch-2.6 (commit tip
e64094484de0e47850148d4c33ca3c189c93c01c)
> hadoop-common module depends on hadoop-build-tools module, but the modules
> are not ordered correctly
>
>
> Key: HADOOP-13297
> URL: https://issues.apache.org/jira/browse/HADOOP-13297
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 2.7.3, 2.6.5
>Reporter: Akira Ajisaka
>Assignee: Sean Busbey
> Attachments: HADOOP-13297.00.patch, HADOOP-13297.1.patch,
> HADOOP-13297.branch-2.6.00.patch, HADOOP-13297.branch-2.6.v1.patch
>
>
> After HADOOP-12893, we are seeing {{mvn install -DskipTests}} failing in
> branch-2.7, branch-2.7.3, and branch-2.6. This failure is caused by the
> followings
> * hadoop-project module depends on hadoop-build-tools module, but
> hadoop-project module does not declare hadoop-build-tools as its submodule.
> Therefore, hadoop-build-tools is not built before building hadoop-project.
> * hadoop-build-tools pom and jar are not uploaded to the snapshot repository
> (https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-build-tools/)
> The build failure occurs if the *both* of the above conditions are satisfied.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371819#comment-15371819 ] Thomas Poepping commented on HADOOP-13344: -- I'm sorry Allen, I don't think I understand how that makes this a script and pom patch for any poms but the root? Wouldn't we just change the directory structure and where we put the slf4j binding, then change the script to optionally *not* add that directory to the classpath based on a variable ($HADOOP_REMOVE_SLF4J_BINDING maybe) ? Thanks for the help. > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-12420) While trying to access Amazon S3 through hadoop-aws(Spark basically) I was getting Exception in thread "main" java.lang.NoSuchMethodError: com.amazonaws.services
[
https://issues.apache.org/jira/browse/HADOOP-12420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371807#comment-15371807
]
Mitesh edited comment on HADOOP-12420 at 7/11/16 10:37 PM:
---
Hadoop 2.8 still hasn't even RC from what I can tell. What workaround is
everyone using? I'm seeing lots of intermittent timeout/connection issues in
Spark that I am fairly sure due to being stuck at
hadoop-2.7.2/aws-java-sdk-1.7.4.
was (Author: masterddt):
Hadoop 2.8 still hasn't even RC from what I can tell. What workaround is
everyone using? I'm seeing lots of intermittent timeout/connection issues that
I am fairly sure due to being stuck at hadoop-2.7.2/aws-java-sdk-1.7.4.
> While trying to access Amazon S3 through hadoop-aws(Spark basically) I was
> getting Exception in thread "main" java.lang.NoSuchMethodError:
> com.amazonaws.services.s3.transfer.TransferManagerConfiguration.setMultipartUploadThreshold(I)V
> --
>
> Key: HADOOP-12420
> URL: https://issues.apache.org/jira/browse/HADOOP-12420
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs/s3
>Affects Versions: 2.7.1
>Reporter: Tariq Mohammad
>Assignee: Tariq Mohammad
>Priority: Minor
> Fix For: 2.8.0
>
>
> While trying to access data stored in Amazon S3 through Apache Spark, which
> internally uses hadoop-aws jar I was getting the following exception :
> Exception in thread "main" java.lang.NoSuchMethodError:
> com.amazonaws.services.s3.transfer.TransferManagerConfiguration.setMultipartUploadThreshold(I)V
> Probable reason could be the fact that aws java sdk expects a long parameter
> for the setMultipartUploadThreshold(long multiPartThreshold) method, but
> hadoop-aws was using a parameter of type int(multiPartThreshold).
> I tried using the downloaded hadoop-aws jar and the build through its maven
> dependency, but in both the cases I encountered the same exception. Although
> I can see private long multiPartThreshold; in hadoop-aws GitHub repo, it's
> not getting reflected in the downloaded jar or in the jar created from maven
> dependency.
> Following lines in the S3AFileSystem class create this difference :
> Build from trunk :
> private long multiPartThreshold;
> this.multiPartThreshold = conf.getLong("fs.s3a.multipart.threshold",
> 2147483647L); => Line 267
> Build through maven dependency :
> private int multiPartThreshold;
> multiPartThreshold = conf.getInt(MIN_MULTIPART_THRESHOLD,
> DEFAULT_MIN_MULTIPART_THRESHOLD); => Line 249
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13043) Add LICENSE.txt entries for bundled javascript dependencies
[ https://issues.apache.org/jira/browse/HADOOP-13043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371814#comment-15371814 ] Vinod Kumar Vavilapalli commented on HADOOP-13043: -- [~andrew.wang], this never made it to branch-2.7 or branch-2.7.3, even though it is in branch-2.6. The merge conflicts seem a little involved, can you please help address them for the 2.7.3 release? Tx. > Add LICENSE.txt entries for bundled javascript dependencies > --- > > Key: HADOOP-13043 > URL: https://issues.apache.org/jira/browse/HADOOP-13043 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.6.4 >Reporter: Andrew Wang >Assignee: Andrew Wang > Fix For: 2.8.0, 2.7.3, 2.6.5 > > Attachments: hadoop-13043.001.patch, hadoop-13043.002.patch > > > None of our bundled javascript dependencies are mentioned in LICENSE.txt. > Let's fix that. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-12420) While trying to access Amazon S3 through hadoop-aws(Spark basically) I was getting Exception in thread "main" java.lang.NoSuchMethodError: com.amazonaws.services
[
https://issues.apache.org/jira/browse/HADOOP-12420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371807#comment-15371807
]
Mitesh edited comment on HADOOP-12420 at 7/11/16 10:35 PM:
---
Hadoop 2.8 still hasn't even RC from what I can tell. What workaround is
everyone using? I'm seeing lots of intermittent timeout/connection issues that
I am fairly sure due to being stuck at hadoop-2.7.2/aws-java-sdk-1.7.4.
was (Author: masterddt):
Hadoop 2.8 still hasn't even RC from what I can tell. What workaround is
everyone using? I'm seeing lots of intermittent timeout/connection issues that
I am fairly sure due to being stuck at aws-java-sdk 1.7.4. I don't really want
to build hadoop though...
> While trying to access Amazon S3 through hadoop-aws(Spark basically) I was
> getting Exception in thread "main" java.lang.NoSuchMethodError:
> com.amazonaws.services.s3.transfer.TransferManagerConfiguration.setMultipartUploadThreshold(I)V
> --
>
> Key: HADOOP-12420
> URL: https://issues.apache.org/jira/browse/HADOOP-12420
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs/s3
>Affects Versions: 2.7.1
>Reporter: Tariq Mohammad
>Assignee: Tariq Mohammad
>Priority: Minor
> Fix For: 2.8.0
>
>
> While trying to access data stored in Amazon S3 through Apache Spark, which
> internally uses hadoop-aws jar I was getting the following exception :
> Exception in thread "main" java.lang.NoSuchMethodError:
> com.amazonaws.services.s3.transfer.TransferManagerConfiguration.setMultipartUploadThreshold(I)V
> Probable reason could be the fact that aws java sdk expects a long parameter
> for the setMultipartUploadThreshold(long multiPartThreshold) method, but
> hadoop-aws was using a parameter of type int(multiPartThreshold).
> I tried using the downloaded hadoop-aws jar and the build through its maven
> dependency, but in both the cases I encountered the same exception. Although
> I can see private long multiPartThreshold; in hadoop-aws GitHub repo, it's
> not getting reflected in the downloaded jar or in the jar created from maven
> dependency.
> Following lines in the S3AFileSystem class create this difference :
> Build from trunk :
> private long multiPartThreshold;
> this.multiPartThreshold = conf.getLong("fs.s3a.multipart.threshold",
> 2147483647L); => Line 267
> Build through maven dependency :
> private int multiPartThreshold;
> multiPartThreshold = conf.getInt(MIN_MULTIPART_THRESHOLD,
> DEFAULT_MIN_MULTIPART_THRESHOLD); => Line 249
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12420) While trying to access Amazon S3 through hadoop-aws(Spark basically) I was getting Exception in thread "main" java.lang.NoSuchMethodError: com.amazonaws.services.s3.t
[
https://issues.apache.org/jira/browse/HADOOP-12420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371807#comment-15371807
]
Mitesh commented on HADOOP-12420:
-
Hadoop 2.8 still hasn't even RC from what I can tell. What workaround is
everyone using? I'm seeing lots of intermittent timeout/connection issues that
I am fairly sure due to being stuck at aws-java-sdk 1.7.4. I don't really want
to build hadoop though...
> While trying to access Amazon S3 through hadoop-aws(Spark basically) I was
> getting Exception in thread "main" java.lang.NoSuchMethodError:
> com.amazonaws.services.s3.transfer.TransferManagerConfiguration.setMultipartUploadThreshold(I)V
> --
>
> Key: HADOOP-12420
> URL: https://issues.apache.org/jira/browse/HADOOP-12420
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs/s3
>Affects Versions: 2.7.1
>Reporter: Tariq Mohammad
>Assignee: Tariq Mohammad
>Priority: Minor
> Fix For: 2.8.0
>
>
> While trying to access data stored in Amazon S3 through Apache Spark, which
> internally uses hadoop-aws jar I was getting the following exception :
> Exception in thread "main" java.lang.NoSuchMethodError:
> com.amazonaws.services.s3.transfer.TransferManagerConfiguration.setMultipartUploadThreshold(I)V
> Probable reason could be the fact that aws java sdk expects a long parameter
> for the setMultipartUploadThreshold(long multiPartThreshold) method, but
> hadoop-aws was using a parameter of type int(multiPartThreshold).
> I tried using the downloaded hadoop-aws jar and the build through its maven
> dependency, but in both the cases I encountered the same exception. Although
> I can see private long multiPartThreshold; in hadoop-aws GitHub repo, it's
> not getting reflected in the downloaded jar or in the jar created from maven
> dependency.
> Following lines in the S3AFileSystem class create this difference :
> Build from trunk :
> private long multiPartThreshold;
> this.multiPartThreshold = conf.getLong("fs.s3a.multipart.threshold",
> 2147483647L); => Line 267
> Build through maven dependency :
> private int multiPartThreshold;
> multiPartThreshold = conf.getInt(MIN_MULTIPART_THRESHOLD,
> DEFAULT_MIN_MULTIPART_THRESHOLD); => Line 249
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12636) Prevent ServiceLoader failure init for unused FileSystems
[ https://issues.apache.org/jira/browse/HADOOP-12636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371802#comment-15371802 ] Vinod Kumar Vavilapalli commented on HADOOP-12636: -- This never made it to branch-2.7.3, even though it was marked so. I just merged it in from branch-2.7. > Prevent ServiceLoader failure init for unused FileSystems > - > > Key: HADOOP-12636 > URL: https://issues.apache.org/jira/browse/HADOOP-12636 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 2.6.2 >Reporter: Inigo Goiri >Assignee: Inigo Goiri > Fix For: 2.7.3 > > Attachments: HADOOP-12636-1.patch, HADOOP-12636-2.patch > > Original Estimate: 1h > Remaining Estimate: 1h > > loadFileSystems() loads all the Filesystems in the path. However, some > Filesystems cannot be initialized. There is no point on failing the startup > because a Filesystem that won't be used. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13297) hadoop-common module depends on hadoop-build-tools module, but the modules are not ordered correctly
[
https://issues.apache.org/jira/browse/HADOOP-13297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371792#comment-15371792
]
Hadoop QA commented on HADOOP-13297:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 11m
45s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
46s{color} | {color:red} root in branch-2.6 failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
6s{color} | {color:green} branch-2.6 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
7s{color} | {color:green} branch-2.6 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
17s{color} | {color:green} branch-2.6 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s{color} | {color:green} branch-2.6 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
8s{color} | {color:green} branch-2.6 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
7s{color} | {color:green} branch-2.6 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
6s{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
7s{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
8s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1131 line(s) that end in whitespace. Use
git apply --whitespace=fix. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
26s{color} | {color:red} The patch 22 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
6s{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
8s{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
7s{color} | {color:green} hadoop-project in the patch passed with JDK
v1.7.0_101. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
27s{color} | {color:red} The patch generated 66 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:44eef0e |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12817274/HADOOP-13297.branch-2.6.v1.patch
|
| JIRA Issue | HADOOP-13297 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit xml |
| uname | Linux 7d71905845c9 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | branch-2.6 / e640944 |
| Default Java | 1.7.0_101 |
| Multi-JDK versions | /usr/lib/jvm/java-8-oracle:1.8.0_91
/usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 |
| mvninstall |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9962/artifact/patchprocess/branch-mvninstall-root.txt
|
| whitespace |
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371790#comment-15371790 ] Allen Wittenauer commented on HADOOP-13344: --- I've been thinking about this off and on all day. IMHO: * I get why slf4j doesn't want multiple bindings in place as a CYA maneuver but it's still annoying. (officially, believe it or not, there is no set order for a JVM to process the classpath. I actually did the research when cleaning up the shell code a few years ago. So the docs are correct in that it might be random, but unofficially... changing that would break the world.) * I think moving it to a place that may be optionally triggered off is a really the only realistic fix here. Munging the classpath to remove it (especially when we tend to use wildcards to speed things up), is just painful. * Since that path is default on, this shouldn't be an incompatible break. Users who turn it off are making the decision to disable it the same way they would a plug-in. This is nearly the same sorts of decisions we give users to turn things on, such as special options to fsck. * Unofficially, this will break things like Apache Big Top which does a lot of strange and weirdo directory munging since it literally re-arranges the distribution as built by Apache Hadoop. But live by the sword, die by the sword. This will effectively turn this into a script + pom patch. As a result, it *might* end up being too big for Jenkins if all of the poms have slf4j listed as a dependency. So we need to keep an eye on that. One thing I forgot to mention: bq. I assume that any documentation that already exists regarding hadoop-config.sh should have this feature documented. branch-2's scripts are completely and totally undocumented. That's not true for trunk. Documentation for the var to turn this on/off should get added to hadoop-env.sh. > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12794) Support additional compression levels for GzipCodec
[
https://issues.apache.org/jira/browse/HADOOP-12794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371784#comment-15371784
]
Vinod Kumar Vavilapalli commented on HADOOP-12794:
--
Even though this landed in 2.7.3, the commit missed the JIRA number. So, the
ticket & commit are disconnected. Here's the SHA, for posterity.
{code}
commit 78a4c348940cd98548650c0a5adb981895cef201
Author: Junping Du
Date: Fri Feb 19 14:21:25 2016 -0800
Support additional compression levels for GzipCodec. Contributed by Ravi
Mutyala.
(cherry picked from commit 18f9b77a321b225677ce23c503b41d21478fc4a7)
(cherry picked from commit e38c2ef6c54521a74cc2b787d78211f629aa07d8)
{code}
> Support additional compression levels for GzipCodec
> ---
>
> Key: HADOOP-12794
> URL: https://issues.apache.org/jira/browse/HADOOP-12794
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
>Affects Versions: 2.7.2
>Reporter: Ravi Mutyala
>Assignee: Ravi Mutyala
> Fix For: 2.8.0, 2.7.3
>
> Attachments: HADOOP-12794.0001.patch, HADOOP-12794.0002.patch,
> HADOOP-12794.0003.patch
>
>
> gzip supports compression levels 1-9. Compression level 4 seems to give best
> compression per CPU time in some of our tests. Right now ZlibCompressor that
> is used by GzipCodec only supports levels 1,9 and six (default).
> Adding all the compression levels that are supported by native ZlibCompressor
> can provide more options to tweak compression levels.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13343) globStatus returns null for file path that exists but is filtered
[ https://issues.apache.org/jira/browse/HADOOP-13343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371778#comment-15371778 ] Jason Lowe commented on HADOOP-13343: - Thanks for chiming in, [~ste...@apache.org] and [~cmccabe]. I filed it without knowing for sure what the "right" fix is. As Colin points out, it may be dangerous to change this behavior since it's been this way for so long. Someone may have intentionally or accidentally relied on the behavior. So I could see this as simply updating the javadoc to explain how globbing and filtering interact without any semantic changes at all. Or we decide to fix the semantics but only in 3.x, etc. My personal preference would be to have it act consistent with globbing patterns failing to match when the base path does exist. And I'm totally fine if we decide this is best done on a major-release boundary to minimize the surprises for users when upgrading. > globStatus returns null for file path that exists but is filtered > - > > Key: HADOOP-13343 > URL: https://issues.apache.org/jira/browse/HADOOP-13343 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 2.7.2 >Reporter: Jason Lowe >Priority: Minor > Attachments: HADOOP-13343.001.patch > > > If a file path without globs is passed to globStatus and the file exists but > the specified input filter suppresses it then globStatus will return null > instead of an empty array. This makes it impossible for the caller to > discern the difference between the file not existing at all vs. being > suppressed by the filter and is inconsistent with the way it handles globs > for an existing dir but fail to match anything within the dir. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13297) hadoop-common module depends on hadoop-build-tools module, but the modules are not ordered correctly
[
https://issues.apache.org/jira/browse/HADOOP-13297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371752#comment-15371752
]
Akira Ajisaka commented on HADOOP-13297:
Make sense, +1. Thanks [~busbey].
> hadoop-common module depends on hadoop-build-tools module, but the modules
> are not ordered correctly
>
>
> Key: HADOOP-13297
> URL: https://issues.apache.org/jira/browse/HADOOP-13297
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 2.7.3, 2.6.5
>Reporter: Akira Ajisaka
>Assignee: Sean Busbey
> Attachments: HADOOP-13297.00.patch, HADOOP-13297.1.patch,
> HADOOP-13297.branch-2.6.00.patch, HADOOP-13297.branch-2.6.v1.patch
>
>
> After HADOOP-12893, we are seeing {{mvn install -DskipTests}} failing in
> branch-2.7, branch-2.7.3, and branch-2.6. This failure is caused by the
> followings
> * hadoop-project module depends on hadoop-build-tools module, but
> hadoop-project module does not declare hadoop-build-tools as its submodule.
> Therefore, hadoop-build-tools is not built before building hadoop-project.
> * hadoop-build-tools pom and jar are not uploaded to the snapshot repository
> (https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-build-tools/)
> The build failure occurs if the *both* of the above conditions are satisfied.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13335) Add an option to suppress the 'use yarn jar' warning or remove it
[ https://issues.apache.org/jira/browse/HADOOP-13335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371741#comment-15371741 ] Allen Wittenauer commented on HADOOP-13335: --- -1 : a) this doesn't solve the user confusion problem. b) there is already a way to disable the messages. > Add an option to suppress the 'use yarn jar' warning or remove it > - > > Key: HADOOP-13335 > URL: https://issues.apache.org/jira/browse/HADOOP-13335 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 2.7.0 >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HADOOP-13335.01.patch, HADOOP-13335.02.patch, > HADOOP-13335.02_branch-2.patch, HADOOP-13335.03.patch, > HADOOP-13335.03_branch-2.patch, HADOOP-13335.04.patch, > HADOOP-13335.05.branch-2.patch, HADOOP-13335.05.trunk.patch > > > https://issues.apache.org/jira/browse/HADOOP-11257 added a 'deprecation' > warning for 'hadoop jar'. > hadoop jar is used for a lot more that starting jobs. As an example - hive > uses it to start all it's services (HiveServer2, the hive client, beeline > etc). > Using 'yarn jar' for to start these services / tools doesn't make a lot of > sense - there's no relation to yarn other than requiring the classpath to > include yarn libraries. > I'd propose reverting the changes where this message is printed if YARN > variables are set (leave it in the help message), or adding a mechanism which > would allow users to suppress this WARNING. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13297) hadoop-common module depends on hadoop-build-tools module, but the modules are not ordered correctly
[
https://issues.apache.org/jira/browse/HADOOP-13297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HADOOP-13297:
-
Target Version/s: 2.8.0, 2.7.3, 2.9.0, 2.6.5, 3.0.0-alpha1 (was: 2.8.0,
2.7.3, 2.6.5)
> hadoop-common module depends on hadoop-build-tools module, but the modules
> are not ordered correctly
>
>
> Key: HADOOP-13297
> URL: https://issues.apache.org/jira/browse/HADOOP-13297
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 2.7.3, 2.6.5
>Reporter: Akira Ajisaka
>Assignee: Sean Busbey
> Attachments: HADOOP-13297.00.patch, HADOOP-13297.1.patch,
> HADOOP-13297.branch-2.6.00.patch, HADOOP-13297.branch-2.6.v1.patch
>
>
> After HADOOP-12893, we are seeing {{mvn install -DskipTests}} failing in
> branch-2.7, branch-2.7.3, and branch-2.6. This failure is caused by the
> followings
> * hadoop-project module depends on hadoop-build-tools module, but
> hadoop-project module does not declare hadoop-build-tools as its submodule.
> Therefore, hadoop-build-tools is not built before building hadoop-project.
> * hadoop-build-tools pom and jar are not uploaded to the snapshot repository
> (https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-build-tools/)
> The build failure occurs if the *both* of the above conditions are satisfied.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13297) hadoop-common module depends on hadoop-build-tools module, but the modules are not ordered correctly
[
https://issues.apache.org/jira/browse/HADOOP-13297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HADOOP-13297:
-
Target Version/s: 2.8.0, 2.7.3, 2.6.5 (was: 2.7.3, 2.6.5)
> hadoop-common module depends on hadoop-build-tools module, but the modules
> are not ordered correctly
>
>
> Key: HADOOP-13297
> URL: https://issues.apache.org/jira/browse/HADOOP-13297
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 2.7.3, 2.6.5
>Reporter: Akira Ajisaka
>Assignee: Sean Busbey
> Attachments: HADOOP-13297.00.patch, HADOOP-13297.1.patch,
> HADOOP-13297.branch-2.6.00.patch, HADOOP-13297.branch-2.6.v1.patch
>
>
> After HADOOP-12893, we are seeing {{mvn install -DskipTests}} failing in
> branch-2.7, branch-2.7.3, and branch-2.6. This failure is caused by the
> followings
> * hadoop-project module depends on hadoop-build-tools module, but
> hadoop-project module does not declare hadoop-build-tools as its submodule.
> Therefore, hadoop-build-tools is not built before building hadoop-project.
> * hadoop-build-tools pom and jar are not uploaded to the snapshot repository
> (https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-build-tools/)
> The build failure occurs if the *both* of the above conditions are satisfied.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13297) hadoop-common module depends on hadoop-build-tools module, but the modules are not ordered correctly
[
https://issues.apache.org/jira/browse/HADOOP-13297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HADOOP-13297:
-
Status: Patch Available (was: In Progress)
The problem is that hadoop-project defines a use of the remote-resources-plugin
that needs the hadoop-build-tool jar but that plugin declaration does not
declare the dependency. You can check this by running {{mvn validate}} to look
at the reactor order. Here you can see that the Hadoop Build Tools module is
way at the end, because Maven doesn’t think anything needs it:
{code}
[INFO]
[INFO] Reactor Build Order:
[INFO]
[INFO] Apache Hadoop Main
[INFO] Apache Hadoop Project POM
[INFO] Apache Hadoop Annotations
[INFO] Apache Hadoop Project Dist POM
[INFO] Apache Hadoop Assemblies
[INFO] Apache Hadoop Maven Plugins
[INFO] Apache Hadoop MiniKDC
[INFO] Apache Hadoop Auth
[INFO] Apache Hadoop Auth Examples
[INFO] Apache Hadoop Common
[INFO] Apache Hadoop NFS
[INFO] Apache Hadoop KMS
[INFO] Apache Hadoop Common Project
[INFO] Apache Hadoop HDFS
[INFO] Apache Hadoop HttpFS
[INFO] Apache Hadoop HDFS BookKeeper Journal
[INFO] Apache Hadoop HDFS-NFS
[INFO] Apache Hadoop HDFS Project
[INFO] hadoop-yarn
[INFO] hadoop-yarn-api
[INFO] hadoop-yarn-common
[INFO] hadoop-yarn-server
[INFO] hadoop-yarn-server-common
[INFO] hadoop-yarn-server-nodemanager
[INFO] hadoop-yarn-server-web-proxy
[INFO] hadoop-yarn-server-applicationhistoryservice
[INFO] hadoop-yarn-server-resourcemanager
[INFO] hadoop-yarn-server-tests
[INFO] hadoop-yarn-client
[INFO] hadoop-yarn-applications
[INFO] hadoop-yarn-applications-distributedshell
[INFO] hadoop-yarn-applications-unmanaged-am-launcher
[INFO] hadoop-yarn-site
[INFO] hadoop-yarn-registry
[INFO] hadoop-yarn-project
[INFO] hadoop-mapreduce-client
[INFO] hadoop-mapreduce-client-core
[INFO] hadoop-mapreduce-client-common
[INFO] hadoop-mapreduce-client-shuffle
[INFO] hadoop-mapreduce-client-app
[INFO] hadoop-mapreduce-client-hs
[INFO] hadoop-mapreduce-client-jobclient
[INFO] hadoop-mapreduce-client-hs-plugins
[INFO] Apache Hadoop MapReduce Examples
[INFO] hadoop-mapreduce
[INFO] Apache Hadoop MapReduce Streaming
[INFO] Apache Hadoop Distributed Copy
[INFO] Apache Hadoop Archives
[INFO] Apache Hadoop Rumen
[INFO] Apache Hadoop Gridmix
[INFO] Apache Hadoop Data Join
[INFO] Apache Hadoop Ant Tasks
[INFO] Apache Hadoop Extras
[INFO] Apache Hadoop Pipes
[INFO] Apache Hadoop OpenStack support
[INFO] Apache Hadoop Amazon Web Services support
[INFO] Apache Hadoop Client
[INFO] Apache Hadoop Mini-Cluster
[INFO] Apache Hadoop Scheduler Load Simulator
[INFO] Apache Hadoop Tools Dist
[INFO] Apache Hadoop Tools
[INFO] Apache Hadoop Distribution
[INFO] Apache Hadoop Build Tools
{code}
after patch on branch-2.6, we can see that the build tools module happens right
away, as it should:
{code}
[INFO]
[INFO] Reactor Build Order:
[INFO]
[INFO] Apache Hadoop Main
[INFO] Apache Hadoop Build Tools
[INFO] Apache Hadoop Project POM
[INFO] Apache Hadoop Annotations
[INFO] Apache Hadoop Project Dist POM
[INFO] Apache Hadoop Assemblies
[INFO] Apache Hadoop Maven Plugins
[INFO] Apache Hadoop MiniKDC
[INFO] Apache Hadoop Auth
[INFO] Apache Hadoop Auth Examples
[INFO] Apache Hadoop Common
[INFO] Apache Hadoop NFS
[INFO] Apache Hadoop KMS
[INFO] Apache Hadoop Common Project
[INFO] Apache Hadoop HDFS
[INFO] Apache Hadoop HttpFS
[INFO] Apache Hadoop HDFS BookKeeper Journal
[INFO] Apache Hadoop HDFS-NFS
[INFO] Apache Hadoop HDFS Project
[INFO] hadoop-yarn
[INFO] hadoop-yarn-api
[INFO] hadoop-yarn-common
[INFO] hadoop-yarn-server
[INFO] hadoop-yarn-server-common
[INFO] hadoop-yarn-server-nodemanager
[INFO] hadoop-yarn-server-web-proxy
[INFO] hadoop-yarn-server-applicationhistoryservice
[INFO] hadoop-yarn-server-resourcemanager
[INFO] hadoop-yarn-server-tests
[INFO] hadoop-yarn-client
[INFO] hadoop-yarn-applications
[INFO] hadoop-yarn-applications-distributedshell
[INFO] hadoop-yarn-applications-unmanaged-am-launcher
[INFO] hadoop-yarn-site
[INFO] hadoop-yarn-registry
[INFO] hadoop-yarn-project
[INFO] hadoop-mapreduce-client
[INFO] hadoop-mapreduce-client-core
[INFO] hadoop-mapreduce-client-common
[INFO] hadoop-mapreduce-client-shuffle
[INFO] hadoop-mapreduce-client-app
[INFO] hadoop-mapreduce-client-hs
[INFO] hadoop-mapreduce-client-jobclient
[INFO] hadoop-mapreduce-client-hs-plugins
[INFO] Apache Hadoop MapReduce Examples
[INFO] hadoop-mapreduce
[INFO] Apache Hadoop MapReduce Streaming
[INFO] Apache Hadoop Distributed Copy
[INFO] Apache Hadoop Archives
[INFO] Apache Hadoop Rumen
[INFO] Apache Hadoop Gridmix
[INFO] Apache Hadoop Data Join
[INFO] Apache Hadoop Ant Tasks
[INFO] Apache Hadoop Extras
[INFO] Apache Hadoop Pipes
[INFO] Apache Hadoop OpenStack support
[INFO] Apache Hadoop Amazon Web Services support
[INFO] Apache Hadoop Client
[jira] [Updated] (HADOOP-13297) hadoop-common module depends on hadoop-build-tools module, but the modules are not ordered correctly
[
https://issues.apache.org/jira/browse/HADOOP-13297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HADOOP-13297:
-
Attachment: HADOOP-13297.branch-2.6.v1.patch
> hadoop-common module depends on hadoop-build-tools module, but the modules
> are not ordered correctly
>
>
> Key: HADOOP-13297
> URL: https://issues.apache.org/jira/browse/HADOOP-13297
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 2.7.3, 2.6.5
>Reporter: Akira Ajisaka
>Assignee: Sean Busbey
> Attachments: HADOOP-13297.00.patch, HADOOP-13297.1.patch,
> HADOOP-13297.branch-2.6.00.patch, HADOOP-13297.branch-2.6.v1.patch
>
>
> After HADOOP-12893, we are seeing {{mvn install -DskipTests}} failing in
> branch-2.7, branch-2.7.3, and branch-2.6. This failure is caused by the
> followings
> * hadoop-project module depends on hadoop-build-tools module, but
> hadoop-project module does not declare hadoop-build-tools as its submodule.
> Therefore, hadoop-build-tools is not built before building hadoop-project.
> * hadoop-build-tools pom and jar are not uploaded to the snapshot repository
> (https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-build-tools/)
> The build failure occurs if the *both* of the above conditions are satisfied.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13297) hadoop-common module depends on hadoop-build-tools module, but the modules are not ordered correctly
[
https://issues.apache.org/jira/browse/HADOOP-13297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HADOOP-13297:
-
Attachment: HADOOP-13297.1.patch
> hadoop-common module depends on hadoop-build-tools module, but the modules
> are not ordered correctly
>
>
> Key: HADOOP-13297
> URL: https://issues.apache.org/jira/browse/HADOOP-13297
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 2.7.3, 2.6.5
>Reporter: Akira Ajisaka
>Assignee: Sean Busbey
> Attachments: HADOOP-13297.00.patch, HADOOP-13297.1.patch,
> HADOOP-13297.branch-2.6.00.patch, HADOOP-13297.branch-2.6.v1.patch
>
>
> After HADOOP-12893, we are seeing {{mvn install -DskipTests}} failing in
> branch-2.7, branch-2.7.3, and branch-2.6. This failure is caused by the
> followings
> * hadoop-project module depends on hadoop-build-tools module, but
> hadoop-project module does not declare hadoop-build-tools as its submodule.
> Therefore, hadoop-build-tools is not built before building hadoop-project.
> * hadoop-build-tools pom and jar are not uploaded to the snapshot repository
> (https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-build-tools/)
> The build failure occurs if the *both* of the above conditions are satisfied.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work started] (HADOOP-13297) hadoop-common module depends on hadoop-build-tools module, but the modules are not ordered correctly
[
https://issues.apache.org/jira/browse/HADOOP-13297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HADOOP-13297 started by Sean Busbey.
> hadoop-common module depends on hadoop-build-tools module, but the modules
> are not ordered correctly
>
>
> Key: HADOOP-13297
> URL: https://issues.apache.org/jira/browse/HADOOP-13297
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 2.7.3, 2.6.5
>Reporter: Akira Ajisaka
>Assignee: Sean Busbey
> Attachments: HADOOP-13297.00.patch, HADOOP-13297.branch-2.6.00.patch
>
>
> After HADOOP-12893, we are seeing {{mvn install -DskipTests}} failing in
> branch-2.7, branch-2.7.3, and branch-2.6. This failure is caused by the
> followings
> * hadoop-project module depends on hadoop-build-tools module, but
> hadoop-project module does not declare hadoop-build-tools as its submodule.
> Therefore, hadoop-build-tools is not built before building hadoop-project.
> * hadoop-build-tools pom and jar are not uploaded to the snapshot repository
> (https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-build-tools/)
> The build failure occurs if the *both* of the above conditions are satisfied.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13297) hadoop-common module depends on hadoop-build-tools module, but the modules are not ordered correctly
[
https://issues.apache.org/jira/browse/HADOOP-13297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HADOOP-13297:
-
Status: Open (was: Patch Available)
> hadoop-common module depends on hadoop-build-tools module, but the modules
> are not ordered correctly
>
>
> Key: HADOOP-13297
> URL: https://issues.apache.org/jira/browse/HADOOP-13297
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 2.7.3, 2.6.5
>Reporter: Akira Ajisaka
>Assignee: Sean Busbey
> Attachments: HADOOP-13297.00.patch, HADOOP-13297.branch-2.6.00.patch
>
>
> After HADOOP-12893, we are seeing {{mvn install -DskipTests}} failing in
> branch-2.7, branch-2.7.3, and branch-2.6. This failure is caused by the
> followings
> * hadoop-project module depends on hadoop-build-tools module, but
> hadoop-project module does not declare hadoop-build-tools as its submodule.
> Therefore, hadoop-build-tools is not built before building hadoop-project.
> * hadoop-build-tools pom and jar are not uploaded to the snapshot repository
> (https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-build-tools/)
> The build failure occurs if the *both* of the above conditions are satisfied.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13240) TestAclCommands.testSetfaclValidations fail
[
https://issues.apache.org/jira/browse/HADOOP-13240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371671#comment-15371671
]
John Zhuge commented on HADOOP-13240:
-
For comparison, I ran the {{setfacl -m "" /path}} on Ubuntu 14.04:
{noformat}
$ cat /etc/issue
Ubuntu 14.04.4 LTS \n \l
$ ls -ld /path
-rw-rw-r--+ 1 root root 0 Jul 11 21:06 /path
$ sudo setfacl -m "" /path
Usage: setfacl [-bkndRLP] { -m|-M|-x|-X ... } file ...
Try `setfacl --help' for more information.
$ sudo setfacl -m "user:ubuntu:rw-" /path
{noformat}
> TestAclCommands.testSetfaclValidations fail
> ---
>
> Key: HADOOP-13240
> URL: https://issues.apache.org/jira/browse/HADOOP-13240
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.4.1, 2.7.1
> Environment: hadoop 2.4.1,as6.5
>Reporter: linbao111
>Assignee: John Zhuge
>Priority: Minor
>
> mvn test -Djava.net.preferIPv4Stack=true -Dlog4j.rootLogger=DEBUG,console
> -Dtest=TestAclCommands#testSetfaclValidations failed with following message:
> ---
> Test set: org.apache.hadoop.fs.shell.TestAclCommands
> ---
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.599 sec <<<
> FAILURE! - in org.apache.hadoop.fs.shell.TestAclCommands
> testSetfaclValidations(org.apache.hadoop.fs.shell.TestAclCommands) Time
> elapsed: 0.534 sec <<< FAILURE!
> java.lang.AssertionError: setfacl should fail ACL spec missing
> at org.junit.Assert.fail(Assert.java:93)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at org.junit.Assert.assertFalse(Assert.java:68)
> at
> org.apache.hadoop.fs.shell.TestAclCommands.testSetfaclValidations(TestAclCommands.java:81)
> i notice from
> HADOOP-10277,hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/AclEntry.java
> code changed
> should
> hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.javabe
> changed to:
> diff --git
> a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.java
>
> b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.java
> index b14cd37..463bfcd
> ---
> a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.java
> +++
> b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.java
> @@ -80,7 +80,7 @@ public void testSetfaclValidations() throws Exception {
> "/path" }));
> assertFalse("setfacl should fail ACL spec missing",
> 0 == runCommand(new String[] { "-setfacl", "-m",
> -"", "/path" }));
> +":", "/path" }));
>}
>
>@Test
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13329) Dockerfile doesn't work on Linux/ppc
[
https://issues.apache.org/jira/browse/HADOOP-13329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371663#comment-15371663
]
Hadoop QA commented on HADOOP-13329:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 11m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 11m
31s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} shellcheck {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 13 new + 75 unchanged - 0 fixed =
88 total (was 75) {color} |
| {color:orange}-0{color} | {color:orange} shelldocs {color} | {color:orange}
0m 10s{color} | {color:orange} The patch generated 8 new + 124 unchanged - 0
fixed = 132 total (was 124) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 3 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m
1s{color} | {color:green} root in the patch passed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
27s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 45m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12817237/HADOOP-13329.3.patch |
| JIRA Issue | HADOOP-13329 |
| Optional Tests | asflicense shellcheck shelldocs mvnsite unit |
| uname | Linux 73b4b80f83a7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 0fd3980 |
| shellcheck | v0.4.4 |
| shellcheck |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9960/artifact/patchprocess/diff-patch-shellcheck.txt
|
| shelldocs |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9960/artifact/patchprocess/diff-patch-shelldocs.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9960/artifact/patchprocess/whitespace-eol.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9960/testReport/ |
| asflicense |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9960/artifact/patchprocess/patch-asflicense-problems.txt
|
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9960/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Dockerfile doesn't work on Linux/ppc
>
>
> Key: HADOOP-13329
> URL: https://issues.apache.org/jira/browse/HADOOP-13329
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha1
>Reporter: Allen Wittenauer
>Assignee: Amir Sanjar
> Attachments: HADOOP-13329.2.patch, HADOOP-13329.3.patch,
> HADOOP-13329.patch
>
>
> We need to rework how the Dockerfile is built to support both Linux/x86 and
> Linux/PowerPC.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13343) globStatus returns null for file path that exists but is filtered
[ https://issues.apache.org/jira/browse/HADOOP-13343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371655#comment-15371655 ] Colin P. McCabe commented on HADOOP-13343: -- bq. This makes it impossible for the caller to discern the difference between the file not existing at all vs. being suppressed by the filter and is inconsistent with the way it handles globs for an existing dir but fail to match anything within the dir. Sorry if this is a nitpick, but it's not impossible for the code to know that the filter is suppressing the file entry. The filter object could set a boolean when it's asked to examine an entry, and the code could check this entry. With that being said, maybe there is case for (re)adopting the behavior proposed here. > globStatus returns null for file path that exists but is filtered > - > > Key: HADOOP-13343 > URL: https://issues.apache.org/jira/browse/HADOOP-13343 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 2.7.2 >Reporter: Jason Lowe >Priority: Minor > Attachments: HADOOP-13343.001.patch > > > If a file path without globs is passed to globStatus and the file exists but > the specified input filter suppresses it then globStatus will return null > instead of an empty array. This makes it impossible for the caller to > discern the difference between the file not existing at all vs. being > suppressed by the filter and is inconsistent with the way it handles globs > for an existing dir but fail to match anything within the dir. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13343) globStatus returns null for file path that exists but is filtered
[ https://issues.apache.org/jira/browse/HADOOP-13343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin P. McCabe updated HADOOP-13343: - Attachment: HADOOP-13343.001.patch It's easy enough to implement this behavior if you want. See patch 001. It seems like the current behavior dates back to at least Hadoop 2.3 though. It was introduced by HADOOP-9817. At this point, changing the behavior to the pre-2.3 behavior may break existing applications. Perhaps this is a change we should make in 3.0? > globStatus returns null for file path that exists but is filtered > - > > Key: HADOOP-13343 > URL: https://issues.apache.org/jira/browse/HADOOP-13343 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 2.7.2 >Reporter: Jason Lowe >Priority: Minor > Attachments: HADOOP-13343.001.patch > > > If a file path without globs is passed to globStatus and the file exists but > the specified input filter suppresses it then globStatus will return null > instead of an empty array. This makes it impossible for the caller to > discern the difference between the file not existing at all vs. being > suppressed by the filter and is inconsistent with the way it handles globs > for an existing dir but fail to match anything within the dir. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13362) DefaultMetricsSystem leaks the source name when a source unregisters
[ https://issues.apache.org/jira/browse/HADOOP-13362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371645#comment-15371645 ] Junping Du commented on HADOOP-13362: - Thanks [~jlowe] for review and commit! > DefaultMetricsSystem leaks the source name when a source unregisters > > > Key: HADOOP-13362 > URL: https://issues.apache.org/jira/browse/HADOOP-13362 > Project: Hadoop Common > Issue Type: Bug > Components: metrics >Affects Versions: 2.7.2 >Reporter: Jason Lowe >Assignee: Junping Du >Priority: Critical > Fix For: 2.7.4 > > Attachments: HADOOP-13362-branch-2.7.patch > > > Ran across a nodemanager that was spending most of its time in GC. Upon > examination of the heap most of the memory was going to the map of names in > org.apache.hadoop.metrics2.lib.UniqueNames. In this case the map had almost > 2 million entries. Looking at a few of the map showed entries like > "ContainerResource_container_e01_1459548490386_8560138_01_002020", > "ContainerResource_container_e01_1459548490386_2378745_01_000410", etc. > Looks like the ContainerMetrics for each container will cause a unique name > to be registered with UniqueNames and the name will never be unregistered. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13240) TestAclCommands.testSetfaclValidations fail
[
https://issues.apache.org/jira/browse/HADOOP-13240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371625#comment-15371625
]
John Zhuge commented on HADOOP-13240:
-
I was able to reproduce the failure by adding a file or directory {{/path}} on
my test host.
[~linbao111], please verify that there is {{/path}} on you test host. Please
remove/rename it temporarily, then run the unit test again.
> TestAclCommands.testSetfaclValidations fail
> ---
>
> Key: HADOOP-13240
> URL: https://issues.apache.org/jira/browse/HADOOP-13240
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.4.1, 2.7.1
> Environment: hadoop 2.4.1,as6.5
>Reporter: linbao111
>Assignee: John Zhuge
>Priority: Minor
>
> mvn test -Djava.net.preferIPv4Stack=true -Dlog4j.rootLogger=DEBUG,console
> -Dtest=TestAclCommands#testSetfaclValidations failed with following message:
> ---
> Test set: org.apache.hadoop.fs.shell.TestAclCommands
> ---
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.599 sec <<<
> FAILURE! - in org.apache.hadoop.fs.shell.TestAclCommands
> testSetfaclValidations(org.apache.hadoop.fs.shell.TestAclCommands) Time
> elapsed: 0.534 sec <<< FAILURE!
> java.lang.AssertionError: setfacl should fail ACL spec missing
> at org.junit.Assert.fail(Assert.java:93)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at org.junit.Assert.assertFalse(Assert.java:68)
> at
> org.apache.hadoop.fs.shell.TestAclCommands.testSetfaclValidations(TestAclCommands.java:81)
> i notice from
> HADOOP-10277,hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/AclEntry.java
> code changed
> should
> hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.javabe
> changed to:
> diff --git
> a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.java
>
> b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.java
> index b14cd37..463bfcd
> ---
> a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.java
> +++
> b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestAclCommands.java
> @@ -80,7 +80,7 @@ public void testSetfaclValidations() throws Exception {
> "/path" }));
> assertFalse("setfacl should fail ACL spec missing",
> 0 == runCommand(new String[] { "-setfacl", "-m",
> -"", "/path" }));
> +":", "/path" }));
>}
>
>@Test
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13335) Add an option to suppress the 'use yarn jar' warning or remove it
[
https://issues.apache.org/jira/browse/HADOOP-13335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371620#comment-15371620
]
Hadoop QA commented on HADOOP-13335:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
28s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
13s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
8s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
40s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 24s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12817244/HADOOP-13335.05.trunk.patch
|
| JIRA Issue | HADOOP-13335 |
| Optional Tests | asflicense mvnsite unit shellcheck shelldocs |
| uname | Linux 7d405b27606a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 0fd3980 |
| shellcheck | v0.4.4 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9961/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9961/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Add an option to suppress the 'use yarn jar' warning or remove it
> -
>
> Key: HADOOP-13335
> URL: https://issues.apache.org/jira/browse/HADOOP-13335
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 2.7.0
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
> Attachments: HADOOP-13335.01.patch, HADOOP-13335.02.patch,
> HADOOP-13335.02_branch-2.patch, HADOOP-13335.03.patch,
> HADOOP-13335.03_branch-2.patch, HADOOP-13335.04.patch,
> HADOOP-13335.05.branch-2.patch, HADOOP-13335.05.trunk.patch
>
>
> https://issues.apache.org/jira/browse/HADOOP-11257 added a 'deprecation'
> warning for 'hadoop jar'.
> hadoop jar is used for a lot more that starting jobs. As an example - hive
> uses it to start all it's services (HiveServer2, the hive client, beeline
> etc).
> Using 'yarn jar' for to start these services / tools doesn't make a lot of
> sense - there's no relation to yarn other than requiring the classpath to
> include yarn libraries.
> I'd propose reverting the changes where this message is printed if YARN
> variables are set (leave it in the help message), or adding a mechanism which
> would allow users to suppress this WARNING.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13362) DefaultMetricsSystem leaks the source name when a source unregisters
[ https://issues.apache.org/jira/browse/HADOOP-13362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated HADOOP-13362: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.7.4 Status: Resolved (was: Patch Available) Thanks, Junping! I committed this to branch-2.7. > DefaultMetricsSystem leaks the source name when a source unregisters > > > Key: HADOOP-13362 > URL: https://issues.apache.org/jira/browse/HADOOP-13362 > Project: Hadoop Common > Issue Type: Bug > Components: metrics >Affects Versions: 2.7.2 >Reporter: Jason Lowe >Assignee: Junping Du >Priority: Critical > Fix For: 2.7.4 > > Attachments: HADOOP-13362-branch-2.7.patch > > > Ran across a nodemanager that was spending most of its time in GC. Upon > examination of the heap most of the memory was going to the map of names in > org.apache.hadoop.metrics2.lib.UniqueNames. In this case the map had almost > 2 million entries. Looking at a few of the map showed entries like > "ContainerResource_container_e01_1459548490386_8560138_01_002020", > "ContainerResource_container_e01_1459548490386_2378745_01_000410", etc. > Looks like the ContainerMetrics for each container will cause a unique name > to be registered with UniqueNames and the name will never be unregistered. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11935) Provide optional native implementation of stat syscall.
[
https://issues.apache.org/jira/browse/HADOOP-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371608#comment-15371608
]
ASF GitHub Bot commented on HADOOP-11935:
-
Github user asfgit closed the pull request at:
https://github.com/apache/hadoop/pull/81
> Provide optional native implementation of stat syscall.
> ---
>
> Key: HADOOP-11935
> URL: https://issues.apache.org/jira/browse/HADOOP-11935
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, native
>Reporter: Chris Nauroth
> Attachments: HADOOP-11935-NativeIO-prelim.patch
>
>
> Currently,
> {{RawLocalFileSystem.DeprecatedRawLocalFileStatus#loadPermissionInfo}} is
> implemented as forking an {{ls}} command and parsing the output. This was
> observed to be a bottleneck in YARN-3491. This issue proposes an optional
> native implementation of a {{stat}} syscall through JNI. We would maintain
> the existing code as a fallback for systems where the native code is not
> available.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13362) DefaultMetricsSystem leaks the source name when a source unregisters
[ https://issues.apache.org/jira/browse/HADOOP-13362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371598#comment-15371598 ] Jason Lowe commented on HADOOP-13362: - +1 lgtm. Verified the test in TestContainerMetrics fails without the change in hadoop-common and passes afterwards. The other test failures appear to be unrelated, and the tests pass for me locally with the patch applied. Committing this. > DefaultMetricsSystem leaks the source name when a source unregisters > > > Key: HADOOP-13362 > URL: https://issues.apache.org/jira/browse/HADOOP-13362 > Project: Hadoop Common > Issue Type: Bug > Components: metrics >Affects Versions: 2.7.2 >Reporter: Jason Lowe >Assignee: Junping Du >Priority: Critical > Attachments: HADOOP-13362-branch-2.7.patch > > > Ran across a nodemanager that was spending most of its time in GC. Upon > examination of the heap most of the memory was going to the map of names in > org.apache.hadoop.metrics2.lib.UniqueNames. In this case the map had almost > 2 million entries. Looking at a few of the map showed entries like > "ContainerResource_container_e01_1459548490386_8560138_01_002020", > "ContainerResource_container_e01_1459548490386_2378745_01_000410", etc. > Looks like the ContainerMetrics for each container will cause a unique name > to be registered with UniqueNames and the name will never be unregistered. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13335) Add an option to suppress the 'use yarn jar' warning or remove it
[ https://issues.apache.org/jira/browse/HADOOP-13335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HADOOP-13335: Status: Patch Available (was: Reopened) > Add an option to suppress the 'use yarn jar' warning or remove it > - > > Key: HADOOP-13335 > URL: https://issues.apache.org/jira/browse/HADOOP-13335 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 2.7.0 >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HADOOP-13335.01.patch, HADOOP-13335.02.patch, > HADOOP-13335.02_branch-2.patch, HADOOP-13335.03.patch, > HADOOP-13335.03_branch-2.patch, HADOOP-13335.04.patch, > HADOOP-13335.05.branch-2.patch, HADOOP-13335.05.trunk.patch > > > https://issues.apache.org/jira/browse/HADOOP-11257 added a 'deprecation' > warning for 'hadoop jar'. > hadoop jar is used for a lot more that starting jobs. As an example - hive > uses it to start all it's services (HiveServer2, the hive client, beeline > etc). > Using 'yarn jar' for to start these services / tools doesn't make a lot of > sense - there's no relation to yarn other than requiring the classpath to > include yarn libraries. > I'd propose reverting the changes where this message is printed if YARN > variables are set (leave it in the help message), or adding a mechanism which > would allow users to suppress this WARNING. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13335) Add an option to suppress the 'use yarn jar' warning or remove it
[ https://issues.apache.org/jira/browse/HADOOP-13335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HADOOP-13335: Attachment: HADOOP-13335.05.trunk.patch Patch that takes option1 and remove the warnings. > Add an option to suppress the 'use yarn jar' warning or remove it > - > > Key: HADOOP-13335 > URL: https://issues.apache.org/jira/browse/HADOOP-13335 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 2.7.0 >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HADOOP-13335.01.patch, HADOOP-13335.02.patch, > HADOOP-13335.02_branch-2.patch, HADOOP-13335.03.patch, > HADOOP-13335.03_branch-2.patch, HADOOP-13335.04.patch, > HADOOP-13335.05.branch-2.patch, HADOOP-13335.05.trunk.patch > > > https://issues.apache.org/jira/browse/HADOOP-11257 added a 'deprecation' > warning for 'hadoop jar'. > hadoop jar is used for a lot more that starting jobs. As an example - hive > uses it to start all it's services (HiveServer2, the hive client, beeline > etc). > Using 'yarn jar' for to start these services / tools doesn't make a lot of > sense - there's no relation to yarn other than requiring the classpath to > include yarn libraries. > I'd propose reverting the changes where this message is printed if YARN > variables are set (leave it in the help message), or adding a mechanism which > would allow users to suppress this WARNING. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13335) Add an option to suppress the 'use yarn jar' warning or remove it
[ https://issues.apache.org/jira/browse/HADOOP-13335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HADOOP-13335: Attachment: HADOOP-13335.05.branch-2.patch > Add an option to suppress the 'use yarn jar' warning or remove it > - > > Key: HADOOP-13335 > URL: https://issues.apache.org/jira/browse/HADOOP-13335 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 2.7.0 >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HADOOP-13335.01.patch, HADOOP-13335.02.patch, > HADOOP-13335.02_branch-2.patch, HADOOP-13335.03.patch, > HADOOP-13335.03_branch-2.patch, HADOOP-13335.04.patch, > HADOOP-13335.05.branch-2.patch > > > https://issues.apache.org/jira/browse/HADOOP-11257 added a 'deprecation' > warning for 'hadoop jar'. > hadoop jar is used for a lot more that starting jobs. As an example - hive > uses it to start all it's services (HiveServer2, the hive client, beeline > etc). > Using 'yarn jar' for to start these services / tools doesn't make a lot of > sense - there's no relation to yarn other than requiring the classpath to > include yarn libraries. > I'd propose reverting the changes where this message is printed if YARN > variables are set (leave it in the help message), or adding a mechanism which > would allow users to suppress this WARNING. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13329) Dockerfile doesn't work on Linux/ppc
[ https://issues.apache.org/jira/browse/HADOOP-13329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amir Sanjar updated HADOOP-13329: - Attachment: HADOOP-13329.3.patch merging patch #1 and #2 > Dockerfile doesn't work on Linux/ppc > > > Key: HADOOP-13329 > URL: https://issues.apache.org/jira/browse/HADOOP-13329 > Project: Hadoop Common > Issue Type: Bug > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Allen Wittenauer >Assignee: Amir Sanjar > Attachments: HADOOP-13329.2.patch, HADOOP-13329.3.patch, > HADOOP-13329.patch > > > We need to rework how the Dockerfile is built to support both Linux/x86 and > Linux/PowerPC. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13290) Appropriate use of generics in FairCallQueue
[
https://issues.apache.org/jira/browse/HADOOP-13290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371523#comment-15371523
]
Hadoop QA commented on HADOOP-13290:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
51s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 23s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 1 new + 34 unchanged - 0 fixed = 35 total (was 34) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
53s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 41m 49s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12816997/HADOOP-13290.001.patch
|
| JIRA Issue | HADOOP-13290 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux fd0de9efec29 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 0fd3980 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9958/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9958/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9958/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Appropriate use of generics in FairCallQueue
>
>
> Key: HADOOP-13290
> URL: https://issues.apache.org/jira/browse/HADOOP-13290
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Affects Versions: 2.6.0
>Reporter:
[jira] [Comment Edited] (HADOOP-13254) Create framework for configurable disk checkers
[
https://issues.apache.org/jira/browse/HADOOP-13254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371473#comment-15371473
]
Yufei Gu edited comment on HADOOP-13254 at 7/11/16 7:49 PM:
Thanks [~rchiang] for reiterating the design goal of {{DiskValidator}}. Agree
to separate {{DiskValidator}} and {{DiskValidatorService}} interfaces. BTW, do
we want to support users' own disk validator plugins in current design as
[~rkanter] mentioned? I don't think it is a good idea to support that since
they can do anything.
I prefer to keep {{DiskValidator}} simple first. If we want a {{Service}}, we
can create a {{Service}} to invoke {{DiskValidator}} just like what
{{LocalDirsHandlerService}} does.
was (Author: yufeigu):
Thanks [~rchiang] for reiterating the design goal of {{DiskValidator}}. Agree
to separate {{DiskValidator}} and {{DiskValidatorService}} interfaces. BTW, do
we want to support users' own disk validator plugins in current design as
[~rkanter] mentioned? I don't think it is a good idea to support that since
they can do anything.
I prefer to keep {{DiskValidator}} simple first. If we want a {{Service}}, we
can create a {{Service}} to invoke {{DiskValidator}} just like
{{LocalDirsHandlerService}}.
> Create framework for configurable disk checkers
> ---
>
> Key: HADOOP-13254
> URL: https://issues.apache.org/jira/browse/HADOOP-13254
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Yufei Gu
>Assignee: Yufei Gu
> Attachments: HADOOP-13254.001.patch, HADOOP-13254.002.patch,
> HADOOP-13254.003.patch, HADOOP-13254.004.patch, HADOOP-13254.005.patch,
> HADOOP-13254.006.patch, HADOOP-13254.007.patch, HADOOP-13254.008.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-13254) Create framework for configurable disk checkers
[
https://issues.apache.org/jira/browse/HADOOP-13254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371473#comment-15371473
]
Yufei Gu edited comment on HADOOP-13254 at 7/11/16 7:41 PM:
Thanks [~rchiang] for reiterating the design goal of {{DiskValidator}}. Agree
to separate {{DiskValidator}} and {{DiskValidatorService}} interfaces. BTW, do
we want to support users' own disk validator plugins in current design as
[~rkanter] mentioned? I don't think it is a good idea to support that since
they can do anything.
I prefer to keep {{DiskValidator}} simple first. If we want a {{Service}}, we
can create a {{Service}} to invoke {{DiskValidator}} just like
{{LocalDirsHandlerService}}.
was (Author: yufeigu):
Thanks [~rchiang] for reiterating the design goal of {{DiskValidator}}. Agree
to separate {{DiskValidator}} and {{DiskValidatorService}} interfaces. BTW, do
we want to support users' own disk validator plugins in current design as
[~rkanter] mentioned? I don't think it is a good idea to support that since
they can do anything.
> Create framework for configurable disk checkers
> ---
>
> Key: HADOOP-13254
> URL: https://issues.apache.org/jira/browse/HADOOP-13254
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Yufei Gu
>Assignee: Yufei Gu
> Attachments: HADOOP-13254.001.patch, HADOOP-13254.002.patch,
> HADOOP-13254.003.patch, HADOOP-13254.004.patch, HADOOP-13254.005.patch,
> HADOOP-13254.006.patch, HADOOP-13254.007.patch, HADOOP-13254.008.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13329) Dockerfile doesn't work on Linux/ppc
[
https://issues.apache.org/jira/browse/HADOOP-13329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371475#comment-15371475
]
Hadoop QA commented on HADOOP-13329:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 3s{color}
| {color:red} HADOOP-13329 does not apply to trunk. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12817220/HADOOP-13329.2.patch |
| JIRA Issue | HADOOP-13329 |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9959/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Dockerfile doesn't work on Linux/ppc
>
>
> Key: HADOOP-13329
> URL: https://issues.apache.org/jira/browse/HADOOP-13329
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha1
>Reporter: Allen Wittenauer
>Assignee: Amir Sanjar
> Attachments: HADOOP-13329.2.patch, HADOOP-13329.patch
>
>
> We need to rework how the Dockerfile is built to support both Linux/x86 and
> Linux/PowerPC.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13254) Create framework for configurable disk checkers
[
https://issues.apache.org/jira/browse/HADOOP-13254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371473#comment-15371473
]
Yufei Gu commented on HADOOP-13254:
---
Thanks [~rchiang] for reiterating the design goal of {{DiskValidator}}. Agree
to separate {{DiskValidator}} and {{DiskValidatorService}} interfaces. BTW, do
we want to support users' own disk validator plugins in current design as
[~rkanter] mentioned? I don't think it is a good idea to support that since
they can do anything.
> Create framework for configurable disk checkers
> ---
>
> Key: HADOOP-13254
> URL: https://issues.apache.org/jira/browse/HADOOP-13254
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Yufei Gu
>Assignee: Yufei Gu
> Attachments: HADOOP-13254.001.patch, HADOOP-13254.002.patch,
> HADOOP-13254.003.patch, HADOOP-13254.004.patch, HADOOP-13254.005.patch,
> HADOOP-13254.006.patch, HADOOP-13254.007.patch, HADOOP-13254.008.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13070) classloading isolation improvements for cleaner and stricter dependencies
[ https://issues.apache.org/jira/browse/HADOOP-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371456#comment-15371456 ] Sean Busbey commented on HADOOP-13070: -- that sounds like a decent first pass sanity check. > classloading isolation improvements for cleaner and stricter dependencies > - > > Key: HADOOP-13070 > URL: https://issues.apache.org/jira/browse/HADOOP-13070 > Project: Hadoop Common > Issue Type: Improvement > Components: util >Reporter: Sangjin Lee >Assignee: Sangjin Lee >Priority: Critical > Attachments: HADOOP-13070.poc.01.patch, Test.java, TestDriver.java, > classloading-improvements-ideas-v.3.pdf, classloading-improvements-ideas.pdf, > classloading-improvements-ideas.v.2.pdf, lib.jar > > > Related to HADOOP-11656, we would like to make a number of improvements in > terms of classloading isolation so that user-code can run safely without > worrying about dependency collisions with the Hadoop dependencies. > By the same token, it should raised the quality of the user code and its > specified classpath so that users get clear signals if they specify incorrect > classpaths. > This will contain a proposal that will include several improvements some of > which may not be backward compatible. As such, it should be targeted to the > next major revision of Hadoop. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13351) TestDFSClientSocketSize buffer size tests are flaky
[
https://issues.apache.org/jira/browse/HADOOP-13351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371427#comment-15371427
]
Mingliang Liu commented on HADOOP-13351:
Thanks for updating the patch, [~fabbri].
Is it better to use try-with-resource for the s1/s2 sockets? My concern is
that, if there is any exception thrown before closing or if there is IOE thrown
when closing {{s1}}, the {{s2}} might not be closed correctly.
Otherwise +1 (non-binding) pending on Jenkins.
> TestDFSClientSocketSize buffer size tests are flaky
> ---
>
> Key: HADOOP-13351
> URL: https://issues.apache.org/jira/browse/HADOOP-13351
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.8.0, 3.0.0-alpha1
>Reporter: Aaron Fabbri
>Assignee: Aaron Fabbri
> Attachments: HADOOP-13551.001.patch, HADOOP-13551.002.patch,
> HADOOP-13551.003.patch
>
>
> {{TestDFSClientSocketSize}} has two tests that assert that a value that was
> set via {{java.net.Socket#setSendBufferSize}} is equal to the value
> subsequently returned by {{java.net.Socket#getSendBufferSize}}.
> These tests are flaky when we run them. The occasionally fail.
> This is expected behavior, actually, because
> {{Socket#setSendBufferSize()}}[is only a
> hint|https://docs.oracle.com/javase/7/docs/api/java/net/Socket.html#setSendBufferSize(int)].
> (Similar to how the underlying libc {{setsockopt(SO_SNDBUF)}} works).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13366) Fix dead link in o.a.h.fs.CommonConfigurationKeysPublic javadoc
[
https://issues.apache.org/jira/browse/HADOOP-13366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371422#comment-15371422
]
Hadoop QA commented on HADOOP-13366:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} hadoop-common-project/hadoop-common: The patch
generated 0 new + 111 unchanged - 69 fixed = 111 total (was 180) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
55s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 44m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12817213/HADOOP-13366-00.patch
|
| JIRA Issue | HADOOP-13366 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 92df74832977 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 0fd3980 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9957/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9957/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Fix dead link in o.a.h.fs.CommonConfigurationKeysPublic javadoc
> ---
>
> Key: HADOOP-13366
> URL: https://issues.apache.org/jira/browse/HADOOP-13366
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: documentation
>Reporter: Rakesh R
>
[jira] [Commented] (HADOOP-13290) Appropriate use of generics in FairCallQueue
[
https://issues.apache.org/jira/browse/HADOOP-13290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371415#comment-15371415
]
Jonathan Hung commented on HADOOP-13290:
[~shv], thanks. The test case is just a sanity check to make sure the MXBean
still works properly, and I noticed there wasn't an existing unit test for it.
> Appropriate use of generics in FairCallQueue
>
>
> Key: HADOOP-13290
> URL: https://issues.apache.org/jira/browse/HADOOP-13290
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Affects Versions: 2.6.0
>Reporter: Konstantin Shvachko
>Assignee: Jonathan Hung
> Labels: newbie++
> Attachments: HADOOP-13290.001.patch
>
>
> # {{BlockingQueue}} is intermittently used with and without generic
> parameters in {{FairCallQueue}} class. Should be parameterized.
> # Same for {{FairCallQueue}}. Should be parameterized. Could be a bit more
> tricky for that one.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13290) Appropriate use of generics in FairCallQueue
[
https://issues.apache.org/jira/browse/HADOOP-13290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hung updated HADOOP-13290:
---
Status: Patch Available (was: Open)
> Appropriate use of generics in FairCallQueue
>
>
> Key: HADOOP-13290
> URL: https://issues.apache.org/jira/browse/HADOOP-13290
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Affects Versions: 2.6.0
>Reporter: Konstantin Shvachko
>Assignee: Jonathan Hung
> Labels: newbie++
> Attachments: HADOOP-13290.001.patch
>
>
> # {{BlockingQueue}} is intermittently used with and without generic
> parameters in {{FairCallQueue}} class. Should be parameterized.
> # Same for {{FairCallQueue}}. Should be parameterized. Could be a bit more
> tricky for that one.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13329) Dockerfile doesn't work on Linux/ppc
[ https://issues.apache.org/jira/browse/HADOOP-13329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amir Sanjar updated HADOOP-13329: - Attachment: HADOOP-13329.2.patch Adding Yetus marker > Dockerfile doesn't work on Linux/ppc > > > Key: HADOOP-13329 > URL: https://issues.apache.org/jira/browse/HADOOP-13329 > Project: Hadoop Common > Issue Type: Bug > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Allen Wittenauer >Assignee: Amir Sanjar > Attachments: HADOOP-13329.2.patch, HADOOP-13329.patch > > > We need to rework how the Dockerfile is built to support both Linux/x86 and > Linux/PowerPC. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13351) TestDFSClientSocketSize buffer size tests are flaky
[
https://issues.apache.org/jira/browse/HADOOP-13351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron Fabbri updated HADOOP-13351:
--
Attachment: HADOOP-13551.003.patch
Attaching v3 patch that also closes the two sockets from the new test.
> TestDFSClientSocketSize buffer size tests are flaky
> ---
>
> Key: HADOOP-13351
> URL: https://issues.apache.org/jira/browse/HADOOP-13351
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.8.0, 3.0.0-alpha1
>Reporter: Aaron Fabbri
>Assignee: Aaron Fabbri
> Attachments: HADOOP-13551.001.patch, HADOOP-13551.002.patch,
> HADOOP-13551.003.patch
>
>
> {{TestDFSClientSocketSize}} has two tests that assert that a value that was
> set via {{java.net.Socket#setSendBufferSize}} is equal to the value
> subsequently returned by {{java.net.Socket#getSendBufferSize}}.
> These tests are flaky when we run them. The occasionally fail.
> This is expected behavior, actually, because
> {{Socket#setSendBufferSize()}}[is only a
> hint|https://docs.oracle.com/javase/7/docs/api/java/net/Socket.html#setSendBufferSize(int)].
> (Similar to how the underlying libc {{setsockopt(SO_SNDBUF)}} works).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371374#comment-15371374 ] Sean Busbey edited comment on HADOOP-13344 at 7/11/16 6:36 PM: --- removing things from the classpath breaks compatibility because downstream folks probably rely on things being there. (though this type of compatibility is not one that hadoop as a project makes promises about ([ref compatibility as of 2.7.2|hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-common/Compatibility.html#Java_Classpath]) was (Author: busbey): removing things from the classpath breaks compatibility because downstream folks probably rely on things being there. (those this type of compatibility is not one that hadoop as a project makes promises about ([ref compatibility as of 2.7.2|hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-common/Compatibility.html#Java_Classpath]) > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371374#comment-15371374 ] Sean Busbey commented on HADOOP-13344: -- removing things from the classpath breaks compatibility because downstream folks probably rely on things being there. (those this type of compatibility is not one that hadoop as a project makes promises about ([ref compatibility as of 2.7.2|hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-common/Compatibility.html#Java_Classpath]) > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13348) delete spurious 'master' branch
[ https://issues.apache.org/jira/browse/HADOOP-13348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-13348. -- Resolution: Fixed Assignee: Andrew Wang With the help of INFRA-12223, the master branch has been deleted. Thanks Sean for reporting this issue! > delete spurious 'master' branch > --- > > Key: HADOOP-13348 > URL: https://issues.apache.org/jira/browse/HADOOP-13348 > Project: Hadoop Common > Issue Type: Task > Components: build >Reporter: Sean Busbey >Assignee: Andrew Wang > > Right now the git repo has a branch named 'master' in addition to our 'trunk' > branch. Since 'master' is the common-place name of the 'most recent' branch > in git repositories, this is misleading to new folks. > It looks like the branch is from ~11 months ago. We should remove it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13290) Appropriate use of generics in FairCallQueue
[
https://issues.apache.org/jira/browse/HADOOP-13290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371360#comment-15371360
]
Konstantin Shvachko commented on HADOOP-13290:
--
[~jhung], the patch looks good. Could you please explain your motivation for
the new test case. Also you should click on the "Submit Patch" button to
trigger Jenkins build.
> Appropriate use of generics in FairCallQueue
>
>
> Key: HADOOP-13290
> URL: https://issues.apache.org/jira/browse/HADOOP-13290
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Affects Versions: 2.6.0
>Reporter: Konstantin Shvachko
>Assignee: Jonathan Hung
> Labels: newbie++
> Attachments: HADOOP-13290.001.patch
>
>
> # {{BlockingQueue}} is intermittently used with and without generic
> parameters in {{FairCallQueue}} class. Should be parameterized.
> # Same for {{FairCallQueue}}. Should be parameterized. Could be a bit more
> tricky for that one.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13254) Create framework for configurable disk checkers
[
https://issues.apache.org/jira/browse/HADOOP-13254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371339#comment-15371339
]
Ray Chiang commented on HADOOP-13254:
-
We're getting dangerously close to wanting a design document for all of this.
Initially, the {{DiskValidator}} functionality was designed for the following:
# Being pluggable
# Gathering one or more write/read style checks under one class
# Saving metrics based on the class
As it was initially designed, it was meant to be a blocking call in the sense
that "return bad if it's not okay to write to the disk".
How defensive should the design be? I'd be okay with separate
{{DiskValidator}} and {{DiskValidatorService}} interfaces.
> Create framework for configurable disk checkers
> ---
>
> Key: HADOOP-13254
> URL: https://issues.apache.org/jira/browse/HADOOP-13254
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Yufei Gu
>Assignee: Yufei Gu
> Attachments: HADOOP-13254.001.patch, HADOOP-13254.002.patch,
> HADOOP-13254.003.patch, HADOOP-13254.004.patch, HADOOP-13254.005.patch,
> HADOOP-13254.006.patch, HADOOP-13254.007.patch, HADOOP-13254.008.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11361) Fix a race condition in MetricsSourceAdapter.updateJmxCache
[
https://issues.apache.org/jira/browse/HADOOP-11361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371286#comment-15371286
]
Yongjun Zhang commented on HADOOP-11361:
Hi [~brahmareddy], [~vinayrpet], [~ozawa],
Thanks for your earlier work/discussion here. Would you please help commenting
on my last comment? Thanks.
--Yongjun
> Fix a race condition in MetricsSourceAdapter.updateJmxCache
> ---
>
> Key: HADOOP-11361
> URL: https://issues.apache.org/jira/browse/HADOOP-11361
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.4.1, 2.5.1, 2.6.0
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Attachments: HADOOP-111361-003.patch, HADOOP-11361-002.patch,
> HADOOP-11361-004.patch, HADOOP-11361.patch, HDFS-7487.patch
>
>
> {noformat}
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateAttrCache(MetricsSourceAdapter.java:247)
> at
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:177)
> at
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getAttribute(MetricsSourceAdapter.java:102)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13329) Dockerfile doesn't work on Linux/ppc
[ https://issues.apache.org/jira/browse/HADOOP-13329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371283#comment-15371283 ] Amir Sanjar commented on HADOOP-13329: -- > - I agree with your statement that Dockerfiles would get moved to x86_64, > ppc64le, ARM64 or whatever. > - I understand your concern, however Ubuntu 15.04 public repository does not > house protobuf v2.5 any more. As an ex-canonical and Ubuntu developer I > assure you https://launchpad.net/ubuntu/+source/protobuf/2.5.0-9ubuntu1/ > launchpad is long lasting. There are repos with protobuf 2.5 for Power, but > security was my main concern. > - Why the custom code for leveldb? leveldb in public maven repository has x86 > specif could that breaks on Power and ARM :( In meanwhile we are working with > Maven community to establish a "secure" maven Power repo. > - Interesting idea, I agree with your statement "We should probably rework > the start-build-env.sh docker call to avoid having two copies of the > hadoop-env-checks script. We could have it copy the correct Dockerfile and > the hadoop-env-checks script to some place under target or patchprocess and > build it from there." > - Regarding Apache Yetus, we are working on it.. > Dockerfile doesn't work on Linux/ppc > > > Key: HADOOP-13329 > URL: https://issues.apache.org/jira/browse/HADOOP-13329 > Project: Hadoop Common > Issue Type: Bug > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Allen Wittenauer >Assignee: Amir Sanjar > Attachments: HADOOP-13329.patch > > > We need to rework how the Dockerfile is built to support both Linux/x86 and > Linux/PowerPC. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13366) Fix dead link in o.a.h.fs.CommonConfigurationKeysPublic javadoc
[
https://issues.apache.org/jira/browse/HADOOP-13366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rakesh R updated HADOOP-13366:
--
Target Version/s: 2.8.0
Status: Patch Available (was: Open)
> Fix dead link in o.a.h.fs.CommonConfigurationKeysPublic javadoc
> ---
>
> Key: HADOOP-13366
> URL: https://issues.apache.org/jira/browse/HADOOP-13366
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: documentation
>Reporter: Rakesh R
>Assignee: Rakesh R
>Priority: Minor
> Attachments: HADOOP-13366-00.patch
>
>
> This jira is to fix the dead link to {{core-default.xml}} in
> [CommonConfigurationKeysPublic|https://hadoop.apache.org/docs/r2.7.2/api/org/apache/hadoop/fs/CommonConfigurationKeysPublic.html]
> javadoc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13366) Fix dead link in o.a.h.fs.CommonConfigurationKeysPublic javadoc
[
https://issues.apache.org/jira/browse/HADOOP-13366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rakesh R updated HADOOP-13366:
--
Attachment: HADOOP-13366-00.patch
> Fix dead link in o.a.h.fs.CommonConfigurationKeysPublic javadoc
> ---
>
> Key: HADOOP-13366
> URL: https://issues.apache.org/jira/browse/HADOOP-13366
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: documentation
>Reporter: Rakesh R
>Assignee: Rakesh R
>Priority: Minor
> Attachments: HADOOP-13366-00.patch
>
>
> This jira is to fix the dead link to {{core-default.xml}} in
> [CommonConfigurationKeysPublic|https://hadoop.apache.org/docs/r2.7.2/api/org/apache/hadoop/fs/CommonConfigurationKeysPublic.html]
> javadoc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13366) Fix dead link in o.a.h.fs.CommonConfigurationKeysPublic javadoc
Rakesh R created HADOOP-13366:
-
Summary: Fix dead link in o.a.h.fs.CommonConfigurationKeysPublic
javadoc
Key: HADOOP-13366
URL: https://issues.apache.org/jira/browse/HADOOP-13366
Project: Hadoop Common
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Priority: Minor
This jira is to fix the dead link to {{core-default.xml}} in
[CommonConfigurationKeysPublic|https://hadoop.apache.org/docs/r2.7.2/api/org/apache/hadoop/fs/CommonConfigurationKeysPublic.html]
javadoc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13254) Create framework for configurable disk checkers
[
https://issues.apache.org/jira/browse/HADOOP-13254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371221#comment-15371221
]
Robert Kanter commented on HADOOP-13254:
Reusing the {{Service}} interface is fine. However, {{DiskValidatorFactory}}
would need to support this by calling the init, start, and stop methods
correctly.
> Create framework for configurable disk checkers
> ---
>
> Key: HADOOP-13254
> URL: https://issues.apache.org/jira/browse/HADOOP-13254
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Yufei Gu
>Assignee: Yufei Gu
> Attachments: HADOOP-13254.001.patch, HADOOP-13254.002.patch,
> HADOOP-13254.003.patch, HADOOP-13254.004.patch, HADOOP-13254.005.patch,
> HADOOP-13254.006.patch, HADOOP-13254.007.patch, HADOOP-13254.008.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13362) DefaultMetricsSystem leaks the source name when a source unregisters
[
https://issues.apache.org/jira/browse/HADOOP-13362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371212#comment-15371212
]
Hadoop QA commented on HADOOP-13362:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 11m
3s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
15s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
48s{color} | {color:green} branch-2.7 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
30s{color} | {color:green} branch-2.7 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
21s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
18s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
29s{color} | {color:green} branch-2.7 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
41s{color} | {color:red} hadoop-common-project/hadoop-common in branch-2.7 has
3 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} branch-2.7 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
20s{color} | {color:green} branch-2.7 passed with JDK v1.7.0_101 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
1s{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
13s{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
1s{color} | {color:red} The patch has 2857 line(s) that end in whitespace. Use
git apply --whitespace=fix. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 1m
33s{color} | {color:red} The patch 124 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 20m 24s{color}
| {color:red} hadoop-common in the patch failed with JDK v1.7.0_101. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 9s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed with JDK
v1.7.0_101. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}127m 17s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_91 Failed junit tests | hadoop.ipc.TestRPC |
[jira] [Comment Edited] (HADOOP-13254) Create framework for configurable disk checkers
[
https://issues.apache.org/jira/browse/HADOOP-13254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371179#comment-15371179
]
Yufei Gu edited comment on HADOOP-13254 at 7/11/16 5:19 PM:
This is another option:
If any implementation of {{DiskValidator}} need threads, it should implement
interface {{org.apache.hadoop.service.Service}}. The {{Servcie}} interface
already provides well designed functions for a service like class. Why not
reuse it and keep the {{DiskValidator}} interface simple. Or, make
{{DiskValidator}} extend interface {{Service}}.
was (Author: yufeigu):
This is another option:
If any implementation of {{DiskValidator}} need threads, it should implement
interface {{org.apache.hadoop.service.Service}}. The {{Servcie}} interface
already provides well designed functions for a service like class. Why not
reuse it and keep the {{DiskValidator}} interface simple.
> Create framework for configurable disk checkers
> ---
>
> Key: HADOOP-13254
> URL: https://issues.apache.org/jira/browse/HADOOP-13254
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Yufei Gu
>Assignee: Yufei Gu
> Attachments: HADOOP-13254.001.patch, HADOOP-13254.002.patch,
> HADOOP-13254.003.patch, HADOOP-13254.004.patch, HADOOP-13254.005.patch,
> HADOOP-13254.006.patch, HADOOP-13254.007.patch, HADOOP-13254.008.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13254) Create framework for configurable disk checkers
[
https://issues.apache.org/jira/browse/HADOOP-13254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371179#comment-15371179
]
Yufei Gu commented on HADOOP-13254:
---
This is another option:
If any implementation of {{DiskValidator}} need threads, it should implement
interface {{org.apache.hadoop.service.Service}}. The {{Servcie}} interface
already provides well designed functions for a service like class. Why not
reuse it and keep the {{DiskValidator}} interface simple.
> Create framework for configurable disk checkers
> ---
>
> Key: HADOOP-13254
> URL: https://issues.apache.org/jira/browse/HADOOP-13254
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Yufei Gu
>Assignee: Yufei Gu
> Attachments: HADOOP-13254.001.patch, HADOOP-13254.002.patch,
> HADOOP-13254.003.patch, HADOOP-13254.004.patch, HADOOP-13254.005.patch,
> HADOOP-13254.006.patch, HADOOP-13254.007.patch, HADOOP-13254.008.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13338) Incompatible change to SortedMapWritable
[
https://issues.apache.org/jira/browse/HADOOP-13338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371170#comment-15371170
]
Siddharth Seth commented on HADOOP-13338:
-
Thanks [~ajisakaa]
> Incompatible change to SortedMapWritable
>
>
> Key: HADOOP-13338
> URL: https://issues.apache.org/jira/browse/HADOOP-13338
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.8.0
>Reporter: Siddharth Seth
>Priority: Critical
>
> Hive does not compile against Hadoop-2.8.0-SNAPSHOT
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile)
> on project hive-contrib: Compilation failure
> [ERROR]
> /Users/sseth/work2/projects/hive/dev/forMvnInstall/contrib/src/java/org/apache/hadoop/hive/contrib/util/typedbytes/TypedBytesWritableOutput.java:[215,70]
> incompatible types: java.lang.Object cannot be converted to
> java.util.Map.Entry
> {code}
> Looks like the change in HADOOP-10465 causes this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13207) Specify FileSystem listStatus and listFiles
[
https://issues.apache.org/jira/browse/HADOOP-13207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-13207:
Attachment: HADOOP-13207-branch-2-008.patch
Patch 008; rebase to branch-2
> Specify FileSystem listStatus and listFiles
> ---
>
> Key: HADOOP-13207
> URL: https://issues.apache.org/jira/browse/HADOOP-13207
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: documentation, fs
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
> Attachments: HADOOP-13207-branch-2-001.patch,
> HADOOP-13207-branch-2-002.patch, HADOOP-13207-branch-2-003.patch,
> HADOOP-13207-branch-2-004.patch, HADOOP-13207-branch-2-005.patch,
> HADOOP-13207-branch-2-006.patch, HADOOP-13207-branch-2-007.patch,
> HADOOP-13207-branch-2-008.patch
>
>
> The many `listStatus`, `listLocatedStatus` and `listFiles` operations have
> not been completely covered in the FS specification. There's lots of implicit
> use of {{listStatus()}} path, but no coverage or tests of the others.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13207) Specify FileSystem listStatus and listFiles
[
https://issues.apache.org/jira/browse/HADOOP-13207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-13207:
Status: Open (was: Patch Available)
> Specify FileSystem listStatus and listFiles
> ---
>
> Key: HADOOP-13207
> URL: https://issues.apache.org/jira/browse/HADOOP-13207
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: documentation, fs
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
> Attachments: HADOOP-13207-branch-2-001.patch,
> HADOOP-13207-branch-2-002.patch, HADOOP-13207-branch-2-003.patch,
> HADOOP-13207-branch-2-004.patch, HADOOP-13207-branch-2-005.patch,
> HADOOP-13207-branch-2-006.patch, HADOOP-13207-branch-2-007.patch,
> HADOOP-13207-branch-2-008.patch
>
>
> The many `listStatus`, `listLocatedStatus` and `listFiles` operations have
> not been completely covered in the FS specification. There's lots of implicit
> use of {{listStatus()}} path, but no coverage or tests of the others.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13208) S3A listFiles(recursive=true) to do a bulk listObjects instead of walking the pseudo-tree of directories
[
https://issues.apache.org/jira/browse/HADOOP-13208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-13208:
Attachment: HADOOP-13208-branch-2-008.patch
Patch 008; rebased to branch-2...some merge problems related to import
declarations in {{S3AFileSystem}} fixed
> S3A listFiles(recursive=true) to do a bulk listObjects instead of walking the
> pseudo-tree of directories
>
>
> Key: HADOOP-13208
> URL: https://issues.apache.org/jira/browse/HADOOP-13208
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Minor
> Attachments: HADOOP-13208-branch-2-001.patch,
> HADOOP-13208-branch-2-007.patch, HADOOP-13208-branch-2-008.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> A major cost in split calculation against object stores turns out be listing
> the directory tree itself. That's because against S3, it takes S3A two HEADs
> and two lists to list the content of any directory path (2 HEADs + 1 list for
> getFileStatus(); the next list to query the contents).
> Listing a directory could be improved slightly by combining the final two
> listings. However, a listing of a directory tree will still be
> O(directories). In contrast, a recursive {{listFiles()}} operation should be
> implementable by a bulk listing of all descendant paths; one List operation
> per thousand descendants.
> As the result of this call is an iterator, the ongoing listing can be
> implemented within the iterator itself
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13208) S3A listFiles(recursive=true) to do a bulk listObjects instead of walking the pseudo-tree of directories
[
https://issues.apache.org/jira/browse/HADOOP-13208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-13208:
Status: Open (was: Patch Available)
> S3A listFiles(recursive=true) to do a bulk listObjects instead of walking the
> pseudo-tree of directories
>
>
> Key: HADOOP-13208
> URL: https://issues.apache.org/jira/browse/HADOOP-13208
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Minor
> Attachments: HADOOP-13208-branch-2-001.patch,
> HADOOP-13208-branch-2-007.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> A major cost in split calculation against object stores turns out be listing
> the directory tree itself. That's because against S3, it takes S3A two HEADs
> and two lists to list the content of any directory path (2 HEADs + 1 list for
> getFileStatus(); the next list to query the contents).
> Listing a directory could be improved slightly by combining the final two
> listings. However, a listing of a directory tree will still be
> O(directories). In contrast, a recursive {{listFiles()}} operation should be
> implementable by a bulk listing of all descendant paths; one List operation
> per thousand descendants.
> As the result of this call is an iterator, the ongoing listing can be
> implemented within the iterator itself
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371160#comment-15371160 ] Thomas Poepping commented on HADOOP-13344: -- I think that's a good solution for trunk. There's less messing about with grep and find and all that. I'm not sure how you guys want to proceed with branch-2. How do you think it might break compatibility? > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-13344: -- Assignee: Thomas Poepping > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371148#comment-15371148 ] Sean Busbey commented on HADOOP-13344: -- yes, for client classpath we have been doing things wrong so far as SLF4J is concerned. (presuming you think of the client classpath as being for using our code as a library to interact with the cluster) > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-13344: - Target Version/s: 2.8.0, 3.0.0-alpha1 (was: 2.8.0) Status: Open (was: Patch Available) moving out of patch-available awaiting implementation for trunk. (I tried to assign things to Thomas, but we still have an enumerated list of contributors and I don't have JIRA admin for this project) > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.7.2, 2.8.0 >Reporter: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371141#comment-15371141 ] Allen Wittenauer commented on HADOOP-13344: --- Reading http://www.slf4j.org/codes.html, I'm inclined to think we're doing this wrong in Hadoop. (Like that never happens...) Maybe we should pull the binding out, put it in a separate dir, then use a var to pull that dir in. I suspect that will break compatibility in 2.x though. (Although if hdfs-client can go in with it's insanity, this isn't much worse.) > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371137#comment-15371137 ] Sean Busbey commented on HADOOP-13344: -- we should do the work for trunk on this jira. once we have a solution that works for trunk we can figure out how much is applicable to branch-2. > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371117#comment-15371117 ] Thomas Poepping commented on HADOOP-13344: -- Would you suggest we keep this open, and wait until the patch is accepted to trunk before coming back to it? > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371115#comment-15371115 ] Thomas Poepping commented on HADOOP-13344: -- The problem is not that the user slf4j binding isn't being used, it's that slf4j will report an error to stderr : "SLF4J: Class path contains multiple SLF4J bindings ... etc" https://github.com/qos-ch/slf4j/blob/c2463021fe0944a03d3fdcc14c3914c86d4a7a4f/slf4j-api/src/main/java/org/slf4j/LoggerFactory.java#L319 This change is to provide users a way around that. We default to *not* using this option (a bit of iffy not logic, I know), so that we don't unintentionally hide this error for people it might negatively affect. To your other points, understood. I'll do some looking into how we can find the offending slf4j binding in the most platform independent way. > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371084#comment-15371084 ] Allen Wittenauer edited comment on HADOOP-13344 at 7/11/16 4:21 PM: Maybe I'm being dense (won't be the first time, won't be the last), but why doesn't just having custom bindings in the classpath first not work? Also: * find -maxdepth, grep -H , -L are all GNU extensions. Those need to get replaced with something POSIXy. * USE_HADOOP_SLF4J_BINDING ... needs to start with HADOOP_. See the https://wiki.apache.org/hadoop/UnixShellScriptProgrammingGuide (grep -s is from SVID, but it looks like POSIX might have adopted it. We'll have to see what the xBSDs are doing.) was (Author: aw): Maybe I'm being dense (won't be the first time, won't be the last), but why doesn't just having custom bindings in the classpath first not work? > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13139) Branch-2: S3a to use thread pool that blocks clients
[ https://issues.apache.org/jira/browse/HADOOP-13139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371093#comment-15371093 ] Steve Loughran commented on HADOOP-13139: - +1. committing to 2.8 and branch-2, not trunk, obviously. > Branch-2: S3a to use thread pool that blocks clients > > > Key: HADOOP-13139 > URL: https://issues.apache.org/jira/browse/HADOOP-13139 > Project: Hadoop Common > Issue Type: Task > Components: fs/s3 >Affects Versions: 2.8.0 >Reporter: Pieter Reuse >Assignee: Pieter Reuse > Attachments: HADOOP-13139-001.patch, HADOOP-13139-branch-2-003.patch, > HADOOP-13139-branch-2-004.patch, HADOOP-13139-branch-2-005.patch, > HADOOP-13139-branch-2-006.patch, HADOOP-13139-branch-2.001.patch, > HADOOP-13139-branch-2.002.patch > > > HADOOP-11684 is accepted into trunk, but was not applied to branch-2. I will > attach a patch applicable to branch-2. > It should be noted in CHANGES-2.8.0.txt that the config parameter > 'fs.s3a.threads.core' has been been removed and the behavior of the > ThreadPool for s3a has been changed. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371084#comment-15371084 ] Allen Wittenauer commented on HADOOP-13344: --- Maybe I'm being dense (won't be the first time, won't be the last), but why doesn't just having custom bindings in the classpath first not work? > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13212) Provide an option to set the socket buffers in S3AFileSystem
[ https://issues.apache.org/jira/browse/HADOOP-13212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371014#comment-15371014 ] Steve Loughran commented on HADOOP-13212: - as usual: which s3a infra did you run the hadoop-aws test suite against? > Provide an option to set the socket buffers in S3AFileSystem > > > Key: HADOOP-13212 > URL: https://issues.apache.org/jira/browse/HADOOP-13212 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HADOOP-13212-branch-2-001.patch > > > It should be possible to provide hints about send/receive buffers to > AmazonS3Client via ClientConfiguration. It would be good to expose these > parameters in S3AFileSystem for perf. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13319) S3A to list InstanceProfileCredentialsProvider after EnvironmentVariableCredentialsProvider
[
https://issues.apache.org/jira/browse/HADOOP-13319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-13319:
Status: Patch Available (was: Open)
> S3A to list InstanceProfileCredentialsProvider after
> EnvironmentVariableCredentialsProvider
> ---
>
> Key: HADOOP-13319
> URL: https://issues.apache.org/jira/browse/HADOOP-13319
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Minor
> Attachments: HADOOP-13252-branch-2-001.patch
>
>
> S3A now has a default list of credential providers to pick up AWS credentials
> from
> The environment variable provider added in HADOOP-12807 should go before the
> {{InstanceProfileCredentialsProvider}} one in the list, as it does a simple
> env var checkup. In contrast {{InstanceProfileCredentialsProvider}} does an
> HTTP request *even when not running on EC2*. It may block for up to 2s to
> await a timeout, and network problems could take longer.
> Checking env vars is a low cost operation that shouldn't have to wait for a
> network timeout before being picked up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
